Actions
Issue #9550
closedEnsure correct tmp location is used when reconstructing chunked uploads
Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:
Description
Currently when uploading content via the chunked uploads, when the file is reconstructed it uses the generic /tmp location as opposed to the are specified by either WORKING_DIRECTORY or FILE_UPLOAD_DIRECTORY.
https://github.com/pulp/pulpcore/blob/master/pulpcore/app/tasks/upload.py#L29
Related issues
Updated by dkliban@redhat.com about 3 years ago
diff --git a/pulpcore/app/tasks/upload.py b/pulpcore/app/tasks/upload.py
index 89d876816..dbd140f90 100644
--- a/pulpcore/app/tasks/upload.py
+++ b/pulpcore/app/tasks/upload.py
@@ -26,7 +26,7 @@ def commit(upload_id, sha256):
return
chunks = models.UploadChunk.objects.filter(upload=upload).order_by("offset")
- with NamedTemporaryFile("ab") as temp_file:
+ with NamedTemporaryFile(mode="ab", dir=".", delete=False) as temp_file:
for chunk in chunks:
temp_file.write(chunk.file.read())
temp_file.flush()
That patch got him as far as trying to create an RPM from a file but then pulp_rpm plugin complained about the file.
[dfurlong@snjlv-sysengutil01 ~]$ time /opt/systems/bin/syseng-pulpctlv3 --username=admin content-upload --type=rpm --platform=rhel7 --repo=acme --release=trunk --path=/home/dfurlong/acme-trino-363-1.el7.x86_64.rpm
2021-11-02 15:12:44|ERROR |Server: ncc1r8.us.acme.net, Task: /pulp/api/v3/tasks/452c9c7e-1715-4751-9462-0f68a649feb3/ is in a failed state. With message: RPM file cannot be parsed for metadata.
Traceback (most recent call last):
File "/opt/systems/bin/syseng-pulpctlv3", line 11, in <module>
controller.run()
File "/home/dfurlong/systems/syseng_code/python/site_syseng/prog/pulpctl/pulpctl.py", line 1107, in run
self.args.action(self)
File "/home/dfurlong/systems/syseng_code/python/site_syseng/prog/pulpctl/pulpctl.py", line 437, in <lambda>
content_upload_subparser.set_defaults(action=lambda s: cls.do_content_upload(s))
File "/home/dfurlong/systems/syseng_code/python/site_syseng/prog/pulpctl/pulpctl.py", line 1601, in do_content_upload
self.pulp3_servers.primary.content_create(artifacts, self.args.type)
File "/home/dfurlong/systems/syseng_code/python/site_syseng/prog/pulpctl/pulpserver.py", line 204, in content_create
href = self.tasks_retrieve_create_resource_7708(task_href=task, type=type)
File "/home/dfurlong/systems/syseng_code/python/site_syseng/prog/pulpctl/pulpserver.py", line 671, in tasks_retrieve_create_resource_7708
raise ValueError(
ValueError: ('task_href: %s, has failed with error %f', '/pulp/api/v3/tasks/452c9c7e-1715-4751-9462-0f68a649feb3/', {'traceback': ' File "/opt/acme/lib/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py", line 272, in _perform_task\n result = func(*args, **kwargs)\n File "/opt/acme/lib/python3.8/site-packages/pulpcore/app/tasks/base.py", line 34, in general_create\n serializer.is_valid(raise_exception=True)\n File "/opt/acme/lib/python3.8/site-packages/rest_framework/serializers.py", line 220, in is_valid\n self._validated_data = self.run_validation(self.initial_data)\n File "/opt/acme/lib/python3.8/site-packages/rest_framework/serializers.py", line 422, in run_validation\n value = self.validate(value)\n File "/opt/acme/lib/python3.8/site-packages/pulpcore/plugin/serializers/content.py", line 104, in validate\n data = self.deferred_validate(data)\n File "/opt/acme/lib/python3.8/site-packages/pulp_rpm/app/serializers/package.py", line 240, in deferred_validate\n raise NotAcceptable(detail="RPM file cannot be parsed for metadata.")\n', 'description': 'RPM file cannot be parsed for metadata.'})
Updated by dkliban@redhat.com about 3 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to dkliban@redhat.com
Updated by pulpbot about 3 years ago
- Status changed from ASSIGNED to POST
Added by dkliban@redhat.com about 3 years ago
Updated by dkliban@redhat.com about 3 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulpcore|b7210102dc02a055d79c4cef4a5e86ccfbbdeb30.
Updated by dkliban@redhat.com about 3 years ago
- Copied to Backport #9556: Backport 9550 to 3.14: Ensure correct tmp location is used when reconstructing chunked uploads added
Updated by ttereshc about 3 years ago
- Related to Task #9569: Eliminate all instances when Pulp writes to /tmp and always use the worker working directory added
Updated by pulpbot almost 3 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions
Assembles chunked_uploads in workers working directory
fixes: #9550 https://pulp.plan.io/issues/9550