Project

Profile

Help

Issue #9551

Error passing upload RPM

Added by wibbit 3 months ago. Updated about 1 month ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 110
Quarter:

Description

Ticket moved to GitHub: "pulp/pulp_rpm/2317":https://github.com/pulp/pulp_rpm/issues/2317


After uploading an RPM (rpm -qp --all .rpm works fine on the source RPM), I'm greeted with the following error.

{'child_tasks': [],
 'created_resources': [],
 'error': {'description': 'RPM file cannot be parsed for metadata.',
           'traceback': '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py", '
                        'line 272, in _perform_task\n'
                        '    result = func(*args, **kwargs)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulpcore/app/tasks/base.py", '
                        'line 34, in general_create\n'
                        '    serializer.is_valid(raise_exception=True)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/rest_framework/serializers.py", '
                        'line 220, in is_valid\n'
                        '    self._validated_data = '
                        'self.run_validation(self.initial_data)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/rest_framework/serializers.py", '
                        'line 422, in run_validation\n'
                        '    value = self.validate(value)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/serializers/content.py", '
                        'line 104, in validate\n'
                        '    data = self.deferred_validate(data)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulp_rpm/app/serializers/package.py", '
                        'line 240, in deferred_validate\n'
                        '    raise NotAcceptable(detail="RPM file cannot be '
                        'parsed for metadata.")\n'},
 'finished_at': datetime.datetime(2021, 11, 2, 19, 12, 12, 382621, tzinfo=tzutc()),
 'logging_cid': '3c5f6c83815947bfae0e71cb661ec91f',
 'name': 'pulpcore.app.tasks.base.general_create',
 'parent_task': None,
 'progress_reports': [],
 'pulp_created': datetime.datetime(2021, 11, 2, 19, 12, 10, 711408, tzinfo=tzutc()),
 'pulp_href': '/pulp/api/v3/tasks/452c9c7e-1715-4751-9462-0f68a649feb3/',
 'reserved_resources_record': ['/pulp/api/v3/artifacts/58eb9338-26f8-4f55-a0dc-ddca290fa1fb/'],
 'started_at': datetime.datetime(2021, 11, 2, 19, 12, 10, 792329, tzinfo=tzutc()),
 'state': 'failed',
 'task_group': None,
 'worker': None}

Related issues

Related to Pulp - Task #9569: Eliminate all instances when Pulp writes to /tmp and always use the worker working directoryCLOSED - DUPLICATE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Copied to RPM Support - Backport #9629: Backport #9551 "Error passing upload RPM" to 3.16.2CLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Copied to RPM Support - Backport #9630: Backport #9551 "Error passing upload RPM" to 3.14.9CLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision fdcc1b4b View on GitHub
Added by ttereshc 2 months ago

Ensure that /tmp is not used but the current worker working directory.

Also log the real traceback to make it possible to troubleshoot the failure.

closes #9551 https://pulp.plan.io/issues/9551

History

#1 Updated by wibbit 3 months ago

For reference this RPM is 1.5GB in size.

#2 Updated by ttereshc 3 months ago

  • Project changed from Pulp to RPM Support
  • Description updated (diff)

#3 Updated by ttereshc 3 months ago

Pulp needs to show the real issue or at least log the traceback and not hide the root cause.

In the meantime, you can patch your setup to see the full traceback by removing try-except block :

diff --git a/pulp_rpm/app/serializers/package.py b/pulp_rpm/app/serializers/package.py
index 3d7660f7..b6628da2 100644
--- a/pulp_rpm/app/serializers/package.py
+++ b/pulp_rpm/app/serializers/package.py
@@ -234,11 +234,7 @@ class PackageSerializer(SingleArtifactContentUploadSerializer, ContentChecksumSe
         """
         data = super().deferred_validate(data)
         # export META from rpm and prepare dict as saveable format
-        try:
-            new_pkg = _prepare_package(data["artifact"], data["relative_path"])
-        except OSError:
-            raise NotAcceptable(detail="RPM file cannot be parsed for metadata.")
-
+        new_pkg = _prepare_package(data["artifact"], data["relative_path"])
         attrs = {key: new_pkg[key] for key in Package.natural_key_fields()}
         package = Package.objects.filter(**attrs)

#4 Updated by ttereshc 2 months ago

The issue was running out of disc space but it is unexpected. The suspicion is that the /tmp is used instead of the worker working directory.

Please give this patch a try

diff --git a/pulp_rpm/app/shared_utils.py b/pulp_rpm/app/shared_utils.py
index 8771c1de..fc52aae1 100644
--- a/pulp_rpm/app/shared_utils.py
+++ b/pulp_rpm/app/shared_utils.py
@@ -23,7 +23,7 @@ def _prepare_package(artifact, filename):
         filename: name of file uploaded by user
     """
     artifact_file = storage.open(artifact.file.name)
-    with tempfile.NamedTemporaryFile("wb", suffix=filename) as temp_file:
+    with tempfile.NamedTemporaryFile("wb", dir=".", suffix=filename) as temp_file:
         shutil.copyfileobj(artifact_file, temp_file)
         temp_file.flush()
         cr_pkginfo = createrepo_c.package_from_rpm(temp_file.name)

#5 Updated by ttereshc 2 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc
  • Sprint set to Sprint 109

#6 Updated by wibbit 2 months ago

I can confirm that this was the issue, and by including the dir="." allowed the package to get uploaded and the rest of the process to complete as expected.

#7 Updated by pulpbot 2 months ago

  • Status changed from ASSIGNED to POST

#8 Updated by ttereshc 2 months ago

  • Related to Task #9569: Eliminate all instances when Pulp writes to /tmp and always use the worker working directory added

#9 Updated by rchan 2 months ago

  • Sprint changed from Sprint 109 to Sprint 110

#10 Updated by ttereshc 2 months ago

  • Status changed from POST to MODIFIED

#11 Updated by dalley 2 months ago

  • Sprint/Milestone set to 3.17.0

#12 Updated by dalley about 2 months ago

  • Copied to Backport #9629: Backport #9551 "Error passing upload RPM" to 3.16.2 added

#13 Updated by dalley about 2 months ago

  • Copied to Backport #9630: Backport #9551 "Error passing upload RPM" to 3.14.9 added

#14 Updated by fao89 about 1 month ago

  • Description updated (diff)
  • Status changed from MODIFIED to CLOSED - DUPLICATE

Also available in: Atom PDF