Project

Profile

Help

Issue #9551

closed

Error passing upload RPM

Added by wibbit over 2 years ago. Updated over 2 years ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 110
Quarter:

Description

Ticket moved to GitHub: "pulp/pulp_rpm/2317":https://github.com/pulp/pulp_rpm/issues/2317


After uploading an RPM (rpm -qp --all .rpm works fine on the source RPM), I'm greeted with the following error.

{'child_tasks': [],
 'created_resources': [],
 'error': {'description': 'RPM file cannot be parsed for metadata.',
           'traceback': '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py", '
                        'line 272, in _perform_task\n'
                        '    result = func(*args, **kwargs)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulpcore/app/tasks/base.py", '
                        'line 34, in general_create\n'
                        '    serializer.is_valid(raise_exception=True)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/rest_framework/serializers.py", '
                        'line 220, in is_valid\n'
                        '    self._validated_data = '
                        'self.run_validation(self.initial_data)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/rest_framework/serializers.py", '
                        'line 422, in run_validation\n'
                        '    value = self.validate(value)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/serializers/content.py", '
                        'line 104, in validate\n'
                        '    data = self.deferred_validate(data)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulp_rpm/app/serializers/package.py", '
                        'line 240, in deferred_validate\n'
                        '    raise NotAcceptable(detail="RPM file cannot be '
                        'parsed for metadata.")\n'},
 'finished_at': datetime.datetime(2021, 11, 2, 19, 12, 12, 382621, tzinfo=tzutc()),
 'logging_cid': '3c5f6c83815947bfae0e71cb661ec91f',
 'name': 'pulpcore.app.tasks.base.general_create',
 'parent_task': None,
 'progress_reports': [],
 'pulp_created': datetime.datetime(2021, 11, 2, 19, 12, 10, 711408, tzinfo=tzutc()),
 'pulp_href': '/pulp/api/v3/tasks/452c9c7e-1715-4751-9462-0f68a649feb3/',
 'reserved_resources_record': ['/pulp/api/v3/artifacts/58eb9338-26f8-4f55-a0dc-ddca290fa1fb/'],
 'started_at': datetime.datetime(2021, 11, 2, 19, 12, 10, 792329, tzinfo=tzutc()),
 'state': 'failed',
 'task_group': None,
 'worker': None}

Related issues

Related to Pulp - Task #9569: Eliminate all instances when Pulp writes to /tmp and always use the worker working directoryCLOSED - DUPLICATE

Actions
Copied to RPM Support - Backport #9629: Backport #9551 "Error passing upload RPM" to 3.16.2CLOSED - CURRENTRELEASEdalley

Actions
Copied to RPM Support - Backport #9630: Backport #9551 "Error passing upload RPM" to 3.14.9CLOSED - CURRENTRELEASEdalley

Actions
Actions #1

Updated by wibbit over 2 years ago

For reference this RPM is 1.5GB in size.

Actions #2

Updated by ttereshc over 2 years ago

  • Project changed from Pulp to RPM Support
  • Description updated (diff)
Actions #3

Updated by ttereshc over 2 years ago

Pulp needs to show the real issue or at least log the traceback and not hide the root cause.

In the meantime, you can patch your setup to see the full traceback by removing try-except block :

diff --git a/pulp_rpm/app/serializers/package.py b/pulp_rpm/app/serializers/package.py
index 3d7660f7..b6628da2 100644
--- a/pulp_rpm/app/serializers/package.py
+++ b/pulp_rpm/app/serializers/package.py
@@ -234,11 +234,7 @@ class PackageSerializer(SingleArtifactContentUploadSerializer, ContentChecksumSe
         """
         data = super().deferred_validate(data)
         # export META from rpm and prepare dict as saveable format
-        try:
-            new_pkg = _prepare_package(data["artifact"], data["relative_path"])
-        except OSError:
-            raise NotAcceptable(detail="RPM file cannot be parsed for metadata.")
-
+        new_pkg = _prepare_package(data["artifact"], data["relative_path"])
         attrs = {key: new_pkg[key] for key in Package.natural_key_fields()}
         package = Package.objects.filter(**attrs)

Actions #4

Updated by ttereshc over 2 years ago

The issue was running out of disc space but it is unexpected. The suspicion is that the /tmp is used instead of the worker working directory.

Please give this patch a try

diff --git a/pulp_rpm/app/shared_utils.py b/pulp_rpm/app/shared_utils.py
index 8771c1de..fc52aae1 100644
--- a/pulp_rpm/app/shared_utils.py
+++ b/pulp_rpm/app/shared_utils.py
@@ -23,7 +23,7 @@ def _prepare_package(artifact, filename):
         filename: name of file uploaded by user
     """
     artifact_file = storage.open(artifact.file.name)
-    with tempfile.NamedTemporaryFile("wb", suffix=filename) as temp_file:
+    with tempfile.NamedTemporaryFile("wb", dir=".", suffix=filename) as temp_file:
         shutil.copyfileobj(artifact_file, temp_file)
         temp_file.flush()
         cr_pkginfo = createrepo_c.package_from_rpm(temp_file.name)

Actions #5

Updated by ttereshc over 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc
  • Sprint set to Sprint 109
Actions #6

Updated by wibbit over 2 years ago

I can confirm that this was the issue, and by including the dir="." allowed the package to get uploaded and the rest of the process to complete as expected.

Actions #7

Updated by pulpbot over 2 years ago

  • Status changed from ASSIGNED to POST
Actions #8

Updated by ttereshc over 2 years ago

  • Related to Task #9569: Eliminate all instances when Pulp writes to /tmp and always use the worker working directory added
Actions #9

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 109 to Sprint 110

Added by ttereshc over 2 years ago

Revision fdcc1b4b | View on GitHub

Ensure that /tmp is not used but the current worker working directory.

Also log the real traceback to make it possible to troubleshoot the failure.

closes #9551 https://pulp.plan.io/issues/9551

Actions #10

Updated by ttereshc over 2 years ago

  • Status changed from POST to MODIFIED
Actions #11

Updated by dalley over 2 years ago

  • Sprint/Milestone set to 3.17.0
Actions #12

Updated by dalley over 2 years ago

  • Copied to Backport #9629: Backport #9551 "Error passing upload RPM" to 3.16.2 added
Actions #13

Updated by dalley over 2 years ago

  • Copied to Backport #9630: Backport #9551 "Error passing upload RPM" to 3.14.9 added
Actions #14

Updated by fao89 over 2 years ago

  • Description updated (diff)
  • Status changed from MODIFIED to CLOSED - DUPLICATE

Also available in: Atom PDF