Issue #9551: Error passing upload RPM - RPM Support - Pulp

Actions

Send by e-mail Copy link

Issue #9551

closed

Error passing upload RPM

Added by wibbit about 3 years ago. Updated almost 3 years ago.

Status:

CLOSED - DUPLICATE

Priority:

Normal

Assignee:

ttereshc

Sprint/Milestone:

3.17.0

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Version:

Platform Release:

OS:

Triaged:

Groomed:

Sprint Candidate:

Tags:

Sprint:

Sprint 110

Quarter:

Description

Ticket moved to GitHub: "pulp/pulp_rpm/2317":https://github.com/pulp/pulp_rpm/issues/2317

After uploading an RPM (rpm -qp --all .rpm works fine on the source RPM), I'm greeted with the following error.

{'child_tasks': [],
 'created_resources': [],
 'error': {'description': 'RPM file cannot be parsed for metadata.',
           'traceback': '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py", '
                        'line 272, in _perform_task\n'
                        '    result = func(*args, **kwargs)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulpcore/app/tasks/base.py", '
                        'line 34, in general_create\n'
                        '    serializer.is_valid(raise_exception=True)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/rest_framework/serializers.py", '
                        'line 220, in is_valid\n'
                        '    self._validated_data = '
                        'self.run_validation(self.initial_data)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/rest_framework/serializers.py", '
                        'line 422, in run_validation\n'
                        '    value = self.validate(value)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/serializers/content.py", '
                        'line 104, in validate\n'
                        '    data = self.deferred_validate(data)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulp_rpm/app/serializers/package.py", '
                        'line 240, in deferred_validate\n'
                        '    raise NotAcceptable(detail="RPM file cannot be '
                        'parsed for metadata.")\n'},
 'finished_at': datetime.datetime(2021, 11, 2, 19, 12, 12, 382621, tzinfo=tzutc()),
 'logging_cid': '3c5f6c83815947bfae0e71cb661ec91f',
 'name': 'pulpcore.app.tasks.base.general_create',
 'parent_task': None,
 'progress_reports': [],
 'pulp_created': datetime.datetime(2021, 11, 2, 19, 12, 10, 711408, tzinfo=tzutc()),
 'pulp_href': '/pulp/api/v3/tasks/452c9c7e-1715-4751-9462-0f68a649feb3/',
 'reserved_resources_record': ['/pulp/api/v3/artifacts/58eb9338-26f8-4f55-a0dc-ddca290fa1fb/'],
 'started_at': datetime.datetime(2021, 11, 2, 19, 12, 10, 792329, tzinfo=tzutc()),
 'state': 'failed',
 'task_group': None,
 'worker': None}

Related issues

Actions

Copy link

Updated by wibbit about 3 years ago

For reference this RPM is 1.5GB in size.

Actions

Copy link

Updated by ttereshc about 3 years ago

Project changed from Pulp to RPM Support
Description updated (diff)

Actions

Copy link

Updated by ttereshc about 3 years ago

Pulp needs to show the real issue or at least log the traceback and not hide the root cause.

In the meantime, you can patch your setup to see the full traceback by removing try-except block :

diff --git a/pulp_rpm/app/serializers/package.py b/pulp_rpm/app/serializers/package.py
index 3d7660f7..b6628da2 100644
--- a/pulp_rpm/app/serializers/package.py
+++ b/pulp_rpm/app/serializers/package.py
@@ -234,11 +234,7 @@ class PackageSerializer(SingleArtifactContentUploadSerializer, ContentChecksumSe
         """
         data = super().deferred_validate(data)
         # export META from rpm and prepare dict as saveable format
-        try:
-            new_pkg = _prepare_package(data["artifact"], data["relative_path"])
-        except OSError:
-            raise NotAcceptable(detail="RPM file cannot be parsed for metadata.")
-
+        new_pkg = _prepare_package(data["artifact"], data["relative_path"])
         attrs = {key: new_pkg[key] for key in Package.natural_key_fields()}
         package = Package.objects.filter(**attrs)

Actions

Copy link

Updated by ttereshc about 3 years ago

The issue was running out of disc space but it is unexpected. The suspicion is that the /tmp is used instead of the worker working directory.

Please give this patch a try

diff --git a/pulp_rpm/app/shared_utils.py b/pulp_rpm/app/shared_utils.py
index 8771c1de..fc52aae1 100644
--- a/pulp_rpm/app/shared_utils.py
+++ b/pulp_rpm/app/shared_utils.py
@@ -23,7 +23,7 @@ def _prepare_package(artifact, filename):
         filename: name of file uploaded by user
     """
     artifact_file = storage.open(artifact.file.name)
-    with tempfile.NamedTemporaryFile("wb", suffix=filename) as temp_file:
+    with tempfile.NamedTemporaryFile("wb", dir=".", suffix=filename) as temp_file:
         shutil.copyfileobj(artifact_file, temp_file)
         temp_file.flush()
         cr_pkginfo = createrepo_c.package_from_rpm(temp_file.name)

Actions

Copy link

Updated by ttereshc about 3 years ago

Status changed from NEW to ASSIGNED
Assignee set to ttereshc
Sprint set to Sprint 109

Actions

Copy link

Updated by wibbit about 3 years ago

I can confirm that this was the issue, and by including the dir="." allowed the package to get uploaded and the rest of the process to complete as expected.

Actions

Copy link