Project

Profile

Help

Issue #3072

Too generic "The importer yum_importer indicated a failed response"

Added by akofink almost 3 years ago. Updated over 1 year ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.13.2
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

I believe this is a generic error message. It would be extremely helpful to tell users what actually went wrong. For example, when uploading a non-rpm unit to an rpm repository, I get the following:

PLP0047: The importer yum_importer indicated a failed response when uploading rpm unit to repository 57b86eaa-0e42-469e-a64b-b7a6a69d711f.

Why not something like "Invalid file type" or something more useful to end users.


Related issues

Related to RPM Support - Issue #2543: RPM Importer swallows exception when one is raised during uploadCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to Pulp - Issue #3090: Uploading an invalid rpm produces an error message: "unexpected error occurred importing uploaded file: 'primary'"CLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

History

#1 Updated by daviddavis almost 3 years ago

@akofink, thanks for the bug report. This error is typically caused by a raised exception that isn't being properly caught. While we've fixed many of these over the years, there are probably still more out there. Do you have some steps to reproduce?

#2 Updated by akofink almost 3 years ago

daviddavis Yes! In a (custom) package repository, simply try to upload a non-rpm. Here's the katello issue that inspired me to report this bug. http://projects.theforeman.org/issues/21288

#3 Updated by daviddavis almost 3 years ago

  • Related to Issue #2543: RPM Importer swallows exception when one is raised during upload added

#4 Updated by daviddavis almost 3 years ago

Here's what the task looks like from the API:

    {
        "_href": "/pulp/api/v2/tasks/6057c7c2-4ec2-4357-907f-887480f6d20d/",
        "_id": {
            "$oid": "59ea1b8a7e8c3485dcf86e86"
        },
        "_ns": "task_status",
        "error": {
            "code": "PLP0047",
            "data": {
                "details": {
                    "errors": [
                        "unexpected error occurred importing uploaded file: error reading package header"
                    ]
                },
                "importer_id": "yum_importer",
                "repo_id": "test",
                "summary": "",
                "unit_type": "rpm"
            },
            "description": "The importer yum_importer indicated a failed response when uploading rpm unit to repository test.",
            "sub_errors": []
        },
        "exception": null,
        "finish_time": "2017-10-20T15:51:38Z",
        "id": "59ea1b8a7e8c3485dcf86e86",
        "progress_report": {},
        "result": null,
        "spawned_tasks": [],
        "start_time": "2017-10-20T15:51:38Z",
        "state": "error",
        "tags": [
            "pulp:repository:test",
            "pulp:action:import_upload"
        ],
        "task_id": "6057c7c2-4ec2-4357-907f-887480f6d20d",
        "task_type": "pulp.server.managers.content.upload.import_uploaded_unit",
        "traceback": "Traceback (most recent call last):\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/home/vagrant
/devel/pulp/server/pulp/server/async/tasks.py\", line 529, in __call__\n    return super(Task, self).__call__(*args, **kwargs)\n  File \"/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py\", line 107, 
in __call__\n    return super(PulpTask, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 438, in __protected_call__\n    return self.run(*args, **kwargs)\n
  File \"/home/vagrant/devel/pulp/server/pulp/server/managers/content/upload.py\", line 223, in import_uploaded_unit\n    unit_type=unit_type_id, summary=result['summary'], details=result['details']\nPulpCoded
Exception: The importer yum_importer indicated a failed response when uploading rpm unit to repository test.\n",
        "worker_name": "reserved_resource_worker-0@pulp2.dev"
    }

Looking at the katello code[0], it's pulling from task[:error][:description] but the most important info is in task[:error][:data][:details][:errors][0].

It seems like a terrible design to me that there are so many places in the task response that could contain error info (e.g. task[:error][:description], task[:exception], task[:error][:data][:details][:errors], etc).

Part of me feels like Pulp should either replace or append the error message from task[:error][:data][:details][:errors] to task[:error][:description]. However, I worry that other places in Pulp might be writing error info to task[:error][:data][:details][:errors].

[0] https://github.com/Katello/katello/blob/master/app/lib/katello/errors.rb#L111

#5 Updated by daviddavis almost 3 years ago

  • Related to deleted (Issue #2543: RPM Importer swallows exception when one is raised during upload)

#6 Updated by daviddavis almost 3 years ago

  • Related to Issue #3090: Uploading an invalid rpm produces an error message: "unexpected error occurred importing uploaded file: 'primary'" added

#7 Updated by daviddavis almost 3 years ago

  • Related to deleted (Issue #3090: Uploading an invalid rpm produces an error message: "unexpected error occurred importing uploaded file: 'primary'")

#8 Updated by daviddavis almost 3 years ago

  • Related to Issue #2543: RPM Importer swallows exception when one is raised during upload added

#9 Updated by daviddavis almost 3 years ago

  • Related to Issue #3090: Uploading an invalid rpm produces an error message: "unexpected error occurred importing uploaded file: 'primary'" added

#10 Updated by bmbouter almost 3 years ago

This is a pervasive problem in Pulp2. The error reporting is hugely inconsistent. I don't think we can make it consistent because we would be removing or modifying parts of the json response which we can't do because of semver. daviddavis, it's even more than a terrible design, it's actually no design. Consider that other plugins actually do it differently even than the response you posted!

The good news is that Pulp3 fully resolves this issue (i feel) with structured error reporting and a structured db.

For Pulp2, since we can't "replace" due to semver, we could "append", but I don't think it's worth it. It won't fully response the inconsistency problem (other task types, other plugins, etc) and the data is already in the json response so it's not going to practically benefit the users.

The one thing that we have done a lot of in this situation before is to take a specific situation that emits an "Importer indicated a failed response", catch that specific error case and instead emit an existing or new PulpCodedException instead.

How the bug is written now, I think it has to be WONTFIX. It's a bug, but it's not one we can fix. :(

@akofink would you be interested in rewriting the issue, or open a new one requesting a PulpCodedException for a specific situation [0] then that would be good.

[0]: http://projects.theforeman.org/issues/21288

#11 Updated by daviddavis almost 3 years ago

bmbouter, thanks for your response. Having the code catch the exception and raise a PulpCodedException sounds like a good idea. However I guess my concern would be: might there be other places in the code that are also logging error info to task[:error][:data][:details][:errors]? If so, I think Katello needs be checking task[:error][:data][:details][:errors] for error info as well.

#12 Updated by bmbouter almost 3 years ago

daviddavis, I agree. A PulpCodedException for one situation will be an improvement, but it won't resolve the inconsistency issue for Katello. I think Katello still needs to check task[:error][:data][:details][:errors] for error info as well.

Since we won't be able to resolve this for Katello (as it was originally described in this ticket) I believe the only thing we can do is close WONTFIX and let Pulp3 resolve this problem.

#13 Updated by daviddavis almost 3 years ago

  • Status changed from NEW to CLOSED - WONTFIX

There is an issue to fix the error message for uploads already:

https://pulp.plan.io/issues/3090

Looks like there's a regression that was introduced a couple months ago there. I added a note about hopefully raising a PulpCodedException.

I've notified Katello that they need to check task[:error][:data][:details][:errors] for error info.

#14 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF