Pulp

We recently hit an exception on our Pulp installation with pulp_docker:

{
 "exception": null, 
 "task_type": "pulp.server.managers.content.upload.import_uploaded_unit", 
 "_href": "/pulp/api/v2/tasks/7f3e683b-0e29-4c6e-a600-b962932a0bb0/", 
 "task_id": "7f3e683b-0e29-4c6e-a600-b962932a0bb0", 
 "tags": [
  "pulp:repository:redhat-rhel", 
  "pulp:action:import_upload"
 ], 
 "finish_time": "2018-08-01T17:44:10Z", 
 "_ns": "task_status", 
 "start_time": "2018-08-01T17:44:10Z", 
 "traceback": "Traceback (most recent call last):\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 529, in __call__\n    return super(Task, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 107, in __call__\n    return super(PulpTask, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 438, in __protected_call__\n    return self.run(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/managers/content/upload.py\", line 223, in import_uploaded_unit\n    unit_type=unit_type_id, summary=result['summary'], details=result['details']\nPulpCodedException: The importer docker_importer indicated a failed response when uploading docker_manifest_list unit to repository redhat-rhel.\n", 
 "spawned_tasks": [], 
 "progress_report": {}, 
 "state": "error", 
 "worker_name": "reserved_resource_worker-3@brew-pulp-docker02.web.prod.ext.phx2.redhat.com", 
 "result": null, 
 "error": {
  "code": "PLP0047", 
  "data": {
   "unit_type": "docker_manifest_list", 
   "importer_id": "docker_importer", 
   "repo_id": "redhat-rhel", 
   "details": {}, 
   "summary": ""
  }, 
  "description": "The importer docker_importer indicated a failed response when uploading docker_manifest_list unit to repository redhat-rhel.", 
  "sub_errors": []
 }, 
 "_id": {
  "$oid": "5b61f151e95a3bea68e6ef47"
 }, 
 "id": "5b61f151e95a3bea68e6ef47"
}

With formatted traceback:

    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
        R = retval = fun(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 529, in __call__
        return super(Task, self).__call__(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
        return super(PulpTask, self).__call__(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
        return self.run(*args, **kwargs)
      File "/usr/lib/python2.7/site-packages/pulp/server/managers/content/upload.py", line 223, in import_uploaded_unit
        unit_type=unit_type_id, summary=result['summary'], details=result['details']
    PulpCodedException: The importer docker_importer indicated a failed response when uploading docker_manifest_list unit to repository redhat-rhel.

Note that the error reported via the API has empty summary & details, the traceback doesn't point at the source of the error, and the system logs contain no useful information.

Looking through the code, it seems this unhelpful error handling is introduced here in plugins/pulp_docker/plugins/importers/importer.py, upload_unit method:

        try:
            upload_step = upload.UploadStep(repo=repo, file_path=file_path, config=config,
                                            metadata=metadata, type_id=type_id)
            upload_step.process_lifecycle()
        except PulpCodedValidationException:
            raise
        except Exception as e:
            return {'success_flag': False, 'summary': e.message, 'details': {}}

Consider if an exception is raised during UploadStep constructor or process_lifecycle. Then Pulp discards all information about the exception except for the "message" (which can be empty as not all types of exceptions use a message). Thus it's virtually impossible to figure out what failed. Can it please be updated to not discard critical error information?

Actual behavior: when an exception other than PulpCodedValidationException occurs during UploadStep on docker imports, all information about the exception is discarded other than the message.

Expected behavior: when an exception occurs during UploadStep, the exception details are available either through logs or through API, including at least the type of the exception and the traceback.

Checked as of pulp_docker commit fd8d4e767ba9ca0.