Project

Profile

Help

Issue #2422

closed

pulp-admin 2.7 does not work with pulp-server 2.10 (pulp-rpm to be more specific)

Added by mihai.ibanescu@gmail.com over 7 years ago. Updated about 5 years ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.10.0
Platform Release:
OS:
CentOS 7
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

I think plugin developers should think more about backwards compatibility with clients.

In this case, I was getting an error:

Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608) unexpected error occurred importing uploaded file: The field 'checksum_type' does not exist on the document 'RPM'
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608) Traceback (most recent call last):
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/upload.py", line 113, in upload
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608)     handlers[type_id](repo, type_id, unit_key, metadata, file_path, conduit, config)
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/upload.py", line 403, in _handle_package
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608)     unit = model_class(**rpm_data)
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py", line 735, in __init__
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608)     super(RpmBase, self).__init__(*args, **kwargs)
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py", line 153, in __init__
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608)     super(NonMetadataPackage, self).__init__(*args, **kwargs)
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608)   File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 55, in __init__
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608)     super(AutoRetryDocument, self).__init__(*args, **kwargs)
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608)   File "/usr/lib/python2.7/site-packages/mongoengine/base/document.py", line 80, in __init__
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608)     raise FieldDoesNotExist(msg)
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp_rpm.plugins.importers.yum.upload:ERROR: (14734-80608) FieldDoesNotExist: The field 'checksum_type' does not exist on the document 'RPM'
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp.server.managers.content.upload:ERROR: (14734-80608) Error from the importer while importing uploaded unit to repository [yocto-x86_64]
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp.server.managers.content.upload:ERROR: (14734-80608) Traceback (most recent call last):
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp.server.managers.content.upload:ERROR: (14734-80608)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/content/upload.py", line 223, in import_uploaded_unit
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp.server.managers.content.upload:ERROR: (14734-80608)     unit_type=unit_type_id, summary=result['summary'], details=result['details']
Nov 14 12:00:40 pulp-dev-win.unx.sas.com pulp[14734]: pulp.server.managers.content.upload:ERROR: (14734-80608) PulpCodedException: The importer yum_importer indicated a failed response when uploading rpm unit to repository yocto-x86_64.

Which didn't make any sense at first blush.

What really happens: the old client is supplying the old-style fields (checksum_type). The pulp_rpm plugin attempts to take all the client-supplied metadata fields and pass them into the Mongo model's constructor.

In an ideal world, when a field is removed or renamed, the plugin should remove or rewrite the user-supplied metadata in the import call, so that old clients continue to work against new servers. I don't think it's overly taxing for the plugin developer to do this.

Actions #1

Updated by mihai.ibanescu@gmail.com over 7 years ago

As a suggestion:

https://github.com/pulp/pulp_rpm/blob/2.10-release/plugins/pulp_rpm/plugins/importers/yum/upload.py#L398

That's one of the problem - it should at least filter it though self._fields.

It is also where checksum_type could be rewritten as checkumtype since the plugin developer knew it has changed.

Actions #2

Updated by bmbouter over 7 years ago

  • Description updated (diff)

The API is governed by semver so what you've identified is an unexpected backwards incompatible change. Semver was adopted after the Pulp 2.4 release.

In theory features offered in older clients (2.4+) should work with newer Pulp server 2.4+ instances. In changing field names, care was taken to prevent regressions but apparently some aspect of this endpoint did regress.

To help us resolve this issue can you provide a curl or httpie reproducer. Or at a minimum could you paste the `-vv` output of pulp-admin 2.7 during that upload call. It would make the issue a lot easier to resolve than trying to bring back a vintage pulp-admin in a dev environment.

Actions #3

Updated by mihai.ibanescu@gmail.com over 7 years ago

2016-11-15 12:43:02,455 - DEBUG - sending POST request to /pulp/api/v2/repositories/yocto-x86_64/actions/import_upload/
2016-11-15 12:43:02,601 - INFO - POST request to /pulp/api/v2/repositories/yocto-x86_64/actions/import_upload/ with parameters {"override_config": {}, "unit_type_id": "rpm", "upload_id": "c9ed44ea-65da-4764-b74b-f8061d341546", "unit_key": {}, "unit_metadata": {"checksum_type": null}}
2016-11-15 12:43:02,601 - INFO - Response status : 202 

2016-11-15 12:43:02,601 - INFO - Response body :
 {
  "spawned_tasks": [
    {
      "_href": "/pulp/api/v2/tasks/21a81a9f-f662-4908-84e0-30686fd6d4c2/", 
      "task_id": "21a81a9f-f662-4908-84e0-30686fd6d4c2"
    }
  ], 
  "result": null, 
  "error": null
}

This command may be exited via ctrl+c without affecting the request.

[-]
Running...
2016-11-15 12:43:03,602 - DEBUG - sending GET request to /pulp/api/v2/tasks/21a81a9f-f662-4908-84e0-30686fd6d4c2/
2016-11-15 12:43:03,739 - INFO - GET request to /pulp/api/v2/tasks/21a81a9f-f662-4908-84e0-30686fd6d4c2/ with parameters None
2016-11-15 12:43:03,739 - INFO - Response status : 200 

2016-11-15 12:43:03,739 - INFO - Response body :
 {
  "exception": null, 
  "task_type": "pulp.server.managers.content.upload.import_uploaded_unit", 
  "_href": "/pulp/api/v2/tasks/21a81a9f-f662-4908-84e0-30686fd6d4c2/", 
  "task_id": "21a81a9f-f662-4908-84e0-30686fd6d4c2", 
  "tags": [
    "pulp:repository:yocto-x86_64", 
    "pulp:action:import_upload"
  ], 
  "finish_time": "2016-11-15T17:57:59Z", 
  "_ns": "task_status", 
  "start_time": "2016-11-15T17:57:59Z", 
  "traceback": "Traceback (most recent call last):\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 488, in __call__\n    return super(Task, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 103, in __call__\n    return super(PulpTask, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 437, in __protected_call__\n    return self.run(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/managers/content/upload.py\", line 223, in import_uploaded_unit\n    unit_type=unit_type_id, summary=result['summary'], details=result['details']\nPulpCodedException: The importer yum_importer indicated a failed response when uploading rpm unit to repository yocto-x86_64.\n", 
  "spawned_tasks": [], 
  "progress_report": {}, 
  "queue": "reserved_resource_worker-0@pulp-dev-win.unx.sas.com.dq", 
  "state": "error", 
  "worker_name": "reserved_resource_worker-0@pulp-dev-win.unx.sas.com", 
  "result": null, 
  "error": {
    "code": "PLP0047", 
    "data": {
      "unit_type": "rpm", 
      "importer_id": "yum_importer", 
      "repo_id": "yocto-x86_64", 
      "details": {
        "errors": [
          "unexpected error occurred importing uploaded file: The field 'checksum_type' does not exist on the document 'RPM'"
        ]
      }, 
      "summary": ""
    }, 
    "description": "The importer yum_importer indicated a failed response when uploading rpm unit to repository yocto-x86_64.", 
    "sub_errors": []
  }, 
  "_id": {
    "$oid": "582b4ca77f5187661abe7e67"
  }, 
  "id": "582b4ca77f5187661abe7e67"
[\]
Running...

Task Failed

The importer yum_importer indicated a failed response when uploading rpm unit to
repository yocto-x86_64.

Deleting the upload request...
2016-11-15 12:43:03,740 - DEBUG - sending DELETE request to /pulp/api/v2/content/uploads/c9ed44ea-65da-4764-b74b-f8061d341546/
2016-11-15 12:43:03,868 - INFO - DELETE request to /pulp/api/v2/content/uploads/c9ed44ea-65da-4764-b74b-f8061d341546/ with parameters None
2016-11-15 12:43:03,868 - INFO - Response status : 200 

2016-11-15 12:43:03,868 - INFO - Response body :
 null

... completed
Actions #4

Updated by bizhang over 7 years ago

  • Severity changed from 2. Medium to 3. High
  • Groomed changed from No to Yes
Actions #5

Updated by bizhang over 7 years ago

  • Triaged changed from No to Yes
  • Groomed changed from Yes to No
Actions #6

Updated by mihai.ibanescu@gmail.com over 7 years ago

rpm -q pulp-admin-client
pulp-admin-client-2.7.1-1.el7.noarch
Actions #7

Updated by bmbouter about 5 years ago

  • Status changed from NEW to CLOSED - WONTFIX
Actions #8

Updated by bmbouter about 5 years ago

Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.

Actions #9

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF