Issue #1478
closedPOST request to /pulp/api/v2/repositories/<reponame>/search/units/ does not transform id correctly when unit results returned
Description
On an empty/fresh pulp system, run the following:
pulp-admin login -u admin -p admin
pulp-admin rpm repo create --repo-id rawhide --feed http://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/x86_64/os/ --skip=rpm,drpm,erratum
pulp-admin rpm repo sync run --repo-id rawhide
Now try to search for all units of type distribution by running:
pulp-admin -vv rpm repo content distribution --repo-id rawhide
You'll receive a traceback as pulp-admin tries to parse the results:
Traceback (most recent call last):
File "/home/vagrant/devel/pulp/client_lib/pulp/client/extensions/core.py", line 474, in run
exit_code = Cli.run(self, args)
File "/usr/lib/python2.7/site-packages/okaara/cli.py", line 974, in run
exit_code = command_or_section.execute(self.prompt, remaining_args)
File "/home/vagrant/devel/pulp/client_lib/pulp/client/extensions/extensions.py", line 210, in execute
return self.method(*arg_list, **clean_kwargs)
File "/home/vagrant/devel/pulp_rpm/extensions_admin/pulp_rpm/extensions/admin/contents.py", line 337, in distribution
self.run_search([TYPE_DISTRIBUTION], self.write_distro, **kwargs)
File "/home/vagrant/devel/pulp_rpm/extensions_admin/pulp_rpm/extensions/admin/contents.py", line 171, in run_search
out_func(units)
File "/home/vagrant/devel/pulp_rpm/extensions_admin/pulp_rpm/extensions/admin/contents.py", line 356, in write_distro
'id': distro['id'],
KeyError: 'id'
The issue is not in Pulp admin. The data returned by the server is not correctly transforming the 'distribution_id' field back into the expected 'id' field. See this snip of data returned:
<snip>
"family": "Fedora",
"distribution_id": "ks-Fedora--rawhide-x86_64",
"timestamp": 1451988072.6876094,
"_last_updated": 1452026990,
"packagedir": "",
"variant": "",
"downloaded": true,
"version": "rawhide",
"version_sort_index": "$rawhide",
"pulp_user_metadata": {},
"_content_type_id": "distribution",
"_id": "4432836c-b6f4-45ea-b579-1ae1350a7ff9",
"arch": "x86_64",
"_ns": "units_distribution"
</snip>
The field renaming occurs with the 2.8.0 release so it is new since then. The fix is the have the server transform the renamed fields correctly.
Related issues
Updated by paji@redhat.com almost 9 years ago
To add to this issue the response returned by
RestClient.post "https://katello-pulp-nightly.example.com/pulp/api/v2/repositories/integration_test_id/search/units/", "{\"criteria\":{\"type_ids\":[\"erratum\"]}}", "Accept"=>"*/*; q=0.5, application/xml", "Accept-Encoding"=>"gzip, deflate", "Content-Length"=>"37", "accept"=>"application/json", "content_type"=>"application/json"
\n# => 200 OK | application/json 645 bytes
contains a bunch of errata with no "id" column under the "metadata" section. In other words
x = respons(post request to search errata)
x[0]["metadata"]["id"]
does not exist in 2.8 while it did in 2.6.
Updated by mhrivnak almost 9 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to semyers
- Triaged changed from No to Yes
Updated by semyers almost 9 years ago
search views don't appear to respect model serializers by default, but I think I see where to hook them
Updated by semyers almost 9 years ago
- Related to Issue #1489: Unit copy not copying package groups or categories added
Updated by semyers almost 9 years ago
- Related to deleted (Issue #1489: Unit copy not copying package groups or categories)
Updated by semyers almost 9 years ago
- Related to Issue #1490: Errata IDs not showing up correctly in the API added
Added by semyers almost 9 years ago
Added by semyers almost 9 years ago
Revision 6fdf3c7d | View on GitHub
content unit search views use content unit serializers
Recent changes in the pymongo > mongoengine conversion have resulted in different ways to represent mongo documents in the REST api views; the correct way is to use the mongoengine Document serializers wherever possible. When that isn't possible, remap_fields_with_serializer exists to use the information in a mongoengine serializer to modify serialized results properly.
Anything that is using the serialization functions in pulp.server.webservices.views.serializers.content should automaticaly benefit from this change, which implicitly fixes redmine #1490.
These changes can also be used to benefit views that don't call the existing serialization functions by explicitly using the new remap_fields_with_serializer function as-needed, such as RepoUnitSearch, which will help (but not fix) redmine #1478.
Updated by semyers almost 9 years ago
- Status changed from ASSIGNED to MODIFIED
https://github.com/pulp/pulp/pull/2310
"old" pymongo objects weren't been serialized when coming out of some search api views dealing with content units. The solution (after normalizing the model serializer location to model.SERIALIZER) was to create a helper function to serialize those objects using the field remapping rules from the newer mongoengine serializers. Most search views will transparently benefit from this change, with the issue in #1490 being a noted exception.
Updated by semyers almost 9 years ago
- Related to Issue #1505: fields missing when pulp-admin tries to present a Distribution added
Added by semyers almost 9 years ago
Revision c7e79237 | View on GitHub
RepoUnitSearch remaps unit fields using legacy field remapper
The reproducer in #1478 will still fail, but with a problem that appears to be specific to the distribution type in pulp_rpm. That problem is being tracked in https://pulp.plan.io/issues/1505
https://pulp.plan.io/issues/1478 fixes #1478
re #1505
Added by semyers almost 9 years ago
Revision c7e79237 | View on GitHub
RepoUnitSearch remaps unit fields using legacy field remapper
The reproducer in #1478 will still fail, but with a problem that appears to be specific to the distribution type in pulp_rpm. That problem is being tracked in https://pulp.plan.io/issues/1505
https://pulp.plan.io/issues/1478 fixes #1478
re #1505
Updated by semyers almost 9 years ago
- Status changed from ASSIGNED to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulp:pulp|c7e792378dfdd1def48e6537353c5d907cfb050f.
Updated by dkliban@redhat.com almost 9 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
content unit search views use content unit serializers
Recent changes in the pymongo > mongoengine conversion have resulted in different ways to represent mongo documents in the REST api views; the correct way is to use the mongoengine Document serializers wherever possible. When that isn't possible, remap_fields_with_serializer exists to use the information in a mongoengine serializer to modify serialized results properly.
Anything that is using the serialization functions in pulp.server.webservices.views.serializers.content should automaticaly benefit from this change, which implicitly fixes redmine #1490.
These changes can also be used to benefit views that don't call the existing serialization functions by explicitly using the new remap_fields_with_serializer function as-needed, such as RepoUnitSearch, which will help (but not fix) redmine #1478.
fixes #1490 https://pulp.plan.io/issues/1490
re #1478 https://pulp.plan.io/issues/1478