Issue #1642
closed
List of Content Unit Types is Incomplete
Description
Pulp supports the GET /pulp/api/v2/plugins/types/ API endpoint. When called, this endpoint returns a list of objects, where each object represents a single content unit type that has been registered with Pulp. Here's a snippet of what that looks like:
[{u'_href': u'/pulp/api/v2/plugins/types/docker_image/',
u'_id': {u'$oid': u'56b8d0c1e779890daae1fe5e'},
u'_ns': u'content_types',
u'description': u'Docker Image',
u'display_name': u'Docker Image',
u'id': u'docker_image',
u'referenced_types': [],
u'search_indexes': [],
u'unit_key': [u'image_id']},
{u'_href': u'/pulp/api/v2/plugins/types/puppet_module/',
u'_id': {u'$oid': u'56b8d0c1e779890daae1fe5f'},
u'_ns': u'content_types',
u'description': u'Puppet Module',
u'display_name': u'Puppet Module',
u'id': u'puppet_module',
u'referenced_types': [],
u'search_indexes': [u'author', u'tag_list'],
u'unit_key': [u'name', u'version', u'author']},
# ... many more objects
This works well on Pulp 2.7. Unfortunately, the output from this endpoint is incomplete on Pulp 2.8. One 2.8 system I have has the following plugins installed:
- Docker
- Ostree
- Puppet
- Python
- RPM
And here's the output from the API call:
[{u'_href': u'/pulp/api/v2/plugins/types/repository/',
u'_id': {u'$oid': u'56b4d13de779895df95db201'},
u'_ns': u'content_types',
u'description': u'A Pulp Repository',
u'display_name': u'Repository',
u'id': u'repository',
u'referenced_types': [],
u'search_indexes': [u'repo_id'],
u'unit_key': [u'repo_id']},
{u'_href': u'/pulp/api/v2/plugins/types/node/',
u'_id': {u'$oid': u'56b4d13de779895df95db202'},
u'_ns': u'content_types',
u'description': u'A Pulp Node',
u'display_name': u'Pulp Node',
u'id': u'node',
u'referenced_types': [],
u'search_indexes': [u'node_id'],
u'unit_key': [u'node_id']}]
Each plugin adds one (or more) content unit types to Pulp. For example:
- The Python plugin adds the
python_packagecontent unit type. - The OSTree plugin adds the
ostreecontent unit type.
This is problematic because it means that it is impossible to determine which plugins are installed.
Updated by Ichimonji10 over 7 years ago
GET /pulp/api/v2/plugins/types/ostree/ and GET /pulp/api/v2/plugins/types/python_package/ return 404s on a Pulp 2.8 server with those plugins installed.
Updated by ipanova@redhat.com over 7 years ago
this is due to mongoengine conversion, now when you run pulp-manage-db you see the node,ostree,repo type loaded. we need to fix that.
$ sudo -u apache pulp-manage-db
Attempting to connect to localhost:27017
Attempting to connect to localhost:27017
Write concern for Mongo connection: {}
Loading content types.
Loading type descriptors [ostree.json, node.json]
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Updating the database with types [ostree, repository, node]
Content types loaded.
....snip........
Updated by Ichimonji10 over 7 years ago
this is due to mongoengine conversion, now when you run pulp-manage-db you see the node,ostree,repo type loaded. we need to fix that.
Can you explain further? I'm not sure if the message is:
- pulp is not correctly reporting which content unit types are present, or
- pulp 2.7 makes use of different content units than pulp 2.8.
Updated by pcreech over 7 years ago
With the mongoengine conversion, it appears we are no longer storing content types in the content_type collection in mongo, which this particular api endpoint pulls data from to populate the response. It appears the content_type collection drives all /pulp/api/v2/plugins/types/*/ as well.
This leads to the incorrect response above, where we don't have all content types listed; as well as the 404's that can be seen for the other content type specific endpoints.
Updated by bmbouter over 7 years ago
The mongoengine types now use entry-points so discovering what content types are installed should be easy. As a possible fix, perhaps pulp-manage-db needs to ensure all content types are written into that collection.
As an aside I'm not sure why we even have that collection except that you may want httpd servers to not have the content-type rpms installed on them?? Perhaps other areas of the code use that? Generally I think fixing the issue of that collection not being up to date is the best thing to do in the short-term and leave the rethink of that collections' purpose for another time.
Updated by pcreech over 7 years ago
Just adding some historical context, The data driving this collection appears to be driven by the /plugins/types/{config}.json files, which for rpm[0] were removed with this commit[1]
[0] https://github.com/pulp/pulp_rpm/tree/2.7-release/plugins/types
[1] https://github.com/pulp/pulp_rpm/commit/944a05f7c131893a3fd46f1a67a53e4a618d95d6
Updated by bmbouter over 7 years ago
- Platform Release set to 2.8.0
- Triaged changed from No to Yes
Updated by bmbouter over 7 years ago
The plugin entry points replaced the .json files so we could use those to easily facilitate the correct API behavior.
Updated by ttereshc over 7 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to ttereshc
Updated by ttereshc over 7 years ago
- Status changed from ASSIGNED to POST
Added by ttereshc over 7 years ago
Added by ttereshc over 7 years ago
Update content_type collection with content unit types of available plugins
Added by ttereshc over 7 years ago
Update content_type collection with content unit types of available plugins
Updated by ttereshc over 7 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulp|ea86e243983db33cf593887e9700f89fbdab375c.
Updated by dkliban@redhat.com about 7 years ago
- Status changed from MODIFIED to 5
Updated by mhrivnak about 7 years ago
The change was made in this commit, seemingly outside a pull request: https://github.com/pulp/pulp/commit/ea86e243983db33cf593887e9700f89fbdab375c
Updated by dkliban@redhat.com about 7 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
Updated by bmbouter almost 3 years ago
- Category deleted (
14)
We are removing the 'API' category per open floor discussion June 16, 2020.
.
Update content_type collection with content unit types of available plugins
re #1642 https://pulp.plan.io/issues/1642