Project

Profile

Help

Issue #1642

List of Content Unit Types is Incomplete

Added by Ichimonji10 over 5 years ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Master
Platform Release:
2.8.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Pulp supports the GET /pulp/api/v2/plugins/types/ API endpoint. When called, this endpoint returns a list of objects, where each object represents a single content unit type that has been registered with Pulp. Here's a snippet of what that looks like:

[{u'_href': u'/pulp/api/v2/plugins/types/docker_image/',
  u'_id': {u'$oid': u'56b8d0c1e779890daae1fe5e'},
  u'_ns': u'content_types',
  u'description': u'Docker Image',
  u'display_name': u'Docker Image',
  u'id': u'docker_image',
  u'referenced_types': [],
  u'search_indexes': [],
  u'unit_key': [u'image_id']},
 {u'_href': u'/pulp/api/v2/plugins/types/puppet_module/',
  u'_id': {u'$oid': u'56b8d0c1e779890daae1fe5f'},
  u'_ns': u'content_types',
  u'description': u'Puppet Module',
  u'display_name': u'Puppet Module',
  u'id': u'puppet_module',
  u'referenced_types': [],
  u'search_indexes': [u'author', u'tag_list'],
  u'unit_key': [u'name', u'version', u'author']},
  # ... many more objects

This works well on Pulp 2.7. Unfortunately, the output from this endpoint is incomplete on Pulp 2.8. One 2.8 system I have has the following plugins installed:

  • Docker
  • Ostree
  • Puppet
  • Python
  • RPM

And here's the output from the API call:

[{u'_href': u'/pulp/api/v2/plugins/types/repository/',
  u'_id': {u'$oid': u'56b4d13de779895df95db201'},
  u'_ns': u'content_types',
  u'description': u'A Pulp Repository',
  u'display_name': u'Repository',
  u'id': u'repository',
  u'referenced_types': [],
  u'search_indexes': [u'repo_id'],
  u'unit_key': [u'repo_id']},
 {u'_href': u'/pulp/api/v2/plugins/types/node/',
  u'_id': {u'$oid': u'56b4d13de779895df95db202'},
  u'_ns': u'content_types',
  u'description': u'A Pulp Node',
  u'display_name': u'Pulp Node',
  u'id': u'node',
  u'referenced_types': [],
  u'search_indexes': [u'node_id'],
  u'unit_key': [u'node_id']}]

Each plugin adds one (or more) content unit types to Pulp. For example:

  • The Python plugin adds the python_package content unit type.
  • The OSTree plugin adds the ostree content unit type.

This is problematic because it means that it is impossible to determine which plugins are installed.

Associated revisions

Revision d05f601f View on GitHub
Added by ttereshc over 5 years ago

Update content_type collection with content unit types of available plugins

re #1642 https://pulp.plan.io/issues/1642

Revision ea86e243 View on GitHub
Added by ttereshc over 5 years ago

Update content_type collection with content unit types of available plugins

closes #1642 https://pulp.plan.io/issues/1642

Revision ea86e243 View on GitHub
Added by ttereshc over 5 years ago

Update content_type collection with content unit types of available plugins

closes #1642 https://pulp.plan.io/issues/1642

History

#1 Updated by Ichimonji10 over 5 years ago

GET /pulp/api/v2/plugins/types/ostree/ and GET /pulp/api/v2/plugins/types/python_package/ return 404s on a Pulp 2.8 server with those plugins installed.

#2 Updated by ipanova@redhat.com over 5 years ago

this is due to mongoengine conversion, now when you run pulp-manage-db you see the node,ostree,repo type loaded. we need to fix that.

$ sudo -u apache pulp-manage-db
Attempting to connect to localhost:27017
Attempting to connect to localhost:27017
Write concern for Mongo connection: {}
Loading content types.
Loading type descriptors [ostree.json, node.json]
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Updating the database with types [ostree, repository, node]
Content types loaded.
....snip........

#3 Updated by Ichimonji10 over 5 years ago

this is due to mongoengine conversion, now when you run pulp-manage-db you see the node,ostree,repo type loaded. we need to fix that.

Can you explain further? I'm not sure if the message is:

  • pulp is not correctly reporting which content unit types are present, or
  • pulp 2.7 makes use of different content units than pulp 2.8.

#4 Updated by pcreech over 5 years ago

With the mongoengine conversion, it appears we are no longer storing content types in the content_type collection in mongo, which this particular api endpoint pulls data from to populate the response. It appears the content_type collection drives all /pulp/api/v2/plugins/types/*/ as well.

This leads to the incorrect response above, where we don't have all content types listed; as well as the 404's that can be seen for the other content type specific endpoints.

#5 Updated by bmbouter over 5 years ago

The mongoengine types now use entry-points so discovering what content types are installed should be easy. As a possible fix, perhaps pulp-manage-db needs to ensure all content types are written into that collection.

As an aside I'm not sure why we even have that collection except that you may want httpd servers to not have the content-type rpms installed on them?? Perhaps other areas of the code use that? Generally I think fixing the issue of that collection not being up to date is the best thing to do in the short-term and leave the rethink of that collections' purpose for another time.

#6 Updated by pcreech over 5 years ago

Just adding some historical context, The data driving this collection appears to be driven by the /plugins/types/{config}.json files, which for rpm[0] were removed with this commit[1]

[0] https://github.com/pulp/pulp_rpm/tree/2.7-release/plugins/types
[1] https://github.com/pulp/pulp_rpm/commit/944a05f7c131893a3fd46f1a67a53e4a618d95d6

#7 Updated by bmbouter over 5 years ago

  • Platform Release set to 2.8.0
  • Triaged changed from No to Yes

#8 Updated by bmbouter over 5 years ago

The plugin entry points replaced the .json files so we could use those to easily facilitate the correct API behavior.

#9 Updated by ttereshc over 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc

#10 Updated by ttereshc over 5 years ago

  • Status changed from ASSIGNED to POST

#11 Updated by ttereshc over 5 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#12 Updated by dkliban@redhat.com over 5 years ago

  • Status changed from MODIFIED to 5

#13 Updated by mhrivnak over 5 years ago

The change was made in this commit, seemingly outside a pull request: https://github.com/pulp/pulp/commit/ea86e243983db33cf593887e9700f89fbdab375c

#14 Updated by dkliban@redhat.com over 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

#15 Updated by bmbouter over 2 years ago

  • Tags Pulp 2 added

#16 Updated by bmbouter over 1 year ago

  • Category deleted (14)

We are removing the 'API' category per open floor discussion June 16, 2020.

Please register to edit this issue

Also available in: Atom PDF