Project

Profile

Help

Story #5356

closed

As a user, I can download a configuration for yum/dnf from an RpmDistribution

Added by dkliban@redhat.com over 4 years ago. Updated almost 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Low
Assignee:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

It is common for repositories to include a repo file that can be used to add the repository to a client. This file can be manually placed into /etc/yum.repos.d/ or it can be specified as a parameter to yum/dnf. e.g.

dnf config-manager --add-repo http://example.com/pulp/content/some/repo/path/config.repo

Related issues

Related to Pulp - Story #6570: as a plugin writer, my custom content handler can handle requests for CONTENT_PATH_PREFIX CLOSED - CURRENTRELEASE

Actions
Actions #1

Updated by ttereshc about 4 years ago

  • Sprint/Milestone set to Priority items (outside of planned milestones/releases)
Actions #2

Updated by ttereshc about 4 years ago

  • Priority changed from Normal to Low
Actions #3

Updated by dkliban@redhat.com almost 4 years ago

The RpmPublicationSerializer[0] will need to be updated to accept an RpmDistribution as an optional parameter. When such parameter is passed in, the RpmPublicationViewset[1] should pass the distribution's PK to the publish task[2]. The publish task should then create a new Artifact and PublishedArtifact for the config.repo file. The file should contain the URL associated with the RpmDistribution that was passed in by the REST API user. If a SigningService is being used to sign the metadata, config.repo should also contain a path to the public key in the publication.

[0] https://github.com/pulp/pulp_rpm/blob/1383fbf86b19a3baacdd46be1903344f1eb24c25/pulp_rpm/app/serializers.py#L337

[1] https://github.com/pulp/pulp_rpm/blob/1383fbf86b19a3baacdd46be1903344f1eb24c25/pulp_rpm/app/viewsets.py#L228

[2] https://github.com/pulp/pulp_rpm/blob/1383fbf86b19a3baacdd46be1903344f1eb24c25/pulp_rpm/app/tasks/publishing.py#L201

Actions #4

Updated by dkliban@redhat.com almost 4 years ago

The solution in the previous comment creates severe limitations for the user. Any publication that includes a config.repo file will need to be recreated each time the user wants to use it with a new Distribution. This file should always be dynamically generated when it is requested from the content app. This can be achieved in the following way:

The RPM plugin needs to introduce a new setting to allow users to specify RPM_CONTENT_PATH_PREFIX.

The RPM plugin needs to provide it's own content handler that inherits from pulpcore.app.content.Handler[0]. The new handler should handle a route that looks like this: settings.RPM_CONTENT_PATH_PREFIX + '{path:.+}'. This is a slight variation on what pulpcore already provides[1].

The handler needs to override the _match_and_stream method[3]. This method needs figure the distribution being requested, call permit(), and then check if the path being requested is equal to 'config.repo'. If it is, it should dynamically generate the config file based on the base_url of the distribution and any information about a public key associated with the publication that's associated with the distribution. If the path requested is not config.repo, then it should simply call super().

[0] https://github.com/pulp/pulpcore/blob/master/pulpcore/content/handler.py#L50 [1] https://github.com/pulp/pulpcore/blob/master/pulpcore/content/__init__.py#L51 [2] https://github.com/pulp/pulpcore/blob/master/pulpcore/content/handler.py#L303

Actions #5

Updated by ipanova@redhat.com almost 4 years ago

makes sense to me, +1

Actions #6

Updated by bmbouter almost 4 years ago

The proposed plan in Comment 4 would work and with what we have currently it's what I would recommend also. I want to propose an alternative for us to consider too.

What if we did these things?

  1. Add a hook method to BaseDistribution named content_handler() which would provide a no-op as it's base functionality and return None by default. It would take subpath as the single param and that would be the url portion after the distribution's base_path (the remaining part).
  2. In the content app just after the Distribution is matched and permit() is called we have the content app call this new content_handler. That would roughly be here.
  3. If the content_handler(subpath) returns a subclass of aiohttp.web.Response(...) then return that and don't continue to call the rest of the code in the content app.

Then the RpmDistribution would implement content_handler(sub_path) and if sub_path == config.repo it would return the dynamically generated repo file.

The reasoning for this idea is to keep the repo file handout near the repo itself. Users think of the repo as the distribution path, and this would keep them together.

Actions #8

Updated by dkliban@redhat.com almost 4 years ago

bmbouter That is a great improvement to the design. It removes the need to add a new URL for serving RPM content. @lieter what do you think about opening another PR against pulpcore with this new interface in BaseDistribution and a call to it from the BaseHandler?

Actions #9

Updated by dkliban@redhat.com almost 4 years ago

  • Subject changed from RpmPublication should include a repo file that clients can use to configure yum/dnf to As a user, I can download a configuration for yum/dnf from an RpmDistribution
Actions #10

Updated by dkliban@redhat.com almost 4 years ago

I added a story for pulpcore to capture the requirements for that piece of the work.

https://pulp.plan.io/issues/6570

Actions #11

Updated by dkliban@redhat.com almost 4 years ago

  • Related to Story #6570: as a plugin writer, my custom content handler can handle requests for CONTENT_PATH_PREFIX added
Actions #12

Updated by rchan almost 4 years ago

  • Status changed from NEW to ASSIGNED
Actions #13

Updated by dkliban@redhat.com almost 4 years ago

The public key is added to the Publication as PublishedMetadata. It's placed into the 'repodata' directory. I am not sure if that's the right path for this file though.

https://github.com/pulp/pulp_rpm/blob/master/pulp_rpm/app/tasks/publishing.py#L434

Actions #14

Updated by dkliban@redhat.com almost 4 years ago

The signing service interface is not strong enough and allows creators of signing scripts to generate arbitrary names for public key files. We should improve this interface and enforce that the name of the public key file is public.key.

Until the interface is made stronger, we should document this requirement and the content_handler for RPM distribution should assume that the public key's relative_path is repodata/public.key.

Actions #15

Updated by mdellweg almost 4 years ago

wrote:

The public key is added to the Publication as PublishedMetadata. It's placed into the 'repodata' directory. I am not sure if that's the right path for this file though.

https://github.com/pulp/pulp_rpm/blob/master/pulp_rpm/app/tasks/publishing.py#L434

You can specify whatever relative_path you desire, when creating the published metadata artifact. (relative_path="repodata/public.key") It should not matter at all what name the signing script chose. Also I believe the public key should be part of the distribution, since you probably do not want to invoke the singing script in the content handler.

Added by Pieter Lexis almost 4 years ago

Revision b1d7979e | View on GitHub

Implement .repo file for RPM repositories

ref #5356

Added by Pieter Lexis almost 4 years ago

Revision 6a661c86 | View on GitHub

Add directory listing for public.key, config.repo

ref #5356

Added by Pieter Lexis almost 4 years ago

Revision c9c226e1 | View on GitHub

Assume the public key is called public.key

ref #5356

Added by Pieter Lexis almost 4 years ago

Revision 6b3b149f | View on GitHub

Add tests for config.repo

closes #5356

Required PR: https://github.com/pulp/pulpcore/pull/678

Actions #16

Updated by Anonymous almost 4 years ago

  • Status changed from ASSIGNED to MODIFIED
  • % Done changed from 0 to 100
Actions #17

Updated by ttereshc almost 4 years ago

  • Sprint/Milestone changed from Priority items (outside of planned milestones/releases) to Pulp RPM 3.4.0
Actions #18

Updated by ttereshc almost 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF