Story #7015
closed
As a user, I can associate a remote with a repository
100%
Description
Currently there is no way to link a repository with a remote, and as such, each time you wish to sync a repository against it's upstream link, you have to keep a record (or query for) the appropriate remote for that repo.
This can be done with careful consideration around naming conventions, however it would be nice if an optional link between a Repository and Remote was possible.
As such, once the Repository and Remote have been created an paired, a simply repository sync would be able to work out which remote to use, and sync.
Design¶
Model¶
Add a many-to-many foreign key between Repositories and Remotes. This association should be optional. For now Repositories can only have 0 or 1 remotes. Remotes can belong to any number of repositories.
Note: In the future we want to be able to support multiple remotes for repositories that can be synced at once. See https://pulp.plan.io/issues/7031. This is the purpose of having a many-to-many instead of one-to-many.
REMOTE_TYPES¶
Add an iterable to Repository similar to CONTENT_TYPES. This iterable will check that when a remote is added to a repo, it is of one of these types.
API¶
Repositories¶
Expose a remote hrefs field for repository endpoints. Users should be able to create/read/update this field. Validate the remote type against REMOTE_TYPES and that a user is only supplying 0..1 remote. Also, indicate in the help text that this field is optional.
Remotes¶
Expose a repository hrefs field for remotes. Users should be able to view this field to see which repos a remote belongs to. Also, indicate in the help text that this field is optional.
Syncing¶
Make the remote parameter on sync optional. If no remote is supplied, the sync task uses the remote field on Repository when syncing.
If a repository does not have a remote and no remote is supplied, raise an error.
Related issues
Updated by fao89 over 4 years ago
- Tracker changed from Issue to Story
- % Done set to 0
Updated by daviddavis over 4 years ago
- Description updated (diff)
- Tags deleted (
API Bindings)
Updated by daviddavis over 4 years ago
- Subject changed from Ability to link Repository with Remote to As a user, I can associate a set of remotes with a repository
Updated by bmbouter over 4 years ago
Can we outline what validation means a bit more? Should the plugin developer specify on a typed Repository what Remote types can be paired with it? For example pulp_ansible has a AnsibleRepository and a CollectionRemote and RolesRemote. So in that case the Repository would specify both CollectionRemote and RolesRemore are allowed.
Also does a Repository only pair with a single Remote? Or are we allowing a Repository to pair with multiple remotes?
The model layer calls out the optional nature of the ManyToMany, but maybe the API should call out that these new href fields are optional as well (for clarity)?
Updated by daviddavis over 4 years ago
- Description updated (diff)
Can we outline what validation means a bit more? Should the plugin developer specify on a typed Repository what Remote types can be paired with it? For example pulp_ansible has a AnsibleRepository and a CollectionRemote and RolesRemote. So in that case the Repository would specify both CollectionRemote and RolesRemore are allowed.
This sounds great. I wonder if this should be a constant like we did for Content or a method. Any thoughts?
Also does a Repository only pair with a single Remote? Or are we allowing a Repository to pair with multiple remotes?
I was thinking many-to-many so a repository could have multiple remotes and a remote could belong to multiple repos. I will update the description to be more clear.
The model layer calls out the optional nature of the ManyToMany, but maybe the API should call out that these new href fields are optional as well (for clarity)?
Good idea. Done.
Updated by ipanova@redhat.com over 4 years ago
Right now you can sync with one remote at time, the way it is described in the story - it would be possible to sync a repo wit X related remotes and as a result only one repo version will be created. Was that intentional/desirable?
How mirror on sync will work with multiple remotes?Or retain old count?
What if one the feed from one of X remotes is broken, or sync of one remote fails in the middle - do we fail all the rest remote syncing? What if one of the remotes is a mirrorlist?
Updated by bmbouter over 4 years ago
+1 to using an iterable as a constant on Repository to define validation.
Is there a way to launch this feature with just a single remote and add on N remotes later? If not, we'll have to discuss more about what multi-remote sync means. I'm ok with trying to figure that out if folks think we should. What do you think?
Updated by daviddavis over 4 years ago
Is there a way to launch this feature with just a single remote and add on N remotes later? If not, we'll have to discuss more about what multi-remote sync means. I'm ok with trying to figure that out if folks think we should. What do you think?
I'm ok with launching this with just supporting a single remote. However, I am not sure how we'd add support for N remotes later in a backwards compatible way?
Updated by bmbouter over 4 years ago
daviddavis wrote:
Is there a way to launch this feature with just a single remote and add on N remotes later? If not, we'll have to discuss more about what multi-remote sync means. I'm ok with trying to figure that out if folks think we should. What do you think?
I'm ok with launching this with just supporting a single remote. However, I am not sure how we'd add support for N remotes later in a backwards compatible way?
List of remotes and fail validation if len(remotes) > 1 for now?
Updated by ipanova@redhat.com over 4 years ago
bmbouter wrote:
daviddavis wrote:
Is there a way to launch this feature with just a single remote and add on N remotes later? If not, we'll have to discuss more about what multi-remote sync means. I'm ok with trying to figure that out if folks think we should. What do you think?
I'm ok with launching this with just supporting a single remote. However, I am not sure how we'd add support for N remotes later in a backwards compatible way?
List of remotes and fail validation if len(remotes) > 1 for now?
that's an option. And open a separate story for syncing form multiple remotes
Updated by ttereshc over 4 years ago
We can probably support multiple remotes if we require configuration parameters (apart from feed and credentials) to be the same.
A more flexible approach would require plugin input (which parameters have to be the same, which can't be used at all, etc.)
+1 to have a list of 1 remote for now.
Updated by daviddavis over 4 years ago
- Subject changed from As a user, I can associate a set of remotes with a repository to As a user, I can associate a remote with a repository
- Description updated (diff)
Updated by bmbouter over 4 years ago
- Groomed changed from No to Yes
- Sprint Candidate changed from No to Yes
+1 to this story. Thanks!
Updated by daviddavis over 4 years ago
- Copied to Story #7135: As a user, I can associate a remote with a repository added
Updated by pulpbot over 4 years ago
- Status changed from NEW to POST
Updated by dalley over 4 years ago
@Brian, is there a good place where I can read about why both CollectionRemote and RolesRemote are necessary and separate? Just curious since it's an atypical case to think about. I have some other questions about the PR that are kinda related to that question.
Updated by daviddavis over 4 years ago
- Copied to Story #7159: As a user, I can associate a remote with a repository added
Updated by daviddavis over 4 years ago
- Copied to Story #7194: As a user, I can associate a remote with a repository added
Added by daviddavis over 4 years ago
Updated by daviddavis over 4 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulpcore|f31c7b61b369b3a7b97899ec73ed1db331f21929.
Updated by pulpbot over 4 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
.
Allow Remotes to be associated to Repositories
fixes #7015 https://pulp.plan.io/issues/7015
Required PR: https://github.com/pulp/pulp_file/pull/415