Pulp

Ticket moved to GitHub: "pulp/pulpcore/2010":https://github.com/pulp/pulpcore/issues/2010

We're beginning to implement in many plugins the behavior that "mirror=True" will automatically create a publication using existing metadata downloaded directly from the remote source. This is partly an optimization, and partly a feature, as it allows perfect mirroring of external repos (including exact intact checksums, timestamps, and signatures).

Most users would not want to use this as a one-off feature, they would want a given repository to always mirror a certain repo. The current UX is sub-optimal for this case. Problems:

• Users have to create both a remote and a repository, and associate them, which is 2 or 3 steps. It would be nice if this were streamlined.
• Users aren't prevented from performing invalid / unexpected operations on "mirrored" repositories. e.g. adding or removing content should be impossible, and there should be no (separate) auto-publish behavior because the sync itself will create a publication.

To fix this, we should introduce a new RemoteRepositoryViewset which will provide a new primitive Pulp endpoint, /pulp/api/v3/remote_repositories/<plugin>/<type>/. This viewset will work with all repositories that have remote set, which is to say that the queryset would be Repository.objects.filter(remote__isnull=False).

This viewset would provide only one action endpoint, "refresh" (or "update"), which would perform a mirror-mode sync and publish with the internally-attached remote. The viewset would not provide the "modify" endpoint since mirrored repositories shouldn't be able to be changed, and it wouldn't provide the "sync" endpoint because there would not be any need to customize the sync in that way. This avoids needing to throw errors if a user attempts an operation they shouldn't do - because the new API provides no mechanism to do it - and also avoids creating a new model.

The old /repositories/<plugin>/<type>/ endpoint will remain exactly as it is currently, even at the code level, since all of the changes are in the new viewset. However we should discourage users from using this API with repositories they want to sync from external sources, and encourage using the new API instead, which has the benefit of being much easier to use for most if not all use cases.

The workflow:

1. Create a "remote repository", specifying name + description + any remote options, which will transparently create both the remote and repository.
2. "refresh" (or "update" or "sync") it - no parameters necessary because it uses the internal remote. syncs for most plugins would always "mirror"

If a user just wants local mirrors, they're already done. For use cases where a user wants to modify or combine repos, we would encourage the "Katello workflow" of just copying all content from their remote repo into the new one.

Compared to syncing from many remotes into one repo, this workflow:

• Is much more efficient (less network traffic, only parse metadata once, no need to query existing Content or Artifacts)
• Is more deterministic / reliable (the remote URLs could change between syncs, or the syncs could accidentally use different mirrors)