Refactor #3341
closedRemove the FK from publishers/importers to repos
Added by daviddavis over 6 years ago. Updated over 4 years ago.
100%
Updated by daviddavis over 6 years ago
- Status changed from ASSIGNED to POST
Updated by daviddavis over 6 years ago
A few todo items:
- We need an association from RemoteArtifact to RepositoryVersion now.
- Distribution needs a repository association. Since publishers are shared across repos, we don't want to auto-distribute different repos the same location on the filesystem.
- Consider adding back ContentAdaptor
Also, looks like pulp_file needs the auto-distribute code. Might have to look in git history to find it.
Edit: Found distribution code - https://git.io/vNFiH
Updated by amacdona@redhat.com over 6 years ago
How would the following situation be handled by the proposed design?
Repo A is synced (deferred downloading) from external source X, creating Repo Av1
Repo B is synced (deferred downloading) from external source X, creating Repo Bv1
Each repo is published.
Before this story, RemoteArtifacts were associated with the Importer. I think this actually makes the most sense conceptually, because the RemoteArtifact that lives on the "remote source" that is represented by the Importer. In the situation above, Av1 and Bv1 would have been synced using the same Importer, which would have allowed them to reference a single set of RemoteArtifacts. The current design suggests that RemoteArtifacts will be associated to a RepositoryVersion.
Problem: If RemoteArtifacts are associated to a RepositoryVersion, we lose the deduplication feature. Lets say a client tries to fetch PackageZ from Av1. It was deferred downloading, so the streamer does its magic and PackageZ is added to Pulp. Now a client tries to fetch PackageZ from Bv2, which references different RemoteArtifacts that are part of the same ContentUnit. The artifact will be streamed again.
AFAICT, this means that for deferred repositories, once they are published, there is no deduplication of RemoteArtifacts.
Updated by amacdona@redhat.com over 6 years ago
To take it a step farther, if we associate with RepositoryVersion, we could have the same problem with multiple versions of a single repo.
Repo A is synced (deferred) 3 times forming 3 versions that are mostly the same, but with some changes in each.
Av3 -> dev
Av2 -> testing
Av1 -> production
All 3 are published. Given that they share most content, it would be expected that after users who stream a pacakage from production, dev and testing copies of the same content would not need to be downloaded again.
Updated by daviddavis over 6 years ago
Talked with @asmacdo and @jortel about the above question @asmacdo asked. The artifact should only get streamed once because once the content_arifact has an artifact, it will use it instead of the artifact. Moreover, a publication points to the content_artifact and not a remote artifact. So duplication of remote_artifacts is ok.
We also talked about joining remote_artifact to repository_content instead of repository_version. The benefit here is that repository_version can be looked up from repository_content and when a repository_version is deleted, we don't have to worry about re-associating remote_artifacts with a newer (aka next) repository_version. Going to make that change.
Updated by jortel@redhat.com over 6 years ago
daviddavis@redhat.com wrote:
A few todo items:
- We need an association from RemoteArtifact to RepositoryVersion now.
- Distribution needs a repository association. Since publishers are shared across repos, we don't want to auto-distribute different repos the same location on the filesystem.
- Consider adding back ContentAdaptor
Also, looks like pulp_file needs the auto-distribute code. Might have to look in git history to find it.
Auto distribution is taken care of by the Publication context manager.
Edit: Found distribution code - https://git.io/vNFiH
Added by daviddavis over 6 years ago
Added by daviddavis over 6 years ago
Revision 7e10351b | View on GitHub
Remove relationship between publishers/importers and repos
Updated by daviddavis over 6 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulp|7e10351bb0a1ed353a6786e4bc20bf750b6f1236.
Updated by bmbouter over 4 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Remove relationship between publishers/importers and repos
closes #3341 https://pulp.plan.io/issues/3341