Story #4209
Updated by bmbouter over 5 years ago
h2. Problem For plugins such as Python and Docker that have many metadata files, we may need a different way to implement lazy sync than the current plan, which is to omit the ArtifactDownloader stage and the ArtifactSaver stage. Because there are many metadata files, (and with Docker, these metadata files are also content units), we are using the ArtifactDownloader stage to download metadata. In this case, the metadata files need to be downloaded, and related content saved, but the non-metadata files should be skipped for downloading and saving. h2. Solution The DeclarativeArtifact gets I see at least 2 ways forward. *Option 1:* Change the implementation of lazy sync by adding a new flag called 'deferred_download' defaulting field to False the DeclarativeArtifact, `da.metadata`, which defaults to specify, that False. If `da.metadata` is True, and the artifact sync task has deferred downloading enabled, then the `ArtifactDownloader` and `ArtifactSaver` stages stop processing it (making it lazy). would skip this DeclarativeArtifact. This would allow lazy to be implemented for all plugins when DeclarativeArtifacts are created, all the logic would live in the pulpcore stages. At I suspect that the same time performance difference vs simply omitting the 'download_artifacts' flag stages would be negligible. Though this way would have 2 extra stages running (for single metadata-file plugins), they would simply be passing each item to the next stage. *Option 2:* Plugin writers with multiple metadata files override the ArtifactDownloader and ArtifactSaver stages to skip certain types of DeclarativeVersion is removed. So the plugin must content. This could be done either use by overriding the deferred_download flag, whole class, or tailor its own Pipeline. by adding a skip hook, which the plugin writer would define.
.