Story #3202
Updated by bmbouter over 6 years ago
As a user, I can sync RPM/SRPM/Erratum from a remote Yum/DNF repository.
Other unknown types of a content should be ignored during sync.
This story is complete when:
* I can initiate a sync from remote Yum/DNF repo
* The sync completes without error
* I can see that the expected content was added to the repo
h2. Pseudo Code First Stage Example
The sync will create a Stages API first stage and then pass all of them to DeclarativeVersion. Here is some pseudocode that can run inside of the <code>__call__()</code> of the first stage.
<pre>
with ProgressBar(message='Downloading Metadata') as pb:
repomd_url = url = self.remote.url + '/repodata/repomd.xml'
downloader = self.remote.get_downloader(repomd_url) # get the downloader
result = await downloader.run() # downloading happens here
pb.increment() # tell the user we downloading something
other_metadata_files = get_other_metadata_files_as_list(result) # gives urls of comps, updateinfo, etc.
downloaders = []
for url in other_metadata_files:
downloader = self.remote.get_downloader(repomd_url) # get the downloader
downloaders.append(downloader.run()) # This only creates the coroutine, it doesn't run it
while downloaders:
done, downloaders = await asyncio.wait(pending, return_when=asyncio.FIRST_COMPLETED)
for finished_downloader in done:
pb.increment() # tell the user we downloading something
in_memory_parsed_metadata = createrepo_c.load_xml(finished_downloader.path)
for package in in_memory_parsed_metadata:
if is_updaterecord(package):
content = UpdateRecord(content_data_from_package)
artifact = Artifact(digest_data_from_package)
da = DeclarativeArtifact(artifact, url, entry.relative_path, self.remote)
dc = DeclarativeContent(content=file, d_artifacts=[da])
await out_q.put(dc)
if is_packagerecord(package):
# Similar to what we did only using RpmContent / SrpmContent
</pre>
The __call__ function will become long, so consider breaking it up into additional coroutine methods on the object. For example if we made a build_da_dc_and_emit() coroutine it would be defined like:
<pre>
async def build_da_dc_and_emit(self, content, artifact):
da = DeclarativeArtifact(artifact, url, entry.relative_path, self.remote)
dc = DeclarativeContent(content=file, d_artifacts=[da])
await out_q.put(dc)
# Then in your code call it with:
await self.build_da_dc_and_emit(content, artifact)
</pre>
.