Actions
Story #3202
closedAs a user, I can sync RPM/SRPM/Erratum from a remote Yum/DNF repository
Start date:
Due date:
% Done:
100%
Estimated time:
(Total: 0:00 h)
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 42
Quarter:
Description
As a user, I can sync RPM/SRPM/Erratum from a remote Yum/DNF repository.
Other unknown types of a content should be ignored during sync.
This story is complete when:
- I can initiate a sync from remote Yum/DNF repo
- The sync completes without error
- I can see that the expected content was added to the repo
Pseudo Code First Stage Example¶
The sync will create a Stages API first stage and then pass all of them to DeclarativeVersion. Here is some pseudocode that can run inside of the __call__()
of the first stage.
with ProgressBar(message='Downloading Metadata') as pb:
repomd_url = url = self.remote.url + '/repodata/repomd.xml'
downloader = self.remote.get_downloader(repomd_url) # get the downloader
result = await downloader.run() # downloading happens here
pb.increment() # tell the user we downloading something
other_metadata_files = get_other_metadata_files_as_list(result) # gives urls of comps, updateinfo, etc.
downloaders = []
for url in other_metadata_files:
downloader = self.remote.get_downloader(repomd_url) # get the downloader
downloaders.append(downloader.run()) # This only creates the coroutine, it doesn't run it
while downloaders:
done, downloaders = await asyncio.wait(pending, return_when=asyncio.FIRST_COMPLETED)
for finished_downloader in done:
pb.increment() # tell the user we downloading something
in_memory_parsed_metadata = createrepo_c.load_xml(finished_downloader.path)
for package in in_memory_parsed_metadata:
if is_updaterecord(package):
content = UpdateRecord(content_data_from_package)
artifact = Artifact(digest_data_from_package)
da = DeclarativeArtifact(artifact, url, entry.relative_path, self.remote)
dc = DeclarativeContent(content=file, d_artifacts=[da])
await out_q.put(dc)
if is_packagerecord(package):
# Similar to what we did only using RpmContent / SrpmContent
The call function will become long, so consider breaking it up into additional coroutine methods on the object. For example if we made a build_da_dc_and_emit() coroutine it would be defined like:
async def build_da_dc_and_emit(self, content, artifact):
da = DeclarativeArtifact(artifact, url, entry.relative_path, self.remote)
dc = DeclarativeContent(content=file, d_artifacts=[da])
await out_q.put(dc)
# Then in your code call it with:
await self.build_da_dc_and_emit(content, artifact)
Related issues
Updated by ttereshc almost 7 years ago
- Blocked by Task #3199: Create model(s) for a Package content type added
Updated by ttereshc almost 7 years ago
- Subject changed from As a user, I can sync RPM/SRPM from a remote Yum/DNF repository to As a user, I can sync RPM/SRPM/Erratum from a remote Yum/DNF repository
Updated by bmbouter over 6 years ago
- Description updated (diff)
Adding an example about breaking the code up into smaller coroutines.
Updated by milan over 6 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to milan
Updated by milan over 6 years ago
- Status changed from ASSIGNED to NEW
- Assignee deleted (
milan)
Updated by ttereshc over 6 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to ttereshc
Updated by ttereshc over 6 years ago
Added by daviddavis about 6 years ago
Added by daviddavis about 6 years ago
Revision fcd156b3 | View on GitHub
Parsing erratum relationships and storing them on the model
Updated by daviddavis about 6 years ago
- Status changed from ASSIGNED to MODIFIED
Updated by kersom about 6 years ago
- Related to Test #4108: Test syncing RPM/DRPM/SRPM/Erratum added
Updated by ttereshc almost 5 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions
Initial pass at syncing rpm and erratum content
This handles creating packages and update records from an RPM remote. UpdateCollections and their packages are not included.
ref #3202 https://pulp.plan.io/issues/3202