Project

Profile

Help

Story #3202

closed

As a user, I can sync RPM/SRPM/Erratum from a remote Yum/DNF repository

Added by ttereshc over 6 years ago. Updated over 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
(Total: 0:00 h)
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 42
Quarter:

Description

As a user, I can sync RPM/SRPM/Erratum from a remote Yum/DNF repository.

Other unknown types of a content should be ignored during sync.

This story is complete when:

  • I can initiate a sync from remote Yum/DNF repo
  • The sync completes without error
  • I can see that the expected content was added to the repo

Pseudo Code First Stage Example

The sync will create a Stages API first stage and then pass all of them to DeclarativeVersion. Here is some pseudocode that can run inside of the __call__() of the first stage.

with ProgressBar(message='Downloading Metadata') as pb:
            repomd_url = url = self.remote.url + '/repodata/repomd.xml'
            downloader = self.remote.get_downloader(repomd_url)  # get the downloader
            result = await downloader.run()  # downloading happens here
            pb.increment()  # tell the user we downloading something

            other_metadata_files = get_other_metadata_files_as_list(result)  # gives urls of comps, updateinfo, etc.
            downloaders = []
            for url in other_metadata_files:
                       downloader = self.remote.get_downloader(repomd_url)  # get the downloader
                       downloaders.append(downloader.run())  # This only creates the coroutine, it doesn't run it

            while downloaders:
                       done, downloaders = await asyncio.wait(pending, return_when=asyncio.FIRST_COMPLETED)
                       for finished_downloader in done:
                                     pb.increment()  # tell the user we downloading something
                                     in_memory_parsed_metadata = createrepo_c.load_xml(finished_downloader.path)
                                     for package in in_memory_parsed_metadata:
                                                 if is_updaterecord(package):
                                                              content = UpdateRecord(content_data_from_package)
                                                              artifact = Artifact(digest_data_from_package)
                                                              da = DeclarativeArtifact(artifact, url, entry.relative_path, self.remote)
                                                              dc = DeclarativeContent(content=file, d_artifacts=[da])
                                                              await out_q.put(dc)
                                                 if is_packagerecord(package):
                                                              #  Similar to what we did  only using RpmContent / SrpmContent

The call function will become long, so consider breaking it up into additional coroutine methods on the object. For example if we made a build_da_dc_and_emit() coroutine it would be defined like:

async def build_da_dc_and_emit(self, content, artifact):
               da = DeclarativeArtifact(artifact, url, entry.relative_path, self.remote)
               dc = DeclarativeContent(content=file, d_artifacts=[da])
               await out_q.put(dc)

# Then in your code call it with:
await self.build_da_dc_and_emit(content, artifact)

Sub-issues 3 (0 open3 closed)

Task #3933: Save UpdateCollection and UpdateCollectionPackage in bulkCLOSED - CURRENTRELEASEbmbouter

Actions
Pulp - Story #3934: As a plugin writer, I can have a stage that removes duplicatesCLOSED - CURRENTRELEASEamacdona@redhat.com

Actions
Task #3942: Store a digest of the fields for an UpdateRecordCLOSED - DUPLICATE

Actions

Related issues

Related to RPM Support - Test #4108: Test syncing RPM/DRPM/SRPM/ErratumCLOSED - COMPLETEActions
Blocked by RPM Support - Task #3199: Create model(s) for a Package content typeCLOSED - CURRENTRELEASEdalley

Actions

Also available in: Atom PDF