Actions
Story #2
closed
As a user, my rpm sync finishes quickly when upstream metadata hasn't changed
Start date:
Due date:
% Done:
100%
Estimated time:
Platform Release:
2.6.2
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
April 2015
Quarter:
Description
The repomd.xml file has its own timestamp in the XML, and then each file it references has its own timestamp. The treeinfo file contains its own timestamp, which already gets checked for freshness. We should check the repomd.xml timestamp and skip the rest of the sync if it hasn't changed. This will reduce the sync time for a big repo from possibly 1-2 minutes to a small number of seconds when the upstream repo hasn't changed. This will also help us be smarter about when the distribution files get re-downloaded.
Deliverables:
- rpm sync checks the timestamp in repomd.xml, and skips all unit types but "distribution" if it hasn't changed since the last sync.
- tell Katello team (specifically jsherrill) about this change once it's in Pulp. They might be able to remove some code on their end once this is in place.
- release notes
Updated by mhrivnak about 9 years ago
- Description updated (diff)
- Tags Groomed added
Updated by mhrivnak about 9 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to mhrivnak
Updated by mhrivnak about 9 years ago
- Description updated (diff)
I'm reducing the scope of this story because doing per-unit-type timestamp checking proved to be far more complex than it's worth. We can revisit once this workflow gets stepified, which should make that much more achievable.
Updated by mhrivnak about 9 years ago
- Status changed from ASSIGNED to POST
- % Done changed from 50 to 90
Updated by mhrivnak about 9 years ago
- Platform Release set to 2.6.2
closed above PR, opened this one instead: https://github.com/pulp/pulp_rpm/pull/676
Added by mhrivnak about 9 years ago
Updated by mhrivnak about 9 years ago
- Status changed from POST to MODIFIED
- % Done changed from 90 to 100
Applied in changeset 183cb1eb4c5c06c61d7e10f2bb46129a28f403ea.
Updated by bmbouter almost 9 years ago
- Groomed set to Yes
- Tags deleted (
Groomed)
Updated by bmbouter almost 9 years ago
- Sprint Candidate set to Yes
- Tags deleted (
Sprint Candidate)
Updated by dkliban@redhat.com almost 9 years ago
- Status changed from MODIFIED to 5
Updated by pthomas@redhat.com almost 9 years ago
- Status changed from 5 to 6
Verified
[root@mgmt12 ~]# rpm -qa pulp-server
pulp-server-2.6.2-0.2.beta.el6.noarch
[root@mgmt12 ~]#
[root@mgmt12 ~]# time pulp-admin rpm repo sync run --repo-id rad-5Client
+----------------------------------------------------------------------+
Synchronizing Repository [rad-5Client]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[\]
... completed
Downloading repository content...
[==================================================] 100%
RPMs: 0/0 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[-]
... completed
Task Succeeded
Copying files
[-]
... completed
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[ ] 0%
0 of 76 items
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[==================================================] 100%
1487 of 1487 items
... completed
Publishing Comps file
[==================================================] 100%
27 of 27 items
... completed
Publishing Metadata.
[-]
... completed
Closing repo metadata
[-]
... completed
Generating sqlite files
[\]
... completed
Publishing files to web
[-]
... completed
Writing Listings File
[-]
... completed
Writing Listings File
[-]
... completed
Task Succeeded
real 0m32.350s
user 0m1.026s
sys 0m0.151s
[root@mgmt12 ~]# time pulp-admin rpm repo sync run --repo-id rhel6
+----------------------------------------------------------------------+
Synchronizing Repository [rhel6]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[\]
... completed
Downloading repository content...
[==================================================] 100%
RPMs: 0/0 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[-]
... completed
Task Succeeded
Copying files
[|]
... completed
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[/]
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[==================================================] 100%
2913 of 2913 items
... completed
Publishing Comps file
[==================================================] 100%
212 of 212 items
... completed
Publishing Metadata.
[-]
... completed
Closing repo metadata
[-]
... completed
Generating sqlite files
... skipped
Publishing files to web
[/]
... completed
Writing Listings File
[-]
... completed
Task Succeeded
real 1m17.722s
user 0m1.785s
sys 0m0.203s
Updated by rbarlow about 8 years ago
- Status changed from 6 to CLOSED - CURRENTRELEASE
Actions
.
Sync now skips most steps if the feed metadata has not changed.
closes #2