Story #236
Don't re-download rpms if they exist on disk
100%
Description
If a content unit exists on disk but is not on the DB, Pulp re-downloads the content at sync time.
Pulp should instead recognize that a rpm plugin unit (rpm, srpm, drpm, distribution) is on disk and use that to create the unit in the database from. The result would be that content which exists on disk is not re-downloaded.
Steps to Reproduce:
1. Sync a large repo
2. Watch it take a while
3. Clear your mongo db
4. rerun pulp-manage-db
5. Sync the same repo
6. Watch it take the same amount of time (all the files are on the file system, so it should take the same amount of time)
Related issues
Associated revisions
Revision 3dd7cefc
View on GitHub
Adds docs for re-using files on disk when DB is lost
History
#1
Updated by mhrivnak over 5 years ago
- Platform Release set to 2.8.0
- Groomed set to No
- Sprint Candidate set to Yes
#3
Updated by bmbouter over 5 years ago
- Subject changed from RFE: Don't re-download rpms if they exist on disk to Don't re-download rpms if they exist on disk
#4
Updated by mhrivnak about 5 years ago
- Checklist item Document the basics of what needs to be backed up and restored from mongodb added
- Checklist item Document backing up /var/lib/pulp/content added
- Checklist item Document using the download all task with verification in a recovery situation added
- Priority changed from High to Normal
- Groomed changed from No to Yes
- Tags Documentation added
This is a documentation task with the checklist items above.
Document what to backup from mongodb, but we should probably avoid documenting specific commands that can change over time.
#5
Updated by jortel@redhat.com about 5 years ago
- Priority changed from Normal to High
- Platform Release deleted (
2.8.0)
#6
Updated by bmbouter about 5 years ago
- Blocked by Task #1600: Store content using consistent and deterministic paths added
#7
Updated by bmbouter about 5 years ago
- Blocked by Task #1601: Migrate /var/lib/pulp/content to new 2.8 storage paths. added
#8
Updated by bmbouter about 5 years ago
- Description updated (diff)
- Private changed from Yes to No
#9
Updated by bmbouter about 5 years ago
I want to know if there is a way this could be a platform functionality. Perhaps platform could provide a hook which plugin writers could call.
#11
Updated by bmbouter about 5 years ago
- Checklist item deleted (
Document the basics of what needs to be backed up and restored from mongodb) - Checklist item deleted (
Document backing up /var/lib/pulp/content)
These checklist items have been moved to #1603.
#13
Updated by mhrivnak almost 5 years ago
- Sprint Candidate changed from Yes to No
#14
Updated by Anonymous over 4 years ago
- Sprint/Milestone set to 24
- Sprint Candidate changed from No to Yes
#15
Updated by mhrivnak over 4 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to mhrivnak
#16
Updated by mhrivnak over 4 years ago
- Sprint/Milestone changed from 24 to 25
#17
Updated by mhrivnak over 4 years ago
- Related to Issue #2177: download_repo task fails to mark some units as downloaded added
#18
Updated by mhrivnak over 4 years ago
- Status changed from ASSIGNED to POST
#19
Updated by mhrivnak over 4 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulp:pulp|3dd7cefc2343b2b7b901dce2ce92ebbf14d3e723.
#20
Updated by semyers over 4 years ago
- Platform Release set to 2.9.3
#21
Updated by semyers over 4 years ago
- Status changed from MODIFIED to 5
#22
Updated by semyers over 4 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
#25
Updated by bmbouter about 3 years ago
- Sprint set to Sprint 7
#26
Updated by bmbouter about 3 years ago
- Sprint/Milestone deleted (
25)
#27
Updated by bmbouter about 2 years ago
- Tags Pulp 2 added
Please register to edit this issue
Adds docs for re-using files on disk when DB is lost
fixes #236 https://pulp.plan.io/issues/236