Project

Profile

Help

Issue #5573

closed

Publish won't create multiple checkecksummed copies of primary.xml, fileliststs.xml etc even when in fast-forward mode

Added by mihai.ibanescu@gmail.com about 5 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.20.0
Platform Release:
2.21.1
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

I had noticed that, even though I was publishing a repository after copying a unit into it (i.e. no removal), I was still not getting multiple copies of primary.xml.gz.

Background: repositories with frequent publishes may cause yum clients to issue 404 errors if the copy of repomd.xml points to metadata files that disappeared. So it's desirable for the metadata files to stick around for a while.

tthereshc found these tickets somewhat related:

https://pulp.plan.io/issues/1684
https://pulp.plan.io/issues/2788
https://pulp.plan.io/issues/3551
https://pulp.plan.io/issues/3816

However, none of them was addressing the behavior I was seeing - that the FileContext was picking the latest file (of which there was only one), moving it, uncompressing it for fast-forward operations, and then removing it. So, in my observations, we were starting with a list of one and ending with a list of one (usually different) file.

I tracked the code in pulp core, and here is the change I am proposing, that seems to keep the file around:

diff --git a/server/pulp/plugins/util/metadata_writer.py b/server/pulp/plugins/util/metadata_writer.py
index 5c459fad5..225528473 100644
--- a/server/pulp/plugins/util/metadata_writer.py
+++ b/server/pulp/plugins/util/metadata_writer.py
@@ -341,7 +341,7 @@ class FastForwardXmlFileContext(XmlFileContext):
             # move the file so that we can still process it if the name is the same
             if self.existing_file:
                 new_file_name = 'original.%s' % self.existing_file
-                shutil.move(os.path.join(working_dir, self.existing_file),
+                shutil.copy(os.path.join(working_dir, self.existing_file),
                             os.path.join(working_dir, new_file_name))
                 self.existing_file = new_file_name

Related issues

Related to RPM Support - Story #1684: Retain Old Repodata on Re-publishCLOSED - WONTFIX

Actions
Related to RPM Support - Story #2788: As a user i can configure removal of old published repodata CLOSED - CURRENTRELEASEjluza

Actions
Related to RPM Support - Issue #3551: RemoveOldRepodataStep for yum publisher not checking repomd.xml to remove old filesCLOSED - CURRENTRELEASEActions
Related to RPM Support - Issue #3816: Pulp removes current metadata when remove_old_repodata_threshold = 0CLOSED - CURRENTRELEASEActions
Actions #1

Updated by mihai.ibanescu@gmail.com about 5 years ago

  • Severity changed from 2. Medium to 4. Urgent

Updating severity, since our deployments fail with 404 errors because of this.

Actions #2

Updated by mihai.ibanescu@gmail.com about 5 years ago

  • Related to Story #1684: Retain Old Repodata on Re-publish added
Actions #3

Updated by mihai.ibanescu@gmail.com about 5 years ago

  • Related to Story #2788: As a user i can configure removal of old published repodata added
Actions #4

Updated by mihai.ibanescu@gmail.com about 5 years ago

  • Related to Issue #3551: RemoveOldRepodataStep for yum publisher not checking repomd.xml to remove old files added
Actions #5

Updated by mihai.ibanescu@gmail.com about 5 years ago

  • Related to Issue #3816: Pulp removes current metadata when remove_old_repodata_threshold = 0 added
Actions #6

Updated by ipanova@redhat.com about 5 years ago

  • Project changed from RPM Support to Pulp

Added by mihai.ibanescu@gmail.com about 5 years ago

Revision d1cf726f | View on GitHub

Preserve older copies of metadata files (primary.xml, etc)

closes #5573

Actions #8

Updated by ttereshc about 5 years ago

  • Severity changed from 4. Urgent to 2. Medium
Actions #9

Updated by fao89 about 5 years ago

  • Triaged changed from No to Yes
Actions #10

Updated by fao89 about 5 years ago

  • Tags Pulp 2 added
Actions #11

Updated by mihai.ibanescu@gmail.com about 5 years ago

  • Status changed from NEW to MODIFIED
Actions #12

Updated by ipanova@redhat.com almost 5 years ago

  • Platform Release set to 2.21.1

Added by mihai.ibanescu@gmail.com almost 5 years ago

Revision 5870b580 | View on GitHub

Preserve older copies of metadata files (primary.xml, etc)

closes #5573

(cherry picked from commit d1cf726fa1ec99bb3aae1a8513bac5de3008b638)

Actions #14

Updated by ipanova@redhat.com almost 5 years ago

  • Status changed from MODIFIED to 5
Actions #15

Updated by ipanova@redhat.com almost 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

Also available in: Atom PDF