Project

Profile

Help

Issue #5573

Publish won't create multiple checkecksummed copies of primary.xml, fileliststs.xml etc even when in fast-forward mode

Added by mihai.ibanescu@gmail.com about 1 year ago. Updated 8 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.20.0
Platform Release:
2.21.1
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

I had noticed that, even though I was publishing a repository after copying a unit into it (i.e. no removal), I was still not getting multiple copies of primary.xml.gz.

Background: repositories with frequent publishes may cause yum clients to issue 404 errors if the copy of repomd.xml points to metadata files that disappeared. So it's desirable for the metadata files to stick around for a while.

tthereshc found these tickets somewhat related:

https://pulp.plan.io/issues/1684
https://pulp.plan.io/issues/2788
https://pulp.plan.io/issues/3551
https://pulp.plan.io/issues/3816

However, none of them was addressing the behavior I was seeing - that the FileContext was picking the latest file (of which there was only one), moving it, uncompressing it for fast-forward operations, and then removing it. So, in my observations, we were starting with a list of one and ending with a list of one (usually different) file.

I tracked the code in pulp core, and here is the change I am proposing, that seems to keep the file around:

diff --git a/server/pulp/plugins/util/metadata_writer.py b/server/pulp/plugins/util/metadata_writer.py
index 5c459fad5..225528473 100644
--- a/server/pulp/plugins/util/metadata_writer.py
+++ b/server/pulp/plugins/util/metadata_writer.py
@@ -341,7 +341,7 @@ class FastForwardXmlFileContext(XmlFileContext):
             # move the file so that we can still process it if the name is the same
             if self.existing_file:
                 new_file_name = 'original.%s' % self.existing_file
-                shutil.move(os.path.join(working_dir, self.existing_file),
+                shutil.copy(os.path.join(working_dir, self.existing_file),
                             os.path.join(working_dir, new_file_name))
                 self.existing_file = new_file_name

Related issues

Related to RPM Support - Story #1684: Retain Old Repodata on Re-publishCLOSED - WONTFIX

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to RPM Support - Story #2788: As a user i can configure removal of old published repodata CLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to RPM Support - Issue #3551: RemoveOldRepodataStep for yum publisher not checking repomd.xml to remove old filesCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to RPM Support - Issue #3816: Pulp removes current metadata when remove_old_repodata_threshold = 0CLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision d1cf726f View on GitHub
Added by mihai.ibanescu@gmail.com about 1 year ago

Preserve older copies of metadata files (primary.xml, etc)

closes #5573

Revision 5870b580 View on GitHub
Added by mihai.ibanescu@gmail.com 8 months ago

Preserve older copies of metadata files (primary.xml, etc)

closes #5573

(cherry picked from commit d1cf726fa1ec99bb3aae1a8513bac5de3008b638)

History

#1 Updated by mihai.ibanescu@gmail.com about 1 year ago

  • Severity changed from 2. Medium to 4. Urgent

Updating severity, since our deployments fail with 404 errors because of this.

#2 Updated by mihai.ibanescu@gmail.com about 1 year ago

  • Related to Story #1684: Retain Old Repodata on Re-publish added

#3 Updated by mihai.ibanescu@gmail.com about 1 year ago

  • Related to Story #2788: As a user i can configure removal of old published repodata added

#4 Updated by mihai.ibanescu@gmail.com about 1 year ago

  • Related to Issue #3551: RemoveOldRepodataStep for yum publisher not checking repomd.xml to remove old files added

#5 Updated by mihai.ibanescu@gmail.com about 1 year ago

  • Related to Issue #3816: Pulp removes current metadata when remove_old_repodata_threshold = 0 added

#6 Updated by ipanova@redhat.com about 1 year ago

  • Project changed from RPM Support to Pulp

#8 Updated by ttereshc about 1 year ago

  • Severity changed from 4. Urgent to 2. Medium

#9 Updated by fao89 about 1 year ago

  • Triaged changed from No to Yes

#10 Updated by fao89 about 1 year ago

  • Tags Pulp 2 added

#11 Updated by mihai.ibanescu@gmail.com about 1 year ago

  • Status changed from NEW to MODIFIED

#12 Updated by ipanova@redhat.com 9 months ago

  • Platform Release set to 2.21.1

#14 Updated by ipanova@redhat.com 8 months ago

  • Status changed from MODIFIED to 5

#15 Updated by ipanova@redhat.com 8 months ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF