Project

Profile

Help

Issue #4477

Duplicated empty collections are published with subsequent sync/publish of modular errata

Added by ragbalak 8 months ago. Updated 6 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Severity:
2. Medium
Version:
2.18.1
Platform Release:
2.19.0
Blocks Release:
2.19.z
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
Yes
Sprint:
Sprint 50

Description

Problem

When a repository with feed "https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-modules/" is created, synced and published, the updateinfo.xml of the published repo contains an inconsistent number of modules.

Steps to reproduce :

Create and sync a repo pointing to the rpm-with-modules fixture.

pulp-admin rpm repo create --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-modules/ --repo-id test-repo

pulp-admin rpm repo sync run --repo-id test-repo

After the repo is synced, get the number of module tags present in the repo

# note the repo id and the timestamp directory after that can change
zcat 8147252315a121c89a1be2058fe3367a52bf81dac61788e53358c271421cd9d6-updateinfo.xml.gz |grep "module" | wc -l
33 # current output of the number of modules in the update_info.xml

If the above repository is deleted, and then again the same steps are followed for a new repo, the total number of modules in the new repo are seen to be increased

pulp-admin rpm repo delete --repo-id test-repo
pulp-admin rpm repo create --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-modules/ --repo-id test-repo-new

pulp-admin rpm repo sync run --repo-id test-repo-new

zcat ef13ff9115725082dd55fef6251880d28d66c1b0acf28abf89ba7fd2b1ade96f-updateinfo.xml.gz |grep module |wc -l
36 # current output of the number of modules in the update_info.xml

As seen the number of modules have increased for the same feed url with different repo names.

Also the update-info.xml of the published repo is different comparing with the actual update-info.xml of the fixtures repo.

Have attached the update-info.xml of both the syncs and the fixtures repo.

Environment details :

# cat /etc/os-release
NAME="Red Hat Enterprise Linux Server" 
VERSION="7.6 (Maipo)" 
ID="rhel" 
ID_LIKE="fedora" 
VARIANT="Server" 
VARIANT_ID="server" 
VERSION_ID="7.6" 
PRETTY_NAME="Employee SKU" 
ANSI_COLOR="0;31" 
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.6:GA:server" 
HOME_URL="https://www.redhat.com/" 
BUG_REPORT_URL="https://bugzilla.redhat.com/" 

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" 
REDHAT_BUGZILLA_PRODUCT_VERSION=7.6
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 
REDHAT_SUPPORT_PRODUCT_VERSION="7.6" 

# rpm -qa|grep pulp|sort
libsolv-0.6.34-2.pulp.el7.x86_64
pulp-admin-client-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
pulp-deb-admin-extensions-1.9.0-0.1.alpha.201812120524giteefc050.el7.noarch
pulp-deb-plugins-1.9.0-0.1.alpha.201812120524giteefc050.el7.noarch
pulp-docker-admin-extensions-3.4.0-0.1.alpha.201812120503gitaee36e5.el7.noarch
pulp-docker-plugins-3.4.0-0.1.alpha.201812120503gitaee36e5.el7.noarch
pulp-ostree-admin-extensions-1.4.0-0.1.alpha.201812120505git6c63638.el7.noarch
pulp-ostree-plugins-1.4.0-0.1.alpha.201812120505git6c63638.el7.noarch
pulp-puppet-admin-extensions-2.19.0-0.1.alpha.201812120507gitc8c9a9a.el7.noarch
pulp-puppet-plugins-2.19.0-0.1.alpha.201812120507gitc8c9a9a.el7.noarch
pulp-puppet-tools-2.19.0-0.1.alpha.201812120507gitc8c9a9a.el7.noarch
pulp-python-admin-extensions-2.1.0-0.1.alpha.201812120501git5e2aa35.el7.noarch
pulp-python-plugins-2.1.0-0.1.alpha.201812120501git5e2aa35.el7.noarch
pulp-rpm-admin-extensions-2.19.0-0.1.alpha.201812120511git9bc9e73.el7.noarch
pulp-rpm-plugins-2.19.0-0.1.alpha.201812120511git9bc9e73.el7.noarch
pulp-selinux-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
pulp-server-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python2-solv-0.6.34-2.pulp.el7.x86_64
python-isodate-0.5.0-4.pulp.el7.noarch
python-pulp-bindings-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-client-lib-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-common-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-deb-common-1.9.0-0.1.alpha.201812120524giteefc050.el7.noarch
python-pulp-docker-common-3.4.0-0.1.alpha.201812120503gitaee36e5.el7.noarch
python-pulp-oid_validation-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-ostree-common-1.4.0-0.1.alpha.201812120505git6c63638.el7.noarch
python-pulp-puppet-common-2.19.0-0.1.alpha.201812120507gitc8c9a9a.el7.noarch
python-pulp-python-common-2.1.0-0.1.alpha.201812120501git5e2aa35.el7.noarch
python-pulp-repoauth-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-rpm-common-2.19.0-0.1.alpha.201812120511git9bc9e73.el7.noarch
python-pulp-streamer-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch

update-info.xml.secondone (12.1 KB) ragbalak, 02/26/2019 07:52 PM update-info.xml.secondone
update-info.xml.firstone (11.4 KB) ragbalak, 02/26/2019 07:52 PM update-info.xml.firstone
update-info.xml.fixture (4.26 KB) ragbalak, 02/26/2019 07:53 PM update-info.xml.fixture

Related issues

Related to Pulp - Test #4112: Verify module information on errata search api CLOSED - COMPLETE Actions
Related to RPM Support - Issue #4056: module information missing from errata search api CLOSED - CURRENTRELEASE Actions

Associated revisions

Revision 641750dc View on GitHub
Added by ttereshc 7 months ago

Handle duplicated modular collections in errata at publish time

closes #4477
https://pulp.plan.io/issues/4477

Revision f0ec47d9 View on GitHub
Added by Bruno Rocha 7 months ago

Adding tests for Modular content module counts after republish

Ensure module count is the same after respublishing 2 different
repositories with the same feed.

closes #4477

History

#1 Updated by ragbalak 8 months ago

#2 Updated by kersom 8 months ago

  • Related to Test #4112: Verify module information on errata search api added

#3 Updated by kersom 8 months ago

  • Related to Issue #4056: module information missing from errata search api added

#4 Updated by CodeHeeler 8 months ago

  • Project changed from Pulp to RPM Support

#5 Updated by ttereshc 8 months ago

@ragbalak, do you see the same behavior on 2.18.1? Or is it specific to 2.19?

#6 Updated by ttereshc 8 months ago

  • Subject changed from Updateinfo information of published repo inconsistent with the actual update info to Duplicated empty collections are published with subsequent sync/publish of modular errata
  • Priority changed from Normal to High

#7 Updated by CodeHeeler 8 months ago

  • Triaged changed from No to Yes

#8 Updated by ttereshc 7 months ago

  • Sprint set to Sprint 49

#9 Updated by rchan 7 months ago

  • Sprint changed from Sprint 49 to Sprint 50

#10 Updated by bherring 7 months ago

Additional Investigation Notes

The same behavior exists in 2.18.1.

As a note from Raga's investigation, removing orphans between runs will stop this behavior, but it seems like a non-intended design decision.

#11 Updated by ttereshc 7 months ago

  • Blocks Release 2.19.z added

#12 Updated by ttereshc 7 months ago

  • Version set to 2.18.1

#13 Updated by ttereshc 7 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc

#14 Updated by ttereshc 7 months ago

I think I know why it is happening.
For every new repository if it synced from feed, erratum collection is created, even if it's a duplicate. That's expected.

The logic of publishing an erratum is complicated and removal of duplicates is a part of it.
Erratum collection won't be published if it's non-modular and has no packages in it.
Erratum collection will be published if it's modular and even if it has no packages.
I guess, the duplicated RPMs are removed, while modular data is still in the collection, so all collections with modular info are published.

#15 Updated by ttereshc 7 months ago

  • Sprint/Milestone set to 2.19.0
  • Platform Release set to 2.19.0

#16 Updated by ttereshc 7 months ago

  • Status changed from ASSIGNED to POST

#17 Updated by ttereshc 7 months ago

  • Status changed from POST to MODIFIED

#18 Updated by ttereshc 7 months ago

  • Verification Required changed from No to Yes

#19 Updated by ttereshc 7 months ago

  • Status changed from MODIFIED to ON_QA

#20 Updated by ttereshc 7 months ago

  • Status changed from ON_QA to CLOSED - CURRENTRELEASE

#21 Updated by bmbouter 6 months ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF