Project

Profile

Help

Issue #4477

closed

Duplicated empty collections are published with subsequent sync/publish of modular errata

Added by ragbalak about 5 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.18.1
Platform Release:
2.19.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 50
Quarter:

Description

Problem

When a repository with feed "https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-modules/" is created, synced and published, the updateinfo.xml of the published repo contains an inconsistent number of modules.

Steps to reproduce :

Create and sync a repo pointing to the rpm-with-modules fixture.

pulp-admin rpm repo create --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-modules/ --repo-id test-repo

pulp-admin rpm repo sync run --repo-id test-repo

After the repo is synced, get the number of module tags present in the repo

# note the repo id and the timestamp directory after that can change
zcat 8147252315a121c89a1be2058fe3367a52bf81dac61788e53358c271421cd9d6-updateinfo.xml.gz |grep "module" | wc -l
33 # current output of the number of modules in the update_info.xml

If the above repository is deleted, and then again the same steps are followed for a new repo, the total number of modules in the new repo are seen to be increased

pulp-admin rpm repo delete --repo-id test-repo
pulp-admin rpm repo create --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-modules/ --repo-id test-repo-new

pulp-admin rpm repo sync run --repo-id test-repo-new

zcat ef13ff9115725082dd55fef6251880d28d66c1b0acf28abf89ba7fd2b1ade96f-updateinfo.xml.gz |grep module |wc -l
36 # current output of the number of modules in the update_info.xml

As seen the number of modules have increased for the same feed url with different repo names.

Also the update-info.xml of the published repo is different comparing with the actual update-info.xml of the fixtures repo.

Have attached the update-info.xml of both the syncs and the fixtures repo.

Environment details :

# cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.6 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.6"
PRETTY_NAME="Employee SKU"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.6:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.6
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.6"

# rpm -qa|grep pulp|sort
libsolv-0.6.34-2.pulp.el7.x86_64
pulp-admin-client-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
pulp-deb-admin-extensions-1.9.0-0.1.alpha.201812120524giteefc050.el7.noarch
pulp-deb-plugins-1.9.0-0.1.alpha.201812120524giteefc050.el7.noarch
pulp-docker-admin-extensions-3.4.0-0.1.alpha.201812120503gitaee36e5.el7.noarch
pulp-docker-plugins-3.4.0-0.1.alpha.201812120503gitaee36e5.el7.noarch
pulp-ostree-admin-extensions-1.4.0-0.1.alpha.201812120505git6c63638.el7.noarch
pulp-ostree-plugins-1.4.0-0.1.alpha.201812120505git6c63638.el7.noarch
pulp-puppet-admin-extensions-2.19.0-0.1.alpha.201812120507gitc8c9a9a.el7.noarch
pulp-puppet-plugins-2.19.0-0.1.alpha.201812120507gitc8c9a9a.el7.noarch
pulp-puppet-tools-2.19.0-0.1.alpha.201812120507gitc8c9a9a.el7.noarch
pulp-python-admin-extensions-2.1.0-0.1.alpha.201812120501git5e2aa35.el7.noarch
pulp-python-plugins-2.1.0-0.1.alpha.201812120501git5e2aa35.el7.noarch
pulp-rpm-admin-extensions-2.19.0-0.1.alpha.201812120511git9bc9e73.el7.noarch
pulp-rpm-plugins-2.19.0-0.1.alpha.201812120511git9bc9e73.el7.noarch
pulp-selinux-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
pulp-server-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python2-solv-0.6.34-2.pulp.el7.x86_64
python-isodate-0.5.0-4.pulp.el7.noarch
python-pulp-bindings-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-client-lib-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-common-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-deb-common-1.9.0-0.1.alpha.201812120524giteefc050.el7.noarch
python-pulp-docker-common-3.4.0-0.1.alpha.201812120503gitaee36e5.el7.noarch
python-pulp-oid_validation-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-ostree-common-1.4.0-0.1.alpha.201812120505git6c63638.el7.noarch
python-pulp-puppet-common-2.19.0-0.1.alpha.201812120507gitc8c9a9a.el7.noarch
python-pulp-python-common-2.1.0-0.1.alpha.201812120501git5e2aa35.el7.noarch
python-pulp-repoauth-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-rpm-common-2.19.0-0.1.alpha.201812120511git9bc9e73.el7.noarch
python-pulp-streamer-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch

Files

update-info.xml.secondone (12.1 KB) update-info.xml.secondone ragbalak, 02/26/2019 07:52 PM
update-info.xml.firstone (11.4 KB) update-info.xml.firstone ragbalak, 02/26/2019 07:52 PM
update-info.xml.fixture (4.26 KB) update-info.xml.fixture ragbalak, 02/26/2019 07:53 PM

Related issues

Related to Pulp - Test #4112: Verify module information on errata search apiCLOSED - COMPLETErochacbrunoActions
Related to RPM Support - Issue #4056: module information missing from errata search apiCLOSED - CURRENTRELEASEmilanActions
Actions #1

Updated by ragbalak about 5 years ago

Actions #2

Updated by kersom about 5 years ago

  • Related to Test #4112: Verify module information on errata search api added
Actions #3

Updated by kersom about 5 years ago

  • Related to Issue #4056: module information missing from errata search api added
Actions #4

Updated by CodeHeeler about 5 years ago

  • Project changed from Pulp to RPM Support
Actions #5

Updated by ttereshc about 5 years ago

ragbalak, do you see the same behavior on 2.18.1? Or is it specific to 2.19?

Actions #6

Updated by ttereshc about 5 years ago

  • Subject changed from Updateinfo information of published repo inconsistent with the actual update info to Duplicated empty collections are published with subsequent sync/publish of modular errata
  • Priority changed from Normal to High
Actions #7

Updated by CodeHeeler about 5 years ago

  • Triaged changed from No to Yes
Actions #8

Updated by ttereshc about 5 years ago

  • Sprint set to Sprint 49
Actions #9

Updated by rchan about 5 years ago

  • Sprint changed from Sprint 49 to Sprint 50
Actions #10

Updated by bherring about 5 years ago

Additional Investigation Notes

The same behavior exists in 2.18.1.

As a note from Raga's investigation, removing orphans between runs will stop this behavior, but it seems like a non-intended design decision.

Actions #12

Updated by ttereshc about 5 years ago

  • Version set to 2.18.1
Actions #13

Updated by ttereshc about 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc
Actions #14

Updated by ttereshc about 5 years ago

I think I know why it is happening.
For every new repository if it synced from feed, erratum collection is created, even if it's a duplicate. That's expected.

The logic of publishing an erratum is complicated and removal of duplicates is a part of it.
Erratum collection won't be published if it's non-modular and has no packages in it.
Erratum collection will be published if it's modular and even if it has no packages.
I guess, the duplicated RPMs are removed, while modular data is still in the collection, so all collections with modular info are published.

Actions #15

Updated by ttereshc about 5 years ago

  • Sprint/Milestone set to 2.19.0
  • Platform Release set to 2.19.0

Added by ttereshc about 5 years ago

Revision 641750dc | View on GitHub

Handle duplicated modular collections in errata at publish time

closes #4477 https://pulp.plan.io/issues/4477

Actions #16

Updated by ttereshc about 5 years ago

  • Status changed from ASSIGNED to POST
Actions #17

Updated by ttereshc about 5 years ago

  • Status changed from POST to MODIFIED
Actions #19

Updated by ttereshc about 5 years ago

  • Status changed from MODIFIED to 5

Added by Bruno Rocha about 5 years ago

Revision f0ec47d9 | View on GitHub

Adding tests for Modular content module counts after republish

Ensure module count is the same after respublishing 2 different repositories with the same feed.

closes #4477

Actions #20

Updated by ttereshc almost 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #21

Updated by bmbouter almost 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF