Maintenance: Planio will be observing a scheduled maintenance window this Tuesday, November 5, 2024 from 03:00 UTC until 06:30 UTC to perform urgent network maintenance in our primary data center. Your Planio account will be unavailable during this maintenance window.
Issue #5951
closedyum_distributor wrongly skips publish after update of multi-repo errata
Description
When multi-repo errata are used, publishing a repo may skip with "Repository content has not changed since last publish", even if erratum units in the repo have been changed since last publish.
Steps to reproduce¶
In summary: put an advisory into multiple repos, publish the repos, then update the advisory via any single repo and publish repos again. All repos other than the one used for the update will skip publish, even though the erratum unit has actually changed.
Detailed steps:
0. Ensure at least two RPM repos exist
e.g., assuming default dev-env setup:
pulp-admin rpm repo create --repo-id zoo2
1. Create new erratum in repo 1
Upload an RPM for the advisory:
pulp-admin rpm repo uploads rpm --repo-id zoo --file devel/pulp_rpm/plugins/test/data/walrus-5.21-1.noarch.rpm
Then the advisory metadata:
echo walrus,5.21,1,0,noarch,walrus-5.21-1.noarch.rpm,e837a635cc99f967a70f34b268baa52e0f412c1502e08e924ff5b09f1f9573f2,sha256,walrus-5.21-1.src.rpm > walrus-pkglist.csv
pulp-admin rpm repo uploads erratum --repo-id zoo --erratum-id RHBA-4020:1234 --title test --description test --version 1 --release 1 --type bugfix --status test --updated 2019-01-01 --issued test --from test --pkglist-csv walrus-pkglist.csv
2. Copy content into repo 2
pulp-admin rpm repo copy all --from-repo-id zoo --to-repo-id zoo2
3. Publish repos
pulp-admin rpm repo publish run --repo-id zoo
pulp-admin rpm repo publish run --repo-id zoo2
4. Verify erratum appears in repodata for both repos
Example:
$ sudo /bin/sh -c 'cd /var/lib/pulp/published/yum/master/yum_distributor/zoo/*/repodata; zcat $(egrep --only-matching "[^/]+updateinfo.xml.gz" ./repomd.xml)'
<?xml version="1.0" encoding="utf-8"?>
<updates>
<update status="test" from="test" version="1" type="bugfix">
<id>RHBA-4020:1234</id>
<issued date="test" />
<title>test</title>
<release>1</release>
<pushcount>1</pushcount>
<description>test</description>
<updated date="2019-01-01" />
<references />
<pkglist>
<collection short="zoo_0_default">
<name>zoo_0_default</name>
<package src="walrus-5.21-1.src.rpm" name="walrus" epoch="0" version="5.21" release="1" arch="noarch">
<filename>walrus-5.21-1.noarch.rpm</filename>
<sum type="sha256">e837a635cc99f967a70f34b268baa52e0f412c1502e08e924ff5b09f1f9573f2</sum>
</package>
</collection>
</pkglist>
</update>
</updates>
$ sudo /bin/sh -c 'cd /var/lib/pulp/published/yum/master/yum_distributor/zoo2/*/repodata; zcat $(egrep --only-matching "[^/]+updateinfo.xml.gz" ./repomd.xml)'
<?xml version="1.0" encoding="utf-8"?>
<updates>
<update status="test" from="test" version="1" type="bugfix">
<id>RHBA-4020:1234</id>
<issued date="test" />
<title>test</title>
<release>1</release>
<pushcount>1</pushcount>
<description>test</description>
<updated date="2019-01-01" />
<references />
<pkglist>
<collection short="zoo2_0_default">
<name>zoo2_0_default</name>
<package src="walrus-5.21-1.src.rpm" name="walrus" epoch="0" version="5.21" release="1" arch="noarch">
<filename>walrus-5.21-1.noarch.rpm</filename>
<sum type="sha256">e837a635cc99f967a70f34b268baa52e0f412c1502e08e924ff5b09f1f9573f2</sum>
</package>
</collection>
</pkglist>
</update>
</updates>
5. Modify the erratum via repo 1
Here, description, version and updated are being changed.
pulp-admin rpm repo uploads erratum --repo-id zoo --erratum-id RHBA-4020:1234 --title test --description UPDATED --version 2 --release 1 --type bugfix --status test --updated 2020-01-01 --issued test --from test --pkglist-csv walrus-pkglist.csv
6. Publish the repos again
pulp-admin rpm repo publish run --repo-id zoo
pulp-admin rpm repo publish run --repo-id zoo2
7. Check repodata again
$ sudo /bin/sh -c 'cd /var/lib/pulp/published/yum/master/yum_distributor/zoo/*/repodata; zcat $(egrep --only-matching "[^/]+updateinfo.xml.gz" ./repomd.xml)'
<?xml version="1.0" encoding="utf-8"?>
<updates>
<update status="test" from="test" version="2" type="bugfix">
<id>RHBA-4020:1234</id>
<issued date="test" />
<title>test</title>
<release>1</release>
<pushcount>1</pushcount>
<description>UPDATED</description>
<updated date="2020-01-01" />
<references />
<pkglist>
<collection short="zoo_0_default">
<name>zoo_0_default</name>
<package src="walrus-5.21-1.src.rpm" name="walrus" epoch="0" version="5.21" release="1" arch="noarch">
<filename>walrus-5.21-1.noarch.rpm</filename>
<sum type="sha256">e837a635cc99f967a70f34b268baa52e0f412c1502e08e924ff5b09f1f9573f2</sum>
</package>
</collection>
</pkglist>
</update>
</updates>
$ sudo /bin/sh -c 'cd /var/lib/pulp/published/yum/master/yum_distributor/zoo2/*/repodata; zcat $(egrep --only-matching "[^/]+updateinfo.xml.gz" ./repomd.xml)'
<?xml version="1.0" encoding="utf-8"?>
<updates>
<update status="test" from="test" version="1" type="bugfix">
<id>RHBA-4020:1234</id>
<issued date="test" />
<title>test</title>
<release>1</release>
<pushcount>1</pushcount>
<description>test</description>
<updated date="2019-01-01" />
<references />
<pkglist>
<collection short="zoo2_0_default">
<name>zoo2_0_default</name>
<package src="walrus-5.21-1.src.rpm" name="walrus" epoch="0" version="5.21" release="1" arch="noarch">
<filename>walrus-5.21-1.noarch.rpm</filename>
<sum type="sha256">e837a635cc99f967a70f34b268baa52e0f412c1502e08e924ff5b09f1f9573f2</sum>
</package>
</collection>
</pkglist>
</update>
</updates>
Actual behavior¶
updateinfo is only up to date in one of the published repos, even though publish was requested on both repos. Pulp skips publishing of the second repo.
Expected behavior¶
Neither publish is skipped, updateinfo is up to date in both published repos.
Additional info¶
For the upload use-case, it's possible for clients to work around this by explicitly importing the updated erratum into every repo containing the erratum (i.e. repeat step 5 for each repo containing the erratum). Although only the first import actually updates the units, the others will still reset last_unit_added on each repo, avoiding skipped publishes.
It seems possible that syncing would have the same problem (not tested).
Tested on current 2-master: bb9195f1dd30f51ed973024cbed8e087205749fc (pulp), cc35af04da2b9725db49290ebbc3253965951abf (pulp_rpm)
Add controller method for multi-repo update of last_unit_added
In rare cases where mutating a single unit can affect multiple repos at once (e.g. erratum units), it's necessary to set last_unit_added on all relevant repos. Add a controller method for this, to be invoked by the relevant importer(s).
re: #5951 https://pulp.plan.io/issues/5951