Issue #4880: Missing rpms in erratum pkglist when an erratum appears in multiple repos caused wrong applicability calculation. - RPM Support - Pulp

Send by e-mail

Issue #4880

Missing rpms in erratum pkglist when an erratum appears in multiple repos caused wrong applicability calculation.

Added by hyu 8 months ago. Updated 6 months ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Normal

Assignee:

ttereshc

Category:

Sprint/Milestone:

Pulp - 2.20.0

Start date:

Due date:

Severity:

3. High

Version:

2.18.1

Platform Release:

2.20.0

Blocks Release:

OS:

RHEL 7

Backwards Incompatible:

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Pulp 2

QA Contact:

Complexity:

Smash Test:

Verified:

Verification Required:

Sprint:

Sprint 54

Description

Description of problem:
Pulp will returns wrong 'pkglist' if an erratum exists in multiple repositories and user syncs multiple of these repositories

For example:
Erratum 'RHSA-2019:1235' exists in both 'rhel-7-server-rpms' repo and 'rhel-7-server-optional-debug-rpms' repo with the following 'pkglist' respectively

rhel-7-server-rpms:
"ruby-libs-2.0.0.648-35.el7_6.x86_64.rpm",
"ruby-libs-2.0.0.648-35.el7_6.i686.rpm",
"ruby-2.0.0.648-35.el7_6.x86_64.rpm",
"rubygem-bigdecimal-1.2.0-35.el7_6.x86_64.rpm",
"rubygem-psych-2.0.0-35.el7_6.x86_64.rpm",
"rubygem-rdoc-4.0.0-35.el7_6.noarch.rpm",
"ruby-irb-2.0.0.648-35.el7_6.noarch.rpm",
"rubygem-io-console-0.4.2-35.el7_6.x86_64.rpm",
"rubygems-2.0.14.1-35.el7_6.noarch.rpm",
"rubygem-json-1.7.7-35.el7_6.x86_64.rpm"

rhel-7-server-optional-debug-rpms:
"ruby-debuginfo-2.0.0.648-35.el7_6.x86_64.rpm",

After syncing these 2 repos to the Satellite, Pulp will only list the 'ruby-debuginfo rpm' in the erratum 'pkglist'. Pulp will calculate the wrong applicability for the consumer because the erratum is listing only 1 rpm.

Test in "foreman-rake console":

Before enabling debug repo, the erratum shows 10 packages:
pp Katello::pulp_server.resources.repository.unit_search("046aa6b4-6369-4ef7-a3f7-3959250bd86f", {:type_ids => ['erratum'], filters: {unit: {"errata_id": "RHSA-2019:1235"}}})[0]['metadata']['pkglist'][0]['packages'].size
10

After enabling and sync debug repo. the erratum shows only 1 package.
pp Katello::pulp_server.resources.repository.unit_search("046aa6b4-6369-4ef7-a3f7-3959250bd86f", {:type_ids => ['erratum'], filters: {unit: {"errata_id": "RHSA-2019:1235"}}})[0]['metadata']['pkglist'][0]['packages'].size
1

pp Katello::pulp_server.resources.repository.unit_search("046aa6b4-6369-4ef7-a3f7-3959250bd86f", {:type_ids => ['erratum'], filters: {unit: {"errata_id": "RHSA-2019:1235"}}})[0]['metadata']['pkglist'][0]['packages'][0]['filename']
"ruby-debuginfo-2.0.0.648-35.el7_6.x86_64.rpm"

exit

Steps to Reproduce:
1. Ensure you have a RHEL 7 client with older ruby version. Lets say Client 'A'.
2. Enable and sync 'rhel-7-server-rpms' repo
3. Go to Web UI -> Hosts -> Content Hosts -> Client 'A' -> Errata tab -> Filter "id = RHSA-2019:1235" should show this erratum is applicable/installable.
4. Enable and sync 'rhel-7-server-optional-debug-rpms' repo
5. Now need to do something so that we can FORCE recalculate the applicability for Client 'A'

On client 'A' do:
subscription-manager repos --enable="rhel-7-server-optional-debug-rpms"
katello-enabled-repos-upload -f

6. Wait for the regenerate applicability task to finish
7. Go to Web UI -> Hosts -> Content Hosts -> Client 'A' -> Errata tab -> Filter "id = RHSA-2019:1235" shows empty result.

Actual results:
Errata RHSA-2019:1235 is not applicable to Client A

Expected results:
Errata RHSA-2019:1235 is applicable to the Client A

Related issues

Associated revisions

Revision 91145f24 View on GitHub
Added by ttereshc 7 months ago

Fix serializer for erratum pkglists

It didn't affect publish results, however it did affect applicability
calculations and erratum content which user query via API.

Kudos to Hyu (reporter of the issue) for the patch.

closes #4880
https://pulp.plan.io/issues/4880

History

#1 Updated by hyu 8 months ago

I think we need to do "$unwind" twice to flatten the array list. I made the following change and test. It seems to fix the issue.

--- a/plugins/serializers.py 2019-05-23 18:32:32.312427907 1000
++ b/plugins/serializers.py 2019-05-23 19:48:38.839722218 +1000
@ -75,15 +75,16 @

match_stage = {'$match': {'errata_id': errata_id}}
             unwind_stage = {'$unwind': '$collections'}
+            unwind_stage2 = {'$unwind': '$collections.packages'}
             group_stage = {'$group': {'_id': '$collections.module',
                                       'packages': {'$addToSet': '$collections.packages'}}}
             collections = models.ErratumPkglist.objects.aggregate(
-                match_stage, unwind_stage, group_stage, allowDiskUse=True)
+                match_stage, unwind_stage, unwind_stage2, group_stage, allowDiskUse=True)
             for collection_idx, collection in enumerate(collections):
    # To preserve the original format of a pkglist the 'short' and 'name'
    # keys are added. 'short' can be an empty string, collection 'name'
    # should be unique within an erratum.
-                item = {'packages': collection['packages'][0],
+                item = {'packages': collection['packages'],
                         'short': '',
                         'name': 'collection-%s' % collection_idx}
                 if collection['_id']: