Project

Profile

Help

Issue #8713

Pulp 3 to Pulp 2 sync fails if comps.xml has a group with an empty packagelist

Added by ttereshc 5 months ago. Updated 22 days ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 101
Quarter:

Description

To reproduce:

  • Pulp 3: sync/publish/distribute RHEL8 Appstream (which has a group "conflicts-appstream" which has an empty packagelist)
  • Pulp 2: sync from the repo above, observe the following error
 'NoneType' object has no attribute 'findall'
 Traceback (most recent call last):
   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 321, in run
     group.GROUP_TAG)
   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 950, in get_comps_file_units
     self.save_fileless_units(group_file_handle, tag, process_func, mutable_type=True)
   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 1006, in save_fileless_units
     for model in package_info_generator:
   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/packages.py", line 64, in package_list_generator
     package_info = process_func(element)
   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/group.py", line 32, in process_group_element
     packagelist.findall('packagereq'))
 AttributeError: 'NoneType' object has no attribute 'findall'
INFO: [ac1cd020] Task failed : [ac1cd020-c06c-4d53-ac08-1adbab8c41b1]
 Task pulp.server.managers.repo.sync.sync[ac1cd020-c06c-4d53-ac08-1adbab8c41b1] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
 Traceback (most recent call last):
   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
     R = retval = fun(*args, **kwargs) 
   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 688, in __call__
     return super(Task, self).__call__(*args, **kwargs)
   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 110, in __call__
     return super(PulpTask, self).__call__(*args, **kwargs)
   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
     return self.run(*args, **kwargs)
   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 860, in sync
     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
 PulpExecutionException: Importer indicated a failed response

Pulp 3 uses libcomps to generate a comps.xml file and it doesn't generate empty XML elements.
Pulp 2 expects packagelist to be always present, even if empty.
It also seems to be a mandatory however it is according to the unofficial spec.


Related issues

Copied to RPM Support - Backport #9193: Backport #8713 "Pulp 3 to Pulp 2 sync fails if comps.xml has a group with an empty packagelist" to 3.14.zCLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Copied to RPM Support - Backport #9195: Backport #8713 "Pulp 3 to Pulp 2 sync fails if comps.xml has a group with an empty packagelist" to 3.11.zCLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 7673cab7 View on GitHub
Added by ttereshc about 2 months ago

Configure createrepo_c to always generate packagelist element within a package group.

Pulp 2 relies on the packagelist element being present, even if it's empty. It also seems to be a mandatory element according to this unofficial spec https://pagure.io/rpm-metadata/blob/master/f/fedora/comps-schema/comps.dtd#_6

Before this fix, the packagelist element would only be generated if there is at least one package in a group.

closes #8713 https://pulp.plan.io/issues/8713

History

#1 Updated by ttereshc 5 months ago

While we are figuring out where to fix it (hopefully in libcomps), here is patch to workaround the problem if you really need it.
It ignores empty groups, they likely do not bring much value, so as a workaround I think it's fine to skip them.

diff --git a/pulp_rpm/app/tasks/publishing.py b/pulp_rpm/app/tasks/publishing.py
index 1604a48..3528808 100644
--- a/pulp_rpm/app/tasks/publishing.py
+++ b/pulp_rpm/app/tasks/publishing.py
@@ -501,7 +501,7 @@ def create_repomd_xml(
 
     comps.toxml_f(
         comps_xml_path,
-        xml_options={"default_explicit": True, "empty_groups": True, "uservisible_explicit": True},
+        xml_options={"default_explicit": True, "empty_groups": False, "uservisible_explicit": True},
     )
 
     pri_xml.close()


#4 Updated by dalley 4 months ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 97

#5 Updated by rchan 4 months ago

  • Sprint changed from Sprint 97 to Sprint 98

#6 Updated by rchan 3 months ago

  • Sprint changed from Sprint 98 to Sprint 99

#7 Updated by rchan 3 months ago

  • Sprint changed from Sprint 99 to Sprint 100

#8 Updated by rchan 2 months ago

  • Sprint changed from Sprint 100 to Sprint 101

#9 Updated by ttereshc about 2 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc

#10 Updated by pulpbot about 2 months ago

  • Status changed from ASSIGNED to POST

#11 Updated by ttereshc about 2 months ago

  • Status changed from POST to MODIFIED

#12 Updated by dalley about 2 months ago

  • Copied to Backport #9193: Backport #8713 "Pulp 3 to Pulp 2 sync fails if comps.xml has a group with an empty packagelist" to 3.14.z added

#13 Updated by dalley about 2 months ago

  • Sprint/Milestone set to 3.15.0

#15 Updated by ttereshc about 2 months ago

  • Copied to Backport #9195: Backport #8713 "Pulp 3 to Pulp 2 sync fails if comps.xml has a group with an empty packagelist" to 3.11.z added

#16 Updated by pulpbot 22 days ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF