Pulp

Hello!

I'm attempting to import an errata into multiple Pulp repositories. The errata applies to multiple EL releases- namely CentOS EL 6 & CentOS EL 7. For this specific case, I'm using the Pulp REST API to import CESA-2017:1204 but this occurs with any errata that applies to multiple repositories.

It imported fine into the EL 6 repository, I'm only getting the exception when trying to import the same errata into the EL 7 repository.

This bug affects the ability to have a single erratum apply to multiple repositories.

I'm using pulp-rpm-plugins-2.13.2-1.el7.noarch with foreman-1.15.2-1.el7.noarch and katello-3.4.3-1.el7.noarch.

It looks like when Pulp encounters importing an erratum that exists already, it's supposed to merge the two and stamp the additional repository onto the already-existing erratum.

Here's the traceback:

pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400) unexpected error occurred importing uploaded file: 'name'
pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400) Traceback (most recent call last):
pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/upload.py", line 118, in upload
pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400)     handlers[type_id](repo, type_id, unit_key, metadata, file_path, conduit, config)
pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/upload.py", line 198, in _handle_erratum
pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400)     existing_unit.merge_errata(new_unit)
pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py", line 1197, in merge_errata
pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400)     self.merge_pkglists_and_save(other)
pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py", line 1258, in merge_pkglists_and_save
pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400)     package_name = p['name']
pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400)   File "/usr/lib/python2.7/site-packages/mongoengine/base/datastructures.py", line 27, in __getitem__
pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400)     value = super(BaseDict, self).__getitem__(key)
pulp_rpm.plugins.importers.yum.upload:ERROR: [669709d5] (2818-98400) KeyError: 'name'
pulp.server.managers.content.upload:ERROR: [669709d5] (2818-98400) Error from the importer while importing uploaded unit to repository [PGDx-CentOS_7-CentOS_7_3_1611_Updates]
pulp.server.managers.content.upload:ERROR: [669709d5] (2818-98400) Traceback (most recent call last):
pulp.server.managers.content.upload:ERROR: [669709d5] (2818-98400)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/content/upload.py", line 223, in import_uploaded_unit
pulp.server.managers.content.upload:ERROR: [669709d5] (2818-98400)     unit_type=unit_type_id, summary=result['summary'], details=result['details']
pulp.server.managers.content.upload:ERROR: [669709d5] (2818-98400) PulpCodedException: The importer yum_importer indicated a failed response when uploading erratum unit to repository PGDx-CentOS_7-CentOS_7_3_1611_Updates.

The code references this method in /usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py line 1258:

    def merge_pkglists_and_save(self, other):
        """
        Merge pkglists of the two errata and save the result to the database.

         - update existing collection if the other collection is newer and from the same
           repository
         - add _pulp_repo_id to old collection if packages are the same
         - otherwise add a new collection

        :param other: The erratum we are combining with the existing one
        :type  other: pulp_rpm.plugins.db.models.Errata

        """
        existing_pkglist_map = {}
        for idx, p in enumerate(self.pkglist):
            package_name = p['name']
            package_repo_id = p.get('_pulp_repo_id')
            pkglist_key = (package_name, package_repo_id)
            existing_pkglist_map[pkglist_key] = idx

I think the issue is that self.pkglist has a packages list that contains dictionaries with the name key. This list iterator doesn't access the packages key of self.pkglist first.

There's similar code at line 1592 of that same file:

    def rpm_search_dicts(self):
        ret = []
        for collection in self.pkglist:
            for package in collection.get('packages', []):
                if len(package.get('sum') or []) == 2:

Here, you can see it's reading information from packages after accessing self.pkglist.

I could not find equivalent code in the GitHub repository tag 2.13-release

I'm not sure why the 2.13 RPM is different, or where that code lives (if anywhere).