RPM Support

When uploading & importing an RPM into Pulp, if unit type ID "srpm" is requested, the unit is instead saved as "rpm".

This itself may not be a major problem, but the saved unit is also corrupt, containing incorrect filename and repodata fields.

Steps to reproduce¶

In development environment:

$ pwd
/home/vagrant/devel/pulp_rpm/plugins/test/data

# Look at checksum of this file for later search
$ sha256sum walrus-5.21-1.noarch.rpm 
e837a635cc99f967a70f34b268baa52e0f412c1502e08e924ff5b09f1f9573f2  walrus-5.21-1.noarch.rpm

# Use pulp-admin to upload it as "wrong" type srpm
$ pulp-admin rpm repo uploads srpm --repo-id zoo -f walrus-5.21-1.noarch.rpm
+----------------------------------------------------------------------+
                              Unit Upload
+----------------------------------------------------------------------+

Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: walrus-5.21-1.noarch.rpm
... completed

Creating upload requests on the server...
[==================================================] 100%
Initializing: walrus-5.21-1.noarch.rpm
... completed

Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.

Uploading: walrus-5.21-1.noarch.rpm
[==================================================] 100%
2445/2445 bytes
... completed

Importing into the repository...
This command may be exited via ctrl+c without affecting the request.

[\]
Running...

Task Succeeded

Deleting the upload request...
... completed

# search for uploaded unit in srpms collection => no result
$ curl -s -k -u admin:admin -d '{"criteria": {"filters": {"checksum": "e837a635cc99f967a70f34b268baa52e0f412c1502e08e924ff5b09f1f9573f2"}}}' https://localhost/pulp/api/v2/content/units/srpm/search/ | python -mjson.tool
[]

# search for uploaded unit in rpms collection => a result, but incorrect filename and repodata!
$ curl -s -k -u admin:admin -d '{"criteria": {"filters": {"checksum": "e837a635cc99f967a70f34b268baa52e0f412c1502e08e924ff5b09f1f9573f2"}}}' https://localhost/pulp/api/v2/content/units/rpm/search/ | python -mjson.tool
[
    {
        "_content_type_id": "rpm",
        "_href": "/pulp/api/v2/content/units/rpm/85291af5-dc1d-4731-b5d7-82f206169d7f/",
        "_id": "85291af5-dc1d-4731-b5d7-82f206169d7f",
        "_last_updated": "2019-04-03T00:22:52Z",
        "_storage_path": "/var/lib/pulp/content/units/rpm/cf/a9bb6ac796b5732019a4e2bb68ce26e8703154289bbe5635079dcfaac0373b/d96579ea-f3ec-41a5-a18b-88aede2f0350",
        "arch": "noarch",
        "build_time": 1331831368,
        "buildhost": "smqe-ws15",
        "changelog": [],
        "checksum": "e837a635cc99f967a70f34b268baa52e0f412c1502e08e924ff5b09f1f9573f2",
        "checksums": {
            "md5": "6a3eec6d45e0ea80eab05870bf7a8d4b",
            "sha1": "8dea2b64fc52062d79d5f96ba6415bffae4d2153",
            "sha256": "e837a635cc99f967a70f34b268baa52e0f412c1502e08e924ff5b09f1f9573f2"
        },
        "checksumtype": "sha256",
        "children": {},
        "description": "A dummy package of walrus",
        "downloaded": true,
        "epoch": "0",
        "file_size": 2445,
        "filename": "walrus-5.21-1.src.rpm",
        "files": {
            "dir": [],
            "file": [
                "/tmp/walrus.txt"
            ]
        },
        "group": "Internet/Applications",
        "header_range": {
            "end": 2293,
            "start": 872
        },
        "is_modular": false,
        "license": "GPLv2",
        "name": "walrus",
        "provides": [
            {
                "epoch": "0",
                "flags": "EQ",
                "name": "walrus",
                "release": "1",
                "version": "5.21"
            }
        ],
        "pulp_user_metadata": {},
        "recommends": [],
        "relativepath": "walrus-5.21-1.src.rpm",
        "release": "1",
        "release_sort_index": "01-1",
        "repodata": {
            "filelists": "<package arch=\"noarch\" name=\"walrus\" pkgid=\"{{ pkgid }}\">\n  <version epoch=\"0\" rel=\"1\" ver=\"5.21\" />\n  <file>/tmp/walrus.txt</file>\n</package>",
            "other": "<package arch=\"noarch\" name=\"walrus\" pkgid=\"{{ pkgid }}\">\n  <version epoch=\"0\" rel=\"1\" ver=\"5.21\" />\n</package>",
            "primary": "<package type=\"rpm\">\n  <name>walrus</name>\n  <arch>noarch</arch>\n  <version epoch=\"0\" rel=\"1\" ver=\"5.21\" />\n  <checksum pkgid=\"YES\" type=\"{{ checksumtype }}\">{{ checksum }}</checksum>\n  <summary>A dummy package of walrus</summary>\n  <description>A dummy package of walrus</description>\n  <packager />\n  <url>http://tstrachota.fedorapeople.org</url>\n  <time build=\"1331831368\" file=\"1554250972\" />\n  <size archive=\"296\" installed=\"42\" package=\"2445\" />\n  <location href=\"Packages/w/walrus-5.21-1.src.rpm\" />\n  <format>\n    <rpm:license>GPLv2</rpm:license>\n    <rpm:vendor />\n    <rpm:group>Internet/Applications</rpm:group>\n    <rpm:buildhost>smqe-ws15</rpm:buildhost>\n    <rpm:sourcerpm>walrus-5.21-1.src.rpm</rpm:sourcerpm>\n    <rpm:header-range end=\"2293\" start=\"872\" />\n    <rpm:provides>\n      <rpm:entry epoch=\"0\" flags=\"EQ\" name=\"walrus\" rel=\"1\" ver=\"5.21\" />\n    </rpm:provides>\n  </format>\n</package>"
        },
        "requires": [],
        "signature": "f78fb195",
        "signing_key": "f78fb195",
        "size": 2445,
        "sourcerpm": "walrus-5.21-1.src.rpm",
        "summary": "A dummy package of walrus",
        "time": 1554250972,
        "url": "http://tstrachota.fedorapeople.org",
        "version": "5.21",
        "version_sort_index": "01-5.02-21"
    }
]

Actual behavior¶

• Request to import/upload a non-source RPM as "srpm" succeeds
• Resulting unit is saved as "rpm"
• Unit has incorrect filename and repodata referring to the file as .src.rpm - duplicating with the real source RPM
• Resulting unit can't be fixed by repeating the upload as correct "rpm" type, because the unit key is duplicate

Expected behavior¶

One of the following:

• Either a request to save an RPM as srpm, and vice-versa, should fail
  • Note this is the way it was before https://pulp.plan.io/issues/2754. But code for that issue removed the checks for no clear reason, leaving error codes RPM1002, RPM1003 now unused.
• Or a request to save an RPM/SRPM should always ignore the user's requested type id and save it as a unit of the correct type as determined by Pulp internally

Additional info¶

Looking in pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/upload.py, _handle_package method, the logic there is inconsistent with respect to type_id.

# type_id comes from the user here...
def _handle_package(repo, type_id, unit_key, metadata, file_path, conduit, config):

    ...

    # but in here, the model and hence type ID comes from generating XML from the RPM
    # and parsing that - here, Pulp figures out for itself whether the unit should be RPM or SRPM
    try:
        if type_id == models.DRPM._content_type_id.default:
            unit = models.DRPM(**_extract_drpm_data(file_path))
        else:
            repodata = rpm_parse.get_package_xml(file_path, sumtype=util.TYPE_SHA256)
            package_xml = (utils.fake_xml_element(repodata['primary'], constants.COMMON_NAMESPACE)
                                .find(primary.PACKAGE_TAG))
            unit = primary.process_package_element(package_xml)

            package_headers = rpm_parse.package_headers(file_path)
            unit.is_modular = rpm_parse.get_package_modular_flag(package_headers)

    except Exception:
        raise PulpCodedException(error_codes.RPM1016)

    ...

    # Then later, there's a branch on the user's passed type_id rather than the type actually
    # being used for the save, this doesn't make sense! If user passed type_id='srpm' but Pulp is
    # actually saving a models.RPM, then this results in a bogus filename.

    if type_id != models.DRPM._content_type_id.default:
        # Extract/adjust the repodata snippets
        unit.signing_key = rpm_parse.package_signature(rpm_parse.package_headers(file_path))
        # construct filename from metadata (BZ #1101168)
        if type_id == models.SRPM._content_type_id.default:
            rpm_basefilename = "%s-%s-%s.src.rpm" % (unit.name, unit.version, unit.release)
        else:
            rpm_basefilename = "%s-%s-%s.%s.rpm" % (unit.name, unit.version, unit.release,
                                                    unit.arch)

        unit.relativepath = rpm_basefilename
        unit.filename = rpm_basefilename
        _update_files(unit, repodata)
        unit.modify_xml(repodata)