Issue #4157: Publish of RPM repo with lazy content and unavailable checksum type does not raise error - Pulp

Actions

Send by e-mail Copy link

Issue #4157

closed

Publish of RPM repo with lazy content and unavailable checksum type does not raise error

Added by kersom over 5 years ago. Updated about 5 years ago.

Status:

CLOSED - NOTABUG

Priority:

Normal

Assignee:

Category:

Sprint/Milestone:

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Version:

Platform Release:

OS:

Triaged:

Groomed:

Sprint Candidate:

Tags:

Pulp 2

Sprint:

Quarter:

Description

Filing this issue to keep everyone informed. I am trying to re-create the same error manually, and I will update as we find any extra info.

The test case UnavailableChecksumTestCase is failing on RPM type.

This test does:

        1. Create and sync repository of type RPM and with the given
           feed. Ensure the repository's importer has a deferred/lazy download
           policy.
        2. Determine which checksum type the units in the repository use. (As
           of this writing, Pulp can handle md5, sha1 and sha256 checksum
           types.)
        3. Publish the repository with checksum types different from what the
           units in the repository use. Assert the publish fails.

Last step of test failed: `AssertionError: TaskReportError not raised ` when publishing with a different checksum type.

[root@r76 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.6 (Maipo

[root@r76 ~]# rpm -qa | grep pulp
python-pulp-deb-common-1.9.0-0.1.alpha.201811071503gitb5fc87b.el7.noarch
python-pulp-repoauth-2.19.0-0.1.alpha.201811071455gita768094.el7.noarch
pulp-python-plugins-2.1.0-0.1.alpha.201811071439git5e2aa35.el7.noarch
pulp-ostree-admin-extensions-1.4.0-0.1.alpha.201811071443git53beed4.el7.noarch
pulp-puppet-admin-extensions-2.19.0-0.1.alpha.201811071446gitc54a546.el7.noarch
python-pulp-common-2.19.0-0.1.alpha.201811071455gita768094.el7.noarch
python-pulp-rpm-common-2.19.0-0.1.alpha.201811071450git21e5f95.el7.noarch
python-pulp-ostree-common-1.4.0-0.1.alpha.201811071443git53beed4.el7.noarch
python-pulp-docker-common-3.4.0-0.1.alpha.201811071441gita762419.el7.noarch
python-pulp-bindings-2.19.0-0.1.alpha.201811071455gita768094.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
python-pulp-puppet-common-2.19.0-0.1.alpha.201811071446gitc54a546.el7.noarch
python-pulp-python-common-2.1.0-0.1.alpha.201811071439git5e2aa35.el7.noarch
pulp-selinux-2.19.0-0.1.alpha.201811071455gita768094.el7.noarch
python-pulp-oid_validation-2.19.0-0.1.alpha.201811071455gita768094.el7.noarch
pulp-server-2.19.0-0.1.alpha.201811071455gita768094.el7.noarch
pulp-rpm-plugins-2.19.0-0.1.alpha.201811071450git21e5f95.el7.noarch
pulp-deb-plugins-1.9.0-0.1.alpha.201811071503gitb5fc87b.el7.noarch
pulp-docker-plugins-3.4.0-0.1.alpha.201811071441gita762419.el7.noarch
pulp-puppet-tools-2.19.0-0.1.alpha.201811071446gitc54a546.el7.noarch
python-pulp-client-lib-2.19.0-0.1.alpha.201811071455gita768094.el7.noarch
pulp-deb-admin-extensions-1.9.0-0.1.alpha.201811071503gitb5fc87b.el7.noarch
pulp-python-admin-extensions-2.1.0-0.1.alpha.201811071439git5e2aa35.el7.noarch
pulp-docker-admin-extensions-3.4.0-0.1.alpha.201811071441gita762419.el7.noarch
pulp-ostree-plugins-1.4.0-0.1.alpha.201811071443git53beed4.el7.noarch
pulp-puppet-plugins-2.19.0-0.1.alpha.201811071446gitc54a546.el7.noarch
pulp-admin-client-2.19.0-0.1.alpha.201811071455gita768094.el7.noarch
pulp-rpm-admin-extensions-2.19.0-0.1.alpha.201811071450git21e5f95.el7.noarch
python-pulp-streamer-2.19.0-0.1.alpha.201811091549gitb6f269d.el7.noarch

Actions

Copy link

Updated by bherring over 5 years ago

Manual Recreation example of the above test snippet:

pulp-admin rpm repo create --repo-id=zoo --feed=https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-unsigned/ --download-policy=on_demand
pulp-admin rpm repo sync run --repo-id=zoo
pulp-admin rpm repo update --repo-id=zoo --checksum-type=sha1
pulp-admin rpm repo publish run --repo-id=zoo

I believe an error should be expected on the publish since the `checksum-type=sha1` is a requested checksum that is not available in that repo.

Actions

Copy link

Updated by kersom over 5 years ago

Description updated (diff)

Actions

Copy link

Updated by daviddavis over 5 years ago

A couple things I noticed is that when I went through the steps bherring posted, the repo metadata had sha256 checksums.

When I ran publish with --force-full:

pulp-admin rpm repo publish run --repo-id=zoo --force-full

I hit an error:

Nov 14 18:28:38 pulp2.dev pulp[7757]: pulp.server.controllers.repository:ERROR: [14a4448e] (7757-12896) PulpCodedException: Checksum type "sha1" is not available for all units in the repository. Make sure those units have been downloaded.

Actions

Copy link

Updated by daviddavis over 5 years ago

FWIW, I was able to reproduce the problem on pulp_rpm 2.17 using @bherring's reproducer.

Actions

Copy link

Updated by kersom over 5 years ago

Some notes and troubleshooting results.

If you run this test in isolation will pass. However if you run other tests, that use the same feed URL, and run this test again, then this test will fail.
Added a call to clean orphans as part of the setupClass for test in failures, and this fixed the failure.

   @classmethod
    def setUpClass(cls):
        """Clean orphans."""
        api.Client(config.get_config()).delete(ORPHANS_PATH)

We have been re-factoring a few tests, and removing a base class that had as part of the tear-down - a call to clean orphans.

Actions

Copy link

Updated by kersom over 5 years ago

This failing test itself was not re-factored.

Actions

Copy link

Updated by kersom over 5 years ago

Description updated (diff)

Actions

Copy link

Updated by kersom over 5 years ago

Status changed from NEW to CLOSED - NOTABUG

Test was updated. It was included a call to clean orphan files before running the tests to assure that Pulp does not have those content units with a different checksum type.

Added by kersom over 5 years ago

Revision 2afc0de9 | View on GitHub

Update UnavailableChecksumTestCase

Add a setUpClass method and a call to clean orphan files to run before the tests are executed. This is necessary to assure that Pulp does not have those content units already with a different checksum type.

Ref: #4157 https://pulp.plan.io/issues/4157

Actions

Copy link