RPM Support

***** Bug 1098703 has been marked as a duplicate of this bug. ***

+ This comment was cloned from Bugzilla #1132659 comment 1 +

One way to fix this would be to add the unit key fields to the repo_content_units table so that an aggregation query could be used to check for duplicates.

Barnaby and I discussed this, it sounds like there are a few options:

• allow repos to be created with duplicate NEVRAs but raise an error when attempting to publish
• do not allow repos to be created with duplicate NEVRAs

The first option is something we can do today. The second option would require a lot more work since we currently don't have a way for a plugin to inject additional code into the association process. For example, if we wanted to perform a query to find an existing NEVRA for each unit during unit association, there's no way to do that today.

Additional functionality to support this could be added in Pulp 2.8 if needed. There may be alternatives such as using versioned repos which may fix this in a more general way.

For 2.7 we think checking for duplicate NEVRAs during publish and raising an error that contains a list of the duplicates is best. It would be better to not allow the user to get into this state to begin with but I think this is a good first step since it catches the issue in Pulp and not on the clients.

cduryee wrote:

For 2.7 we think checking for duplicate NEVRAs during publish and raising an error that contains a list of the duplicates is best. It would be better to not allow the user to get into this state to begin with but I think this is a good first step since it catches the issue in Pulp and not on the clients.

Making this type of change in 2.7 would be good. Perhaps that is what this bug could be about?

Separate from that how can I learn more about the "duplicate NVREA problem" in general. I keep hearing about it, but I don't fully understand it.

The downstream bz has a bit more info, but the problem is basically if you create a repo somewhere and sync it down into pulp, then sign the rpms and sync/publish the repo, Pulp will attempt to publish both the signed and unsigned RPMs (which have the same filename but different checksums). The last one will win, which can cause issues since the rpm may no longer have the same checksum as what the repodata thinks it should have.

I've seen this firsthand on other systems but have not reproduced it myself.

I can confirm this issue, we were using EPEL third party repository and it prevented our machines from retrieving the correct RPM version.

In the case of a person republishing a RPM of identical version but differing content. This caused machines in the field to fail to update those packages or retrieve them at all. This should be handled by matching the the RPM to the metadata in the remote repository, so the correct one is provided to clients of the pulp repository.

Otherwise this will end up being chaos when a user accidentally creates a new RPM without bumping the version then populates a custom repository or third party repository. We as users do not have control of upstream RPM repositories like EPEL, but it shouldn't prevent the clients connecting to pulp from being served the correct package that matches the metadata in the repository.

Just walked another user through fixing this

jsherril@redhat.com wrote:

Just walked another user through fixing this

There are two repository options related to cleaning packages:

Repository Contents Behavior
--remove-missing - if "true", units that were previously in the external
feed but are no longer found will be removed from the
repository; defaults to false
--retain-old-count - count indicating how many non-latest versions of a unit
to keep in a repository

How are these two options related? How do they combine?
Shouldn't retain-old-count 0 (default) imply remove-missing?
What's the difference between retain-old-count 5 with and without remove-missing?

--retain-old count only considers how many versions of a package are currently in the pulp repo.

--remove-missing only considers whether a package currently exists in the remote repo.

To solve this, let's start by making the sync process remove a package from the pulp repo if a different package with the same nevra is retrieved from the remote feed.

I suggest adding a call before this line: https://github.com/pulp/pulp_rpm/blob/2.6-dev/plugins/pulp_rpm/plugins/importers/yum/listener.py#L85

that removes any units from the repo that have the same NEVRA.