RPM Support

++ This bug was initially created as a clone of Bugzilla Bug #1041317 ++

Description of problem:

Created attachment 835839
Metadata file with duplicate packages that have the same name/version/release number but have different checksums

Description of problem:
Uploading to pulp an rpm with the same name/version/release but with a different checksum doesn't overwrite the existing one, it creates 2 versions of the same rpm package.

Version-Release number of selected component (if applicable):
pulp-server-2.2.0-1.el6.noarch running on CentOS 6.4

How reproducible:
Upload to a pulp repo the package <name>-0.0.1-70 , publish the pulp repo.
Upload the same <name>-0.0.1-70 package but with a different checksum.
On querying pulp will return 2 versions of the same package name and yum install, on a server using the pulp server as yum repo, will fail

[stefcoco@pulp-001 ~]$ pulp-admin rpm repo content rpm --repo-id cps-stable --match 'name=goggins' --match 'version=0.0.1' --match 'r
elease=70'
Arch: noarch
Buildhost: inf-jks-002.cpp.o.itv.net.uk
Checksum: 5f29f29217bf710260f04fa080bb199e49610851eb5806de2b95aed02416b464
Checksumtype: sha256
Description: Goggins java app
Epoch: 0
Filename: goggins-0.0.1-70.noarch.rpm
License: Proprietary
Name: goggins
Provides: config(goggins) = 0.0.1-70-0, goggins = 0.0.1-70-0
Release: 70
Requires: /bin/sh, /bin/sh, java-1.6.0-openjdk
Vendor: None
Version: 0.0.1

Arch: noarch
Buildhost: inf-jks-002.cpp.o.itv.net.uk
Checksum: d2dec47020faa72d01bb93117181659231bb3fd12334808afbb372198c91b284
Checksumtype: sha256
Description: Goggins java app
Epoch: 0
Filename: goggins-0.0.1-70.noarch.rpm
License: Proprietary
Name: goggins
Provides: config(goggins) = 0.0.1-70-0, goggins = 0.0.1-70-0
Release: 70
Requires: /bin/sh, /bin/sh, java-1.6.0-openjdk
Vendor: None
Version: 0.0.1

Yum install fails with:

[Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=itv-pulp-cps-stable clean metadata
Trying other mirror.

Error Downloading Packages:
goggins-0.0.1-70.noarch: failure: goggins-0.0.1-70.noarch.rpm from itv-pulp-cps-stable: [Errno 256] No more mirrors to try.

because yum metadata contains the same package twice with different checksums.
Have a look at the attached metadata files to see what I mean ( search for goggins rel=70 )

Steps to Reproduce:
1. Create a pulp repo
2. Upload an rpm to it and publish it
3. Upload a second rpm with the same name/version/release but with a different checksum and publish it
4. Try installing, on another server, from this repository via regular yum repos.

Actual results:
Pulp treats these packages as 2 different packages but yum install fails with

[Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=itv-pulp-cps-stable clean metadata
Trying other mirror.

Error Downloading Packages:
goggins-0.0.1-70.noarch: failure: goggins-0.0.1-70.noarch.rpm from itv-pulp-cps-stable: [Errno 256] No more mirrors to try.

Expected results:

Pulp should re-place the current package with the one that is being uploaded. That will create just one checksum inside the metadata files and yum would be able to install that package.

Additional info:

--- Additional comment from stefan.cocora@gmail.com at 12/12/2013 14:53:47 ---

The published repo, on disk, only contains one package version of goggins rel=70 but the metadata file contains 2 checksums which breaks yum.

[stefcoco@pulp-001 ~]$ ll /var/lib/pulp/published/http/repos/cps-stable/
total 60
lrwxrwxrwx 1 apache apache 144 Nov 27 16:58 dawkins-0.0.1-118.noarch.rpm -> /var/lib/pulp/content/rpm/dawkins/0.0.1/118/noarch/c5b2b5
b48ab700ef8f2c15fb46dff6623052b8e328b95a8fdf6728520c109bfd/dawkins-0.0.1-118.noarch.rpm
lrwxrwxrwx 1 apache apache 140 Nov 27 16:58 fnord-0.0.1-215.noarch.rpm -> /var/lib/pulp/content/rpm/fnord/0.0.1/215/noarch/a9a196e85d
95de9d9fdc1a6f675fdcc55f248b62f99514515b11a9546b9bee46/fnord-0.0.1-215.noarch.rpm
lrwxrwxrwx 1 apache apache 142 Nov 27 16:58 goggins-0.0.1-61.noarch.rpm -> /var/lib/pulp/content/rpm/goggins/0.0.1/61/noarch/6b1f0459
16b9aa6b6425f9249af15c3475325193d5b379cc5d270c1404c5fbef/goggins-0.0.1-61.noarch.rpm
lrwxrwxrwx 1 apache apache 142 Nov 29 16:01 goggins-0.0.1-65.noarch.rpm -> /var/lib/pulp/content/rpm/goggins/0.0.1/65/noarch/152fa050
6d259da3652ebb849f9832e65ed3db5804496c0ee4974891ff85eca1/goggins-0.0.1-65.noarch.rpm
lrwxrwxrwx 1 apache apache 142 Dec 12 12:15 goggins-0.0.1-67.noarch.rpm -> /var/lib/pulp/content/rpm/goggins/0.0.1/67/noarch/97df5a25
e4ee78c0cdcc2ae1b71a7b43e0acca8c87eabcca93133fdc1fec2caa/goggins-0.0.1-67.noarch.rpm
lrwxrwxrwx 1 apache apache 142 Dec 12 11:29 goggins-0.0.1-70.noarch.rpm -> /var/lib/pulp/content/rpm/goggins/0.0.1/70/noarch/d2dec470
20faa72d01bb93117181659231bb3fd12334808afbb372198c91b284/goggins-0.0.1-70.noarch.rpm
lrwxrwxrwx 1 apache apache 146 Nov 27 16:58 greendale-0.0.1-87.noarch.rpm -> /var/lib/pulp/content/rpm/greendale/0.0.1/87/noarch/f2b9
e97e9bd6214e6d04f9dda523d752bac9f6d2b13e98ff94ef17a2cdb1ce82/greendale-0.0.1-87.noarch.rpm
lrwxrwxrwx 1 apache apache 136 Nov 27 16:58 jess-0.0.1-58.noarch.rpm -> /var/lib/pulp/content/rpm/jess/0.0.1/58/noarch/1c7be39c2d4284
ece87f3eb0b1d98ad8102d102a9535559ed4c1cc26a60b4461/jess-0.0.1-58.noarch.rpm
lrwxrwxrwx 1 apache apache 142 Nov 27 16:58 minerva-0.0.1-40.noarch.rpm -> /var/lib/pulp/content/rpm/minerva/0.0.1/40/noarch/97d40821
de5be66b2fc6f04538b3185645ae21a1b90bdd9c71b5338ab25c5e28/minerva-0.0.1-40.noarch.rpm
lrwxrwxrwx 1 apache apache 67 Nov 27 14:22 Packages -> /var/lib/pulp/working/repos/cps-stable/distributors/yum_distributor
lrwxrwxrwx 1 apache apache 134 Nov 27 16:58 pat-0.0.1-75.noarch.rpm -> /var/lib/pulp/content/rpm/pat/0.0.1/75/noarch/c15cd481d2349fab
28af275f3a3374969f28829ac759f173981bed3fea2acf47/pat-0.0.1-75.noarch.rpm
drwxr-xr-x 2 apache apache 4096 Dec 12 12:17 repodata
drwxr-xr-x 2 apache apache 4096 Dec 12 12:15 repodata.old
lrwxrwxrwx 1 apache apache 148 Nov 27 16:58 swagger-ui-0.0.1-39.noarch.rpm -> /var/lib/pulp/content/rpm/swagger-ui/0.0.1/39/noarch/a4
5184d839b5459b1b9d4fc5f7d9089b1388be957be5d02560f0476308755c45/swagger-ui-0.0.1-39.noarch.rpm
lrwxrwxrwx 1 apache apache 140 Nov 27 16:58 venus-0.0.1-127.noarch.rpm -> /var/lib/pulp/content/rpm/venus/0.0.1/127/noarch/5878912453
6dca80510c3dbfe2a511644e6f0d82194e37a3c34f05a8981119c3/venus-0.0.1-127.noarch.rpm

--- Additional comment from bcourt@redhat.com at 12/12/2013 15:12:16 ---

To add clarification here. The problem is that pulp is using the checksum as part of the uniqueness key for an RPM. A shorter term fix could be to remove the checksum value from the uniqueness constraint and keep the checksum type as part of the uniqueness constraint.

In the long term the request requires decoupling the checksum type & value from the uniqueness key and having pulp generate multiple checksum types and the various checksum values for a particular rpm.

--- Additional comment from mhrivnak@redhat.com at 12/18/2013 17:09:47 ---

We will consider validation in the yum distributor at publish time as a future feature.