Project

Profile

Help

Issue #1556

Not able install rpm from an uploaded repo

Added by pthomas@redhat.com over 5 years ago. Updated over 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
2.8.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Not able to install rpm from a repo with rpm uploaded and pulished

1. Create a repo
2. Upload a rpm and publish the repo
3. Register a consumer/bind the repo to the the consumer
4. Run rpm install

[root@mgmt6 ~]# pulp-admin rpm consumer package install run   --consumer-id mypulp -n pulp-dot
Install task created with id [ 1c306224-ef7e-4668-ace2-de40fea2b1bb ]

This command may be exited via ctrl+c without affecting the request.

Refresh Repository Metadata             [ OK ]
Downloading Packages                    [ FAILED ]

Install Failed

[u'Errors were encountered while downloading packages.',
u'pulp-dot-2.0-test-0.1.2-1.fc11.x86_64: failure:
4995dc91-dd73-4d22-9f7b-df22e4666511 from upload: [Errno 256] No more mirrors to
try.']
Jan 19 16:57:49 mgmt6 pulp: pulp.server.agent.direct.services:INFO: (23478-98976) {'reboot': {'scheduled': False, 'details': {}}, 'details': {'rpm': {'details': {'message': "[u'Errors were encountered while downloading packages.', u'pulp-dot-2.0-test-0.1.2-1.fc11.x86_64: failure: 4995dc91-dd73-4d22-9f7b-df22e4666511 from upload: [Errno 256] No more mirrors to try.']", 'trace': 'Traceback (most recent call last):\n\n  File "/usr/lib/python2.6/site-packages/pulp/agent/lib/dispatcher.py", line 61, in install\n    _report = handler.install(conduit, units, dict(options))\n\n  File "/usr/lib/python2.6/site-packages/pulp_rpm/handlers/rpm.py", line 100, in install\n    details = pkg.install(names)\n\n  File "/usr/lib/python2.6/site-packages/pulp_rpm/handlers/rpmtools.py", line 159, in install\n    yb.processTransaction()\n\n  File "/usr/lib/python2.6/site-packages/pulp_rpm/handlers/rpmtools.py", line 604, in processTransaction\n    YumBase.processTransaction(self, callback, rpmDisplay=display)\n\n  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 5046, in processTransaction\n    pkgs = self._downloadPackages(callback)\n\n  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 5081, in _downloadPackages\n    raise Errors.YumDownloadError, errstr\n\nYumDownloadError: [u\'Errors were encountered while downloading packages.\', u\'pulp-dot-2.0-test-0.1.2-1.fc11.x86_64: failure: 4995dc91-dd73-4d22-9f7b-df22e4666511 from upload: [Errno 256] No more mirrors to try.\']\n'}, 'succeeded': False}}, 'succeeded': False, 'num_changes': 0}
>

Associated revisions

Revision 4ec4261e View on GitHub
Added by semyers over 5 years ago

set uploaded unit href to unit filename in repo

When uploading a unit, the href in the generated xml's location tag is based on the content unit file path, which is an autogenerated unique value. This sets the href back to the unit filename, which matches the published repo layout and allows uploaded units to again be downloaded from the published repo.

fixes #1556 https://pulp.plan.io/issues/1556

History

#1 Updated by jortel@redhat.com over 5 years ago

  • Project changed from Pulp to RPM Support
  • Category deleted (15)
  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Platform Release set to 2.8.0
  • Triaged changed from No to Yes

#2 Updated by jortel@redhat.com over 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to jortel@redhat.com

#3 Updated by jortel@redhat.com over 5 years ago

  • Status changed from ASSIGNED to NEW
  • Assignee deleted (jortel@redhat.com)

#4 Updated by semyers over 5 years ago

  • Status changed from NEW to ASSIGNED

#5 Updated by semyers over 5 years ago

After updating to the latest pulp_rpm and pulp platform, I'm unable to reproduce this.

Can you try again and see if this has been fixed?

#6 Updated by ipanova@redhat.com over 5 years ago

  • Assignee set to semyers

#7 Updated by ipanova@redhat.com over 5 years ago

$ sudo yum install wolf
Loaded plugins: langpacks, pulp-profile-update
adobe-linux-x86_64                                                                                                                                                     |  951 B  00:00:00     
bluejeans                                                                                                                                                              | 2.9 kB  00:00:00     
docker-main-repo                                                                                                                                                       | 2.9 kB  00:00:00     
http://repos.fedorapeople.org/repos/pulpdist/pulpdist/fedora-21/x86_64/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found
Trying other mirror.
google-chrome                                                                                                                                                          |  951 B  00:00:00     
pulp-2-stable                                                                                                                                                          | 3.6 kB  00:00:00     
pulp-2.5-testing                                                                                                                                                       | 3.6 kB  00:00:00     
rpmfusion-free-updates                                                                                                                                                 | 2.7 kB  00:00:00     
updates                                                                                                                                                                | 4.7 kB  00:00:00     
updates-testing/21/x86_64/metalink                                                                                                                                     |  22 kB  00:00:00     
upload-repo                                                                                                                                                            | 2.1 kB  00:00:00     
(1/3): upload-repo/group                                                                                                                                               |  130 B  00:00:00     
(2/3): upload-repo/primary                                                                                                                                             |  619 B  00:00:00     
(3/3): upload-repo/updateinfo                                                                                                                                          |   93 B  00:00:05     
upload-repo                                                                                                                                                                               1/1
Resolving Dependencies
--> Running transaction check
---> Package wolf.noarch 0:9.4-2 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================
 Package                                    Arch                                         Version                                      Repository                                         Size
==============================================================================================================================================================================================
Installing:
 wolf                                       noarch                                       9.4-2                                        upload-repo                                       2.4 k

Transaction Summary
==============================================================================================================================================================================================
Install  1 Package

Total download size: 2.4 k
Installed size: 42  
Is this ok [y/d/N]: y
Downloading packages:
b3ac6f23-6a35-463f-a967-f678d7 FAILED                                          
https://example.com/pulp/repos/upload-repo/b3ac6f23-6a35-463f-a967-f678d760a9ed: [Errno 14] HTTPS Error 404 - Not Found                      ]  0.0 B/s |    0 B  --:--:-- ETA 
Trying other mirror.

Error downloading packages:
  wolf-9.4-2.noarch: [Errno 256] No more mirrors to try.

]$ pulp-admin rpm consumer package install run --consumer-id c1 -n wolf

Install task created with id [ 632a10b9-ef32-49c4-9f6c-1b3cc1351b52 ]

This command may be exited via ctrl+c without affecting the request.

Refresh Repository Metadata             [ OK ]
Downloading Packages                    [ FAILED ]

Install Failed

[u'Errors were encountered while downloading packages.', u'wolf-9.4-2.noarch:
[Errno 256] No more mirrors to try.']

I noticed that it looks for https://example.com/pulp/repos/upload-repo/b3ac6f23-6a35-463f-a967-f678d760a9ed and instead b3ac6f23-6a35-463f-a967-f678d760a9ed i see the name of the package:

ll /var/www/pub/yum/https/repos/upload-repo/
total 8
drwxr-sr-x. 2 apache apache 4096 Feb  2 10:23 repodata
lrwxrwxrwx. 1 apache apache  110 Feb  2 10:23 wolf-9.4-2.noarch.rpm -> /var/lib/pulp/content/units/rpm/4e62/4e6211d4-57a3-4ddc-b199-37a3f7dc1e67/b3ac6f23-6a35-463f-a967-f678d760a9ed

#8 Updated by semyers over 5 years ago

Based on what you get with a "normal" sync/publish, I'm assuming that the name of the package is what should actually be in the repo metadata. Instead, here's what I get on publishing a repo with an uploaded rpm:

<package type="rpm">
  <name>shark</name>
  <arch>noarch</arch>
  <version epoch="0" ver="0.1" rel="1"/>
  <checksum type="sha256" pkgid="YES">951e0eacf3e6e6102b10acb2e689243b5866ec2c7720e783749dbd32f4a69ab3</checksum>
  <summary>A dummy package of shark</summary>
  <description>A dummy package of shark</description>
  <packager></packager>
  <url>http://tstrachota.fedorapeople.org</url>
  <time file="1454607148" build="1331831369"/>
  <size package="2441" installed="42" archive="296"/>
  <location href="4c47548d-5d8f-49d3-bb9c-b99f4dd63a70"/>
  <format>
    <rpm:license>GPLv2</rpm:license>
    <rpm:vendor/>
    <rpm:group>Internet/Applications</rpm:group>
    <rpm:buildhost>smqe-ws15</rpm:buildhost>
    <rpm:sourcerpm>shark-0.1-1.src.rpm</rpm:sourcerpm>
    <rpm:header-range start="872" end="2289"/>
    <rpm:provides>
      <rpm:entry name="shark" flags="EQ" epoch="0" ver="0.1" rel="1"/>
    </rpm:provides>
  </format>
</package>

Note the very wrong location tag.

Here's the same package metadata as seen in a synced/published zoo repo with the correct location tag:

<package type="rpm">
  <name>shark</name>
  <arch>noarch</arch>
  <version epoch="0" rel="1" ver="0.1" />
  <checksum pkgid="YES" type="sha256">951e0eacf3e6e6102b10acb2e689243b5866ec2c7720e783749dbd32f4a69ab3</checksum>
  <summary>A dummy package of shark</summary>
  <description>A dummy package of shark</description>
  <packager />
  <url>http://tstrachota.fedorapeople.org</url>
  <time build="1331831369" file="1331832459" />
  <size archive="296" installed="42" package="2441" />
<location href="shark-0.1-1.noarch.rpm"/>
  <format>
    <rpm:license>GPLv2</rpm:license>
    <rpm:vendor />
    <rpm:group>Internet/Applications</rpm:group>
    <rpm:buildhost>smqe-ws15</rpm:buildhost>
    <rpm:sourcerpm>shark-0.1-1.src.rpm</rpm:sourcerpm>
    <rpm:header-range end="2289" start="872" />
    <rpm:provides>
      <rpm:entry epoch="0" flags="EQ" name="shark" rel="1" ver="0.1" />
    </rpm:provides>
  </format>
</package>

The difference in indentation of the location tag might be notable, and exists in the pulp-generated metadata files.

Now to find out why... :)

#9 Updated by semyers over 5 years ago

  • Status changed from ASSIGNED to POST

https://github.com/pulp/pulp_rpm/pull/793

The difference in indentation actually did turn out to be a clue. We rather rudely rewrite that line when we get the metadata from a repo, but didn't do anything with it when generating the package xml from an uploaded RPM. My PR employs similar XML rewriting for uploaded units (without messing up the indentation)

#10 Updated by semyers over 5 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#11 Updated by dkliban@redhat.com over 5 years ago

  • Status changed from MODIFIED to 5

#12 Updated by pthomas@redhat.com over 5 years ago

  • Status changed from 5 to 6

verified
[root@mgmt5 ~]# rpm -qa pulp-server
pulp-server-2.8.0-0.9.rc.el7.noarch
[root@mgmt5 ~]#
[root@mgmt5 ~]# pulp-admin rpm consumer package install run --consumer-id starfish --name pulp-test-package
Install task created with id [ 17d3205c-bf6a-4d0c-b2a6-f4b63ba67d22 ]

This command may be exited via ctrl+c without affecting the request.

Refresh Repository Metadata [ OK ]
Downloading Packages [ OK ]
Check Package Signatures [ OK ]
Running Test Transaction [ OK ]
Running Transaction [ OK ]

Install Succeeded

--------------------------------------------------------------------
Installed
--------------------------------------------------------------------

Name: pulp-test-package
Version: 0.2.1
Arch: x86_64
Repoid: upload

#13 Updated by dkliban@redhat.com over 5 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE

#15 Updated by bmbouter over 2 years ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF