Project

Profile

Help

Issue #2276

closed

Can't kickstart against an on_demand file:// repo

Added by jsherril@redhat.com over 7 years ago. Updated almost 5 years ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.8.7
Platform Release:
OS:
CentOS 7
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

A katello user reported an issue with an on_demand repository. During kickstart he gets the error:

 16:45:42,449 INFO packaging: 323 packages selected totalling 1179.47 MiB
 16:45:42,496 INFO packaging: Running anaconda-yum to install packages
 16:45:42,996 INFO packaging:  populate transaction set
 16:54:15,002 ERR packaging:  error populating transaction after 10 retries: failure: tar-1.26-29.el7.x86_64.rpm from anaconda:

Watching the apache logs, the requests from anaconda look a bit strange:

        172.16.246.118 - - [21/Sep/2016:11:46:46 -0500] "GET /pulp/repos/Centric_Group/Library/custom/oel/o7ks/tar-1.26-29.el7.x86_64.rpm HTTP/1.1" 302 - "-" "urlgrabber/3.10 yum/3.4.3"
        172.16.246.118 - - [21/Sep/2016:11:46:46 -0500] "GET /streamer/var/lib/pulp/content/units/rpm/9d/01dcf30886eea92cb007017bdb36159cddd565cd2a6d1cc9174eefadb63b86/tar-1.26-29.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4xMTgifSwgInJlc291cmNlIjogIi9zdHJlYW1lci92YXIvbGliL3B1bHAvY29udGVudC91bml0cy9ycG0vOWQvMDFkY2YzMDg4NmVlYTkyY2IwMDcwMTdiZGIzNjE1OWNkZGQ1NjVjZDJhNmQxY2M5MTc0ZWVmYWRiNjNiODYvdGFyLTEuMjYtMjkuZWw3Lng4Nl82NC5ycG0iLCAiZXhwaXJhdGlvbiI6IDE0NzQ0NzY0OTZ9;signature=CFuH5ePYDRkMqH4SEqdhisEE4eZo7ojCTVPpU7d96TRRXPPgl7p03yjs7jHKITZeoxHpnd2RMEyoTCWkmoq7RDMG419TJvjtisJ4N5-6BiXXz1DgCoyROxTIIWXI5LpM0ZAAZHY8Bv37IlY1jkrHz4Efwzx5bZYlENG7ylDiLjhwjiJ-eQzOTnzHPV_2zvwaFsYv2MmlCYhhzWTrX7IQZRxOSWRg5Wmn52tPpE7gnWFsOtwh51CUATIvpDJ_AEm0Y1R0TfeWMWy-riRYv9cWehMrOdXwGnY3eSjastmrhpVzU8P-tNhLTFbXhsfvXVHMu6UmrpTTfh-RB87eV8cwQg%3D%3D HTTP/1.1" 200 253952 "-" "urlgrabber/3.10 yum/3.4.3"

mainly the '253952' part. Every time a kickstart is attempted it seems to be a different size. All these sizes have been observed:

57344 143360 208896 282624 237568 262144 135168 73728 57344 49152 73728

curl'ing the rpm results in the file properly downloading and it being the correct size:

$ curl -vvvv -L http://katello.domain.com/pulp/repos/domain_Group/Library/custom/oel/o7ks/tar-1.26-29.el7.x86_64.rpm > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 172.16.246.31...
* Connected to katello.domain.com (172.16.246.31) port 80 (#0)
> GET /pulp/repos/Domain_Group/Library/custom/oel/o7ks/tar-1.26-29.el7.x86_64.rpm HTTP/1.1
> Host: katello.domain.com
> User-Agent: curl/7.47.1
> Accept: */*
> 
< HTTP/1.1 302 FOUND
< Date: Wed, 21 Sep 2016 21:39:51 GMT
< Server: Apache/2.4.6 (CentOS)
< Content-Length: 0
< Location: http://katello.domain.com:80/streamer/var/lib/pulp/content/units/rpm/9d/01dcf30886eea92cb007017bdb36159cddd565cd2a6d1cc9174eefadb63b86/tar-1.26-29.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4yNDIifSwgInJlc291cmNlIjogIi9zdHJlYW1lci92YXIvbGliL3B1bHAvY29udGVudC91bml0cy9ycG0vOWQvMDFkY2YzMDg4NmVlYTkyY2IwMDcwMTdiZGIzNjE1OWNkZGQ1NjVjZDJhNmQxY2M5MTc0ZWVmYWRiNjNiODYvdGFyLTEuMjYtMjkuZWw3Lng4Nl82NC5ycG0iLCAiZXhwaXJhdGlvbiI6IDE0NzQ0OTQwODF9;signature=ccIk_3mWKbBQgY04tX7QW2Ns6F_9Uh6i_Uz1wAv1Vg_Vg3rvjZQ2LIXBq3cncjgLww-JA4nzcEJTX05hENClivxKKiQEOfWWlLZpdeyK7o4XLTf041whrpWa1xdYC8zwO99mIhKGxXsUuNozJ7c66ZrSy22h4YAgsS9-twBuzeqeI6a2D_MoIa9Z0G9IGN6zvYz7vsD_XCrEqfHEbYIE7UQiUm3zy5XvwM-kmiAHiIs3Fir8WZ4CBA_e1V_UhNgheXAmH5C7VLEkCi0f_PtB3jixtGdmH3oVlM9W1OtSyNOXuVmErWsgIBq-awLUWJ5S57DAvUyfmn56ug8U--zvKw%3D%3D
< Content-Type: text/html; charset=utf-8
< 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Connection #0 to host katello.domain.com left intact
* Issue another request to this URL: 'http://katello.domain.com:80/streamer/var/lib/pulp/content/units/rpm/9d/01dcf30886eea92cb007017bdb36159cddd565cd2a6d1cc9174eefadb63b86/tar-1.26-29.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4yNDIifSwgInJlc291cmNlIjogIi9zdHJlYW1lci92YXIvbGliL3B1bHAvY29udGVudC91bml0cy9ycG0vOWQvMDFkY2YzMDg4NmVlYTkyY2IwMDcwMTdiZGIzNjE1OWNkZGQ1NjVjZDJhNmQxY2M5MTc0ZWVmYWRiNjNiODYvdGFyLTEuMjYtMjkuZWw3Lng4Nl82NC5ycG0iLCAiZXhwaXJhdGlvbiI6IDE0NzQ0OTQwODF9;signature=ccIk_3mWKbBQgY04tX7QW2Ns6F_9Uh6i_Uz1wAv1Vg_Vg3rvjZQ2LIXBq3cncjgLww-JA4nzcEJTX05hENClivxKKiQEOfWWlLZpdeyK7o4XLTf041whrpWa1xdYC8zwO99mIhKGxXsUuNozJ7c66ZrSy22h4YAgsS9-twBuzeqeI6a2D_MoIa9Z0G9IGN6zvYz7vsD_XCrEqfHEbYIE7UQiUm3zy5XvwM-kmiAHiIs3Fir8WZ4CBA_e1V_UhNgheXAmH5C7VLEkCi0f_PtB3jixtGdmH3oVlM9W1OtSyNOXuVmErWsgIBq-awLUWJ5S57DAvUyfmn56ug8U--zvKw%3D%3D'
* Found bundle for host katello.domain.com: 0x55abc3b5d4c0 [can pipeline]
* Re-using existing connection! (#0) with host katello.domain.com
* Connected to katello.domain.com (172.16.246.31) port 80 (#0)
> GET /streamer/var/lib/pulp/content/units/rpm/9d/01dcf30886eea92cb007017bdb36159cddd565cd2a6d1cc9174eefadb63b86/tar-1.26-29.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4yNDIifSwgInJlc291cmNlIjogIi9zdHJlYW1lci92YXIvbGliL3B1bHAvY29udGVudC91bml0cy9ycG0vOWQvMDFkY2YzMDg4NmVlYTkyY2IwMDcwMTdiZGIzNjE1OWNkZGQ1NjVjZDJhNmQxY2M5MTc0ZWVmYWRiNjNiODYvdGFyLTEuMjYtMjkuZWw3Lng4Nl82NC5ycG0iLCAiZXhwaXJhdGlvbiI6IDE0NzQ0OTQwODF9;signature=ccIk_3mWKbBQgY04tX7QW2Ns6F_9Uh6i_Uz1wAv1Vg_Vg3rvjZQ2LIXBq3cncjgLww-JA4nzcEJTX05hENClivxKKiQEOfWWlLZpdeyK7o4XLTf041whrpWa1xdYC8zwO99mIhKGxXsUuNozJ7c66ZrSy22h4YAgsS9-twBuzeqeI6a2D_MoIa9Z0G9IGN6zvYz7vsD_XCrEqfHEbYIE7UQiUm3zy5XvwM-kmiAHiIs3Fir8WZ4CBA_e1V_UhNgheXAmH5C7VLEkCi0f_PtB3jixtGdmH3oVlM9W1OtSyNOXuVmErWsgIBq-awLUWJ5S57DAvUyfmn56ug8U--zvKw%3D%3D HTTP/1.1
> Host: katello.domain.com
> User-Agent: curl/7.47.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Wed, 21 Sep 2016 21:39:51 GMT
< Server: TwistedWeb/12.2.0
< Content-Type: text/html; charset=UTF-8
< X-Cache: MISS from katello.domain.com
< X-Cache-Lookup: MISS from katello.domain.com:3128
< Via: 1.1 katello.domain.com (squid/3.3.8)
< Via: 1.1 katello.domain.com
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< 
{ [12312 bytes data]
100  841k    0  841k    0     0  20.2M      0 --:--:-- --:--:-- --:--:-- 20.2M
* Connection #0 to host katello.domain.com left intact

Notice that the Content-Type coming from the server is 'Content-Type: text/html; charset=UTF-8'. Which does not seem correct. When i try it on my server I get "Content-Type: application/x-rpm".

In addition I have a tcpdump of the kickstart starting. Its ~270 MB in size, so i won't upload it here. I am uploading a trimmed version showing just this one file request (at least the initial part).

The kickstart in question is Oracle Linux 7.2


Files

trimmed.bin.pcap (43.7 KB) trimmed.bin.pcap trimmed tcpdump jsherril@redhat.com, 09/22/2016 03:55 PM
Actions #1

Updated by jsherril@redhat.com over 7 years ago

To add some more information:

  • This repository is backed by a file:// repo pointing at the file system. The repository synced with no issue
  • Pulp doesn't seem to be 'saving' the tar rpm that is being fetched. It seems to get the redirect every time.
  • The user has an on_demand centos 7 kickstart repo that is working fine, here's a curl from it (but the rpm is already downloaded):
$ curl -vvvv -L http://katello.ipa.domain.lab/pulp/repos/domain_Group/Library/custom/centos/c7/tar-1.26-29.el7.x86_64.rpm > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 172.16.246.31...
* Connected to katello.ipa.domain.lab (172.16.246.31) port 80 (#0)
> GET /pulp/repos/domain_Group/Library/custom/centos/c7/tar-1.26-29.el7.x86_64.rpm HTTP/1.1
> Host: katello.ipa.domain.lab
> User-Agent: curl/7.47.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Thu, 22 Sep 2016 13:55:33 GMT
< Server: Apache/2.4.6 (CentOS)
< Last-Modified: Wed, 14 Sep 2016 18:07:17 GMT
< ETag: "d2c34-53c7b997bfc46"
< Content-Length: 863284
< Content-Type: application/x-rpm
< 
{ [14261 bytes data]
100  843k  100  843k    0     0  35.7M      0 --:--:-- --:--:-- --:--:-- 37.4M
* Connection #0 to host katello.ipa.domain.lab left intact

The user is going to try to change to an http-based repo and see if that changes things.

Actions #2

Updated by jsherril@redhat.com over 7 years ago

  • Subject changed from Can't kickstart against an on_demand repo to Can't kickstart against an on_demand file:// repo

The user confirmed that it works fine when they recreated the repo via http://

Actions #4

Updated by pcreech over 7 years ago

  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes
Actions #5

Updated by mhrivnak over 7 years ago

The design of on-demand downloading depends on the use of HTTP. There may be an opportunity to make things fail more gracefully when a user tries file:///, but we are not able to support on-demand content access via a file:/// URL.

I'm not actually sure why anaconda failed the way it did. Can anyone shed more light on what it was doing, and how it got to those errors? Obviously it was trying http for some requests.

Actions #6

Updated by jsherril@redhat.com over 7 years ago

Is it documented anywhere that pulp doesn't support on_demand (or i assume background?) with file:// repos? Should this be a validation within pulp to prevent this type of repo from being created?

Actions #7

Updated by jortel@redhat.com about 7 years ago

wrote:

Is it documented anywhere that pulp doesn't support on_demand (or i assume background?) with file:// repos? Should this be a validation within pulp to prevent this type of repo from being created?

The limitation on http URLs does not seem to be documented. Probably a good idea to both document this and enforce with validation.

Actions #8

Updated by mhrivnak about 7 years ago

This is related to, but not a blocker for, this issue: https://bugzilla.redhat.com/show_bug.cgi?id=1378514

Actions #11

Updated by bmbouter almost 5 years ago

  • Status changed from NEW to CLOSED - WONTFIX

Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.

Actions #12

Updated by bmbouter almost 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF