Project

Profile

Help

Issue #1539

Anaconda receives 404 from Pulp repo during install

Added by dkliban@redhat.com almost 5 years ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
2.8.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Steps to reproduce:

1. Sync repo from https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/kickstart/
2. Point anaconda to install from that repo.
3. Watch anaconda fail due to 404s from the server

Associated revisions

Revision 17fb6c89 View on GitHub
Added by Jeremy Cline almost 5 years ago

Update save_revision to work with multi-file units.

Previously, the query used to get catalog revisions would retrieve all entries for all files in a multi-file content unit. This ensures that only catalog entries for a single file are retrieved. In combination with a pulp_rpm PR, this closes issue 1539.

Revision 936de8b2 View on GitHub
Added by Jeremy Cline almost 5 years ago

Allow the catalog checksum and checksum_type to be None.

The pulp_distribution.xml is missing checksum and checksum_type fields, among other things. We have to allow these fields to be None for the present.

History

#1 Updated by jcline@redhat.com almost 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to jcline@redhat.com

#2 Updated by mhrivnak almost 5 years ago

  • Triaged changed from No to Yes

#3 Updated by jcline@redhat.com almost 5 years ago

  • Triaged changed from Yes to No

I started attempting to reproduce this by syncing the stage CDN equivalent:

pulp-admin rpm repo create --repo-id el7-ks --feed http://cdn.stage.redhat.com/content/dist/rhel/server/7/7Server/x86_64/kickstart/ --download-policy on_demand 

This failed initially with the following traceback:

Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776) 'NoneType' object has no attribute 'lower'
Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776) Traceback (most recent call last):
Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/sync.py", line 235, in run
Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776)     dist_sync.run()
Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 125, in run
Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776)     self._run(tmp_dir)
Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 165, in _run
Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776)     self.update_unit_files(unit, files)
Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 242, in update_unit_
Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776)     CHECKSUM_TYPE: verification.sanitize_checksum_type(_file[CHECKSUM_TYPE])})
Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/verification.py", line 67, in sanitize_checksum_type
Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776)     lowercase_checksum_type = checksum_type.lower()
Jan 15 21:53:11 dev pulp[14792]: pulp_rpm.plugins.importers.yum.sync:ERROR: (14792-07776) AttributeError: 'NoneType' object has no attribute 'lower'

This occurs because some files come from the http://cdn.stage.redhat.com/content/dist/rhel/server/7/7Server/x86_64/kickstart/PULP_DISTRIBUTION.xml which does not have a checksum or checksum type. Nor does it include an XML schema (I will file a separate issue about this). The schema is located at https://github.com/pulp/pulp_rpm/blob/1ebbd75e8b076fd5d00c95b91c1c1ad26006ed52/plugins/usr/share/pulp-rpm/pulp_distribution.xsd. To handle this problem I added the following:

diff --git a/plugins/pulp_rpm/plugins/importers/yum/parse/treeinfo.py b/plugins/pulp_rpm/plugins/importers/yum/parse/treeinfo.py
index aafc292..b2afe7f 100644
--- a/plugins/pulp_rpm/plugins/importers/yum/parse/treeinfo.py
+++ b/plugins/pulp_rpm/plugins/importers/yum/parse/treeinfo.py
@@ -236,10 +237,12 @@ class DistSync(object):
         if not isinstance(unit.files, list):
             _list = list(unit.files)
         for _file in files:
+            if _file[CHECKSUM_TYPE] is not None:
+                _file[CHECKSUM_TYPE] = verification.sanitize_checksum_type(_file[CHECKSUM_TYPE])
             _list.append({
                 RELATIVE_PATH: _file[RELATIVE_PATH],
                 CHECKSUM: _file[CHECKSUM],
-                CHECKSUM_TYPE: verification.sanitize_checksum_type(_file[CHECKSUM_TYPE])})
+                CHECKSUM_TYPE: _file[CHECKSUM_TYPE]})
         unit.files = _list

     def download_files(self, tmp_dir, files):

This allowed me to complete a sync, but I was unable to kickstart from the repository. The following is the httpd request log while attempting to kickstart from http://dev.example.com/pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/

192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "GET /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/.treeinfo HTTP/1.1" 404 89 "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "GET /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/content HTTP/1.1" 404 87 "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/Fedora HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/current/images/MANIFEST HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/daily/MANIFEST HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/current/images/MANIFEST HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/install/netboot/version.info HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/SL HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/CentOS HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/VERSION HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/.disk/info HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/Server HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/Client HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/RedHat HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/images/pxeboot/vmlinuz HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/images/boot.iso HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/boot/boot.iso HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/current/images/netboot/mini.iso HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/install/images/boot.iso HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:45 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/ HTTP/1.1" 500 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "GET /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/.treeinfo HTTP/1.1" 404 89 "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "GET /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/content HTTP/1.1" 404 87 "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/Fedora HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/current/images/MANIFEST HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/daily/MANIFEST HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/current/images/MANIFEST HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/install/netboot/version.info HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/SL HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/CentOS HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/VERSION HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/.disk/info HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/Server HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/Client HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/RedHat HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/images/pxeboot/vmlinuz HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/images/boot.iso HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/boot/boot.iso HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/current/images/netboot/mini.iso HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/install/images/boot.iso HTTP/1.1" 404 - "-" "python-requests/2.9.1"
192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/ HTTP/1.1" 500 - "-" "python-requests/2.9.1"

That 500 is interesting (I will file a separate issue), but the surprising thing is this entry:

192.168.121.1 - - [15/Jan/2016:21:28:52 +0000] "HEAD /pulp/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/images/pxeboot/vmlinuz HTTP/1.1" 404

This is in the published repo (note that these were all broken links):

[root@dev pxeboot]# pwd
/var/www/pub/yum/http/repos/content/dist/rhel/server/7/7Server/x86_64/kickstart/images/pxeboot

[root@dev pxeboot]# ls -lah
total 20K
drwxr-sr-x. 2 apache apache 4.0K Jan 15 22:10 .
drwxr-sr-x. 3 apache apache 4.0K Jan 15 22:10 ..
lrwxrwxrwx. 1 apache apache  108 Jan 15 22:10 initrd.img -> /var/lib/pulp/content/units/distribution/3e82/3e82b040-8b09-434f-ae1f-a076a825cbc6/images/pxeboot/initrd.img
lrwxrwxrwx. 1 apache apache  109 Jan 15 22:10 upgrade.img -> /var/lib/pulp/content/units/distribution/3e82/3e82b040-8b09-434f-ae1f-a076a825cbc6/images/pxeboot/upgrade.img
lrwxrwxrwx. 1 apache apache  105 Jan 15 22:10 vmlinuz -> /var/lib/pulp/content/units/distribution/3e82/3e82b040-8b09-434f-ae1f-a076a825cbc6/images/pxeboot/vmlinuz

[root@dev pxeboot]# ls -lah /var/lib/pulp/content/units/distribution/3e82/3e82b040-8b09-434f-ae1f-a076a825cbc6/images/pxeboot/
ls: cannot access /var/lib/pulp/content/units/distribution/3e82/3e82b040-8b09-434f-ae1f-a076a825cbc6/images/pxeboot/: No such file or directory

But not in the catalog:

> db.lazy_content_catalog.find({"path": "/var/lib/pulp/content/units/rpm/48b4/48b40206-5723-420e-9a9b-2ec69e6ad5eb/zlib-devel-1.2.7-13.el7.x86_64.rpm"}).pretty()
{
        "_id" : ObjectId("56996a44e7798939c82be7b3"),
        "_ns" : "lazy_content_catalog",
        "path" : "/var/lib/pulp/content/units/rpm/48b4/48b40206-5723-420e-9a9b-2ec69e6ad5eb/zlib-devel-1.2.7-13.el7.x86_64.rpm",
        "importer_id" : "569969b5e77989373afc9dd0",
        "unit_id" : "48b40206-5723-420e-9a9b-2ec69e6ad5eb",
        "unit_type_id" : "rpm",
        "url" : "http://cdn.stage.redhat.com/content/dist/rhel/server/7/7Server/x86_64/kickstart/Packages/zlib-devel-1.2.7-13.el7.x86_64.rpm",
        "revision" : 1,
        "data" : {

        }
}
> db.lazy_content_catalog.find({"path": "/var/lib/pulp/content/units/distribution/3e82/3e82b040-8b09-434f-ae1f-a076a825cbc6/images/pxeboot/vmlinuz"}).pretty()

#4 Updated by jcline@redhat.com almost 5 years ago

  • Triaged changed from No to Yes

#6 Updated by jcline@redhat.com almost 5 years ago

  • Status changed from POST to MODIFIED

#7 Updated by rbarlow almost 5 years ago

  • Status changed from MODIFIED to 5

#8 Updated by dkliban@redhat.com over 4 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

#9 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF