Issue #1608
closedsync from cdn is broken
Description
Syncing from the feed below fails but reports that the sync was successful. Syncs from this feed works on 2.7-dev branch.
pulp-admin rpm repo create --repo-id test --feed-ca-cert ./redhat-uep.pem --feed-cert ./cert.pem --feed-key ./key.pem --feed https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.1/os/
Successfully created repository [test]
(pulp)[vagrant@dev pulp]$ pulp-admin rpm repo sync run --repo-id=test
+----------------------------------------------------------------------+
Synchronizing Repository [test]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[|]
... completed
Downloading repository content...
[==================================================] 100%
RPMs: 79/79 items
Delta RPMs: 0/0 items
... completed
Individual package errors encountered during sync:
Package:
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.1/o
s/python-qpid-proton-0.9-4.el7.x86_64.rpm
Error:
Package:
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.1/o
s/rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch.rpm
Error:
Package:
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.1/o
s/rubygem-hammer_cli_foreman_docker-0.0.3.9-1.el7sat.noarch.rpm
Error:
Package:
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.1/o
s/rubygem-powerbar-1.0.11-8.el7sat.noarch.rpm
Error:
Package:
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.1/o
s/rubygem-hammer_cli_gutterball-0.0.1.3-1.el7sat.noarch.rpm
Error:
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[-]
... completed
Task Succeeded
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[-]
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[==================================================] 100%
6 of 6 items
... completed
Publishing Comps file
[-]
... completed
Publishing Metadata.
[-]
... completed
Closing repo metadata
[-]
... completed
Generating sqlite files
... skipped
Publishing files to web
[-]
... completed
Writing Listings File
[-]
... completed
Task Succeeded
Updated by bmbouter about 8 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to bmbouter
The symptom is that all 79 packages are erroring so 0 are actually downloaded.
Updated by bmbouter about 8 years ago
I reproduced this in my Vagrant environment w/ a 2.8.0 checkout install. I confirmed that nectar is receiving a 404 for each rpm requested. For example:
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.1/os/rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch.rpm
I confirmed that curl reports the same thing:
curl --cacert ./redhat-uep.pem --cert ./cert.pem --key ./key.pem https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.1/os/rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch.rpm
File not found."
@beav suggested that that URL was incorrect and that is the root cause of the 404. He suggested it should be the URL below, which he got from the CDN primary.xml
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.1/os/Packages/rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch.rpm
Which sounds right because it returns a 200 with curl as follows:
curl --cacert ./redhat-uep.pem --cert ./cert.pem --key ./key.pem https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6.1/os/Packages/rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch.rpm
Updated by bmbouter about 8 years ago
- Status changed from ASSIGNED to NEW
- Assignee deleted (
bmbouter)
I need to put this back to NEW since I can't continue at the moment, but there is something going wrong with the parsing of the metadata in primary.xml which is causing the urls to be incorrect and 404.
Updated by amacdona@redhat.com about 8 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to amacdona@redhat.com
Added by Austin Macdonald about 8 years ago
Updated by Anonymous about 8 years ago
- Status changed from ASSIGNED to MODIFIED
- % Done changed from 0 to 100
Applied in changeset b01406f689c352ec64b58025c6208c6f33ebeb6b.
Updated by amacdona@redhat.com about 8 years ago
- Status changed from MODIFIED to POST
Updated by bmbouter about 8 years ago
Lots of comments are posted on that PR with more information about why some repos are affected and others aren't.
Updated by amacdona@redhat.com about 8 years ago
- Status changed from POST to MODIFIED
Updated by dkliban@redhat.com about 8 years ago
- Status changed from MODIFIED to 5
Updated by pthomas@redhat.com about 8 years ago
- Status changed from 5 to 6
verified
[root@mgmt5 ~]# pulp-admin rpm repo list --repo-id rhel6 --details
--------------------------------------------------------------------
RPM Repositories
--------------------------------------------------------------------
Id: rhel6
Display Name: None
Description: None
Content Unit Counts:
Distribution: 1
Package Category: 10
Package Group: 202
Rpm: 3819
Yum Repo Metadata File: 1
Notes:
Scratchpad:
Checksum Type: sha256
Importers:
Config:
Download Policy: on_demand
Feed: https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.7/
x86_64/kickstart/
Version: 3 (0x2)ificate:
91:8a:cf:ca:85:00:00:00
Issuer: Signature Algorithm: sha1WithRSAEncryption
C=US, ST=North Carolina, L=Raleigh, O=Red Hat, Inc., OU=Red
Hat Network, CN=Entitlement Master
Validity CA/emailAddress=ca-support@redhat.com
Not Afterot Before: Mar 18 11:24:54 2010 GMT
Subject: C=US, ST=North:24:54 2030 GMT
Carolina, O=Red Hat, Inc., OU=Red Hat Network, CN=Red Hat
Entitlement Operations
Authority/emailAddress=ca-support@redhat.com
Public Key Algorithm:c Key Info:
RSA Public Key: (4096 bit)
Modulus (4096 bit):
00:bb:26:8a:88:43:9c:db:c8:a4:13:19:54:94:79:
a5:b3:fa:1a:b1:39:07:87:68:52:6b:d3:79:1e:17:
27:90:ba:ea:eb:c5:4f:0b:86:4d:58:a0:35:70:0b:
fb:45:16:7f:da:f6:03:0c:4a:10:f0:b2:eb:98:7b:
Exponent: 65537 (0x10001)
X509v3 Subject Keyensions:
Identifier:
C4:49:78:56:45:9D:1E:2C:61:60:C3:66:4B:58:05:37:2F:03:22:7B
X509v3 Authority Key Identifier:
keyid:88:4B:A6:45:C4:46:EC:8F:D6:CF:A6:F6:1A:CF:26:3C:90:CC
DirName:/C=US/ST=North
Carolina/L=Raleigh/O=Red Hat, Inc./OU=Red Hat
Network/CN=Entitlement Master
CA/emailAddress=ca-support@redhat.com
X509v3 Basical:E6:FE:42:28:25:C9:E6:5E
X509v3:TRUEstraints:
Certificate Sign, CRL Sign
SSL CA, S/MIME CA Type:
X509v3 Subject Alternative Name:
X509v3 Issuer:ca-support@redhat.com
Alternative Name:
Signature Algorithm:l:ca-support@redhat.com
sha1WithRSAEncryption
5b:4d:2c:fe:50:85:cf:01:52:13:d1:95:4b:d5:f9:42:1d:1d:
75:dd:a8:f6:8f:75:01:66:46:a8:0e:44:b1:b4:47:da:d6:7a:
b2:b2:63:74:96:5a:6a:87:83:55:04:bd:da:9c:41:58:c6:89:
0b:85:0d:8f:07:c9:a6:b8:3e:44:b9:66:8d:c6:75:d3:73:84:
ab:e5:db:0e:72:39:05:89:4d:7d:ee:5c:9b:68:92:bb:c3:83:
b3:43:64:20:fd:7f:ed:e3:e4:7b:f1:c7:cc:64:af:ea:1e:98:
ee:c9:3d:cd:c8:71:ef:0a:e8:3f:5b:39:2f:53:4f:69:46:bd:
9a:54:72:3e:7e:7e:bc:bb:5f:13:47:33:c9:34:ab:de:81:1e:
18:78:0d:ef:b3:24:16:06:01:1c:f2:c5:3b:2a:b0:2f:8a:41:
a6:38:7d:45:fa:f7:ee:99:92:b3:5b:bf:0b:95:50:f6:41:8d:
79:ae:60:a1:d2:2f:7e:01:40:a2:52:c3:72:61:d2:5b:49:90:
-----BEGIN CERTIFICATE-----3:c1:c2:5c:14:a1
MIIHZTCCBU2gAwIBAgIJAJGKz8qFAAAAMA0GCSqGSIb3DQEBBQUAMIGwMQs
wCQYD
-----ENDAUCiUsNyYdJbSZDPiFPBwlwUoQ==
-----BEGIN CERTIFICATE-----CATE-----
MIIHZDCCBUygAwIBAgIJAOb+QiglyeZeMA0GCSqGSIb3DQEBBQUAMIGwMQs
13PQU jboSsRM+ -----END CERTIFICATE-----
SSL Client Key: -----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAz17SsKSWsn/O6zofxfB6qgrcOTaGM55e3KN6g+cBMIC
m
CkURC
eRS776hNmYNWreUbFoXfcW9WpPWAxF0FcSPtUQAXrnOz0pL5EEc38A==
-----END RSA PRIVATE KEY-----
Id: yum_importer
Importer Type Id: yum_importer
Last Sync: 2016-03-14T14:50:41Z
Repo Id: rhel6
Scratchpad:
Previous Skip List:
Repomd Revision: 1435822714
Distributors:
Auto Publish: True
Config:
Checksum Type: sha256
Http: False
Https: True
Relative URL: content/dist/rhel/rhui/server/6/6.7/x86_64/kickstart/
Distributor Type Id: yum_distributor
Id: yum_distributor
Last Publish: 2016-03-14T14:52:03Z
Repo Id: rhel6
Scratchpad:
Auto Publish: False
Config:
Http: False
Https: True
Relative URL: content/dist/rhel/rhui/server/6/6.7/x86_64/kickstart/
Distributor Type Id: export_distributor
Id: export_distributor
Last Publish: None
Repo Id: rhel6
Scratchpad:
[root@mgmt5 ~]# pulp-admin rpm repo sync run --repo-id rhel6
--------------------------------------------------------------------
Synchronizing Repository [rhel6]
--------------------------------------------------------------------
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[\]
... completed
Downloading repository content...
[-]
[==================================================] 100%
RPMs: 0/0 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[-]
... completed
Cleaning duplicate packages...
[-]
... completed
Task Succeeded
Copying files
[-]
... completed
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[\]
... completed
Publishing RPMs
[\]
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[-]
... completed
Publishing Comps file
[==================================================] 100%
212 of 212 items
... completed
Publishing Metadata.
[-]
... completed
Closing repo metadata
[-]
... completed
Generating sqlite files
... skipped
Publishing files to web
[/]
... completed
Writing Listings File
[-]
... completed
Task Succeeded
Updated by dkliban@redhat.com about 8 years ago
- Status changed from 6 to CLOSED - CURRENTRELEASE
use download path rather than file name
closes #1608