Project

Profile

Help

Issue #1170

closed

rpm importer does not support switching protocols from http to ftp mid download using a redirect- fails downloading distribution files

Added by osabina over 8 years ago. Updated almost 5 years ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.6.3
Platform Release:
OS:
CentOS 6
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

I was unsure whether to open this as an issue here, with Nectar or with RPM Support so defaulting to top level.

To reproduce:

pulp-admin rpm repo create --relative-url test --feed http://www.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/ --repo-id test
pulp-admin rpm repo sync run --repo-id test
[...]
Downloading distribution files...
[===================================== ] 75%
Distributions: 3/4 items

Errors encountered during distribution sync:

File: http://www.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso
Error
Code: None
Error Message: None

This appears to be because Nectar is using requests.py which does not support ftp. From my logs:

Aug 7 18:28:20 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): www.gtlib.gatech.edu
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) No connection adapters were found for 'ftp://ftp.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso'
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) Traceback (most recent call last):
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 190, in _fetch
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) response = session.get(request.url, headers=headers)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 469, in get
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) return self.request('GET', url, **kwargs)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 457, in request
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) resp = self.send(prep, **send_kwargs)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 595, in send
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) history = [resp for resp in gen] if allow_redirects else []
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 189, in resolve_redirects
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) allow_redirects=False,
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 563, in send
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) adapter = self.get_adapter(url=request.url)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 642, in get_adapter
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) raise InvalidSchema("No connection adapters were found for '%%s'" %% url)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) InvalidSchema: No connection adapters were found for 'ftp://ftp.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso'
Aug 7 18:28:55 pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: some distro file downloads failed
Aug 7 18:28:55 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Sync complete.

This is likely a related issue - https://pulp.plan.io/issues/197

I worked around this by finding another mirror that didn't point to their ftp server to download this iso.


Related issues

Related to Nectar - Issue #1736: Nectar fails to handle redirects to FTP gracefully.CLOSED - WONTFIXActions

Also available in: Atom PDF