Issue #1170
closedrpm importer does not support switching protocols from http to ftp mid download using a redirect- fails downloading distribution files
Description
I was unsure whether to open this as an issue here, with Nectar or with RPM Support so defaulting to top level.
To reproduce:
pulp-admin rpm repo create --relative-url test --feed http://www.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/ --repo-id test
pulp-admin rpm repo sync run --repo-id test
[...]
Downloading distribution files...
[===================================== ] 75%
Distributions: 3/4 items
Errors encountered during distribution sync:
File: http://www.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso
Error
Code: None
Error Message: None
This appears to be because Nectar is using requests.py which does not support ftp. From my logs:
Aug 7 18:28:20 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): www.gtlib.gatech.edu
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) No connection adapters were found for 'ftp://ftp.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso'
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) Traceback (most recent call last):
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 190, in _fetch
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) response = session.get(request.url, headers=headers)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 469, in get
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) return self.request('GET', url, **kwargs)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 457, in request
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) resp = self.send(prep, **send_kwargs)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 595, in send
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) history = [resp for resp in gen] if allow_redirects else []
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 189, in resolve_redirects
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) allow_redirects=False,
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 563, in send
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) adapter = self.get_adapter(url=request.url)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 642, in get_adapter
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) raise InvalidSchema("No connection adapters were found for '%%s'" %% url)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) InvalidSchema: No connection adapters were found for 'ftp://ftp.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso'
Aug 7 18:28:55 pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: some distro file downloads failed
Aug 7 18:28:55 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Sync complete.
This is likely a related issue - https://pulp.plan.io/issues/197
I worked around this by finding another mirror that didn't point to their ftp server to download this iso.
Related issues