Issue #1170
closed
rpm importer does not support switching protocols from http to ftp mid download using a redirect- fails downloading distribution files
Description
I was unsure whether to open this as an issue here, with Nectar or with RPM Support so defaulting to top level.
To reproduce:
pulp-admin rpm repo create --relative-url test --feed http://www.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/ --repo-id test
pulp-admin rpm repo sync run --repo-id test
[...]
Downloading distribution files...
[===================================== ] 75%
Distributions: 3/4 items
Errors encountered during distribution sync:
File: http://www.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso
Error
Code: None
Error Message: None
This appears to be because Nectar is using requests.py which does not support ftp. From my logs:
Aug 7 18:28:20 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): www.gtlib.gatech.edu
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) No connection adapters were found for 'ftp://ftp.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso'
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) Traceback (most recent call last):
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 190, in _fetch
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) response = session.get(request.url, headers=headers)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 469, in get
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) return self.request('GET', url, **kwargs)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 457, in request
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) resp = self.send(prep, **send_kwargs)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 595, in send
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) history = [resp for resp in gen] if allow_redirects else []
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 189, in resolve_redirects
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) allow_redirects=False,
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 563, in send
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) adapter = self.get_adapter(url=request.url)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 642, in get_adapter
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) raise InvalidSchema("No connection adapters were found for '%%s'" %% url)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) InvalidSchema: No connection adapters were found for 'ftp://ftp.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso'
Aug 7 18:28:55 pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: some distro file downloads failed
Aug 7 18:28:55 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Sync complete.
This is likely a related issue - https://pulp.plan.io/issues/197
I worked around this by finding another mirror that didn't point to their ftp server to download this iso.
Related issues
Updated by bcourt over 8 years ago
- Subject changed from rpm importer does not support ftp - fails downloading distribution files to rpm importer does not support switching protocols from http to ftp mid download using a redirect- fails downloading distribution files
- Triaged changed from No to Yes
This appears to be happening because the web server is redirecting from an http to an ftp url and nectar is not following the protocol change.
Updated by osabina over 8 years ago
From what I could tell nectar only uses Requests.py which only supports http. I don’t think the switch is the problem.
Oz
On Aug 21, 2015, at 10:45, Pulp <dropbox+pulp+c71e@plan.io> wrote:
Updated by mhrivnak about 8 years ago
- Related to Issue #1736: Nectar fails to handle redirects to FTP gracefully. added
Updated by bmbouter about 5 years ago
- Status changed from NEW to CLOSED - WONTFIX
Updated by bmbouter about 5 years ago
Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.
.