Project

Profile

Help

Issue #1170

rpm importer does not support switching protocols from http to ftp mid download using a redirect- fails downloading distribution files

Added by osabina about 5 years ago. Updated over 1 year ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.6.3
Platform Release:
OS:
CentOS 6
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

I was unsure whether to open this as an issue here, with Nectar or with RPM Support so defaulting to top level.

To reproduce:

pulp-admin rpm repo create --relative-url test --feed http://www.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/ --repo-id test
pulp-admin rpm repo sync run --repo-id test
[...]
Downloading distribution files...
[===================================== ] 75%
Distributions: 3/4 items

Errors encountered during distribution sync:

File: http://www.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso
Error
Code: None
Error Message: None

This appears to be because Nectar is using requests.py which does not support ftp. From my logs:

Aug 7 18:28:20 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): www.gtlib.gatech.edu
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) No connection adapters were found for 'ftp://ftp.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso'
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) Traceback (most recent call last):
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 190, in _fetch
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) response = session.get(request.url, headers=headers)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 469, in get
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) return self.request('GET', url, **kwargs)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 457, in request
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) resp = self.send(prep, **send_kwargs)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 595, in send
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) history = [resp for resp in gen] if allow_redirects else []
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 189, in resolve_redirects
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) allow_redirects=False,
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 563, in send
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) adapter = self.get_adapter(url=request.url)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 642, in get_adapter
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) raise InvalidSchema("No connection adapters were found for '%%s'" %% url)
Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) InvalidSchema: No connection adapters were found for 'ftp://ftp.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso'
Aug 7 18:28:55 pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: some distro file downloads failed
Aug 7 18:28:55 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Sync complete.

This is likely a related issue - https://pulp.plan.io/issues/197

I worked around this by finding another mirror that didn't point to their ftp server to download this iso.


Related issues

Related to Nectar - Issue #1736: Nectar fails to handle redirects to FTP gracefully.CLOSED - WONTFIX<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

History

#1 Updated by bcourt about 5 years ago

  • Subject changed from rpm importer does not support ftp - fails downloading distribution files to rpm importer does not support switching protocols from http to ftp mid download using a redirect- fails downloading distribution files
  • Triaged changed from No to Yes

This appears to be happening because the web server is redirecting from an http to an ftp url and nectar is not following the protocol change.

#2 Updated by osabina about 5 years ago

From what I could tell nectar only uses Requests.py which only supports http. I don’t think the switch is the problem.

Oz

On Aug 21, 2015, at 10:45, Pulp <> wrote:

#3 Updated by mhrivnak over 4 years ago

  • Related to Issue #1736: Nectar fails to handle redirects to FTP gracefully. added

#4 Updated by bmbouter over 1 year ago

  • Status changed from NEW to CLOSED - WONTFIX

#5 Updated by bmbouter over 1 year ago

Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.

#6 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF