Project

Profile

Help

Issue #1771

closed

requests or urllib3 can't read a file which causes Nectar to fail mysteriously

Added by jcline@redhat.com about 8 years ago. Updated almost 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.8.0
Platform Release:
2.8.3
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 1
Quarter:

Description

Katello encountered this traceback while lazy-syncing while syncing a whole bunch of repos.

Mar 14 08:22:26 sat-r220-04.lab.eng.rdu2.redhat.com pulp_streamer[29271]: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: cdn.redhat.com
Mar 14 08:22:26 sat-r220-04.lab.eng.rdu2.redhat.com pulp_streamer[29271]: nectar.downloaders.threaded:WARNING: Skipping requests to cdn.redhat.com due to repeated connection failures: [Errno 2] No such file or directory
Mar 14 08:22:26 sat-r220-04.lab.eng.rdu2.redhat.com pulp_streamer[29271]: nectar.downloaders.base:ERROR: (29271-90048) u'response_code'
Mar 14 08:22:26 sat-r220-04.lab.eng.rdu2.redhat.com pulp_streamer[29271]: nectar.downloaders.base:ERROR: (29271-90048) Traceback (most recent call last):
Mar 14 08:22:26 sat-r220-04.lab.eng.rdu2.redhat.com pulp_streamer[29271]: nectar.downloaders.base:ERROR: (29271-90048)   File "/usr/lib/python2.7/site-packages/nectar/downloaders/base.py", line 145, in _fire_event_to_listener
Mar 14 08:22:26 sat-r220-04.lab.eng.rdu2.redhat.com pulp_streamer[29271]: nectar.downloaders.base:ERROR: (29271-90048)     event_listener_callback(*args, **kwargs)
Mar 14 08:22:26 sat-r220-04.lab.eng.rdu2.redhat.com pulp_streamer[29271]: nectar.downloaders.base:ERROR: (29271-90048)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 101, in download_failed
Mar 14 08:22:26 sat-r220-04.lab.eng.rdu2.redhat.com pulp_streamer[29271]: nectar.downloaders.base:ERROR: (29271-90048)     '%%(response_msg)s' %% error_report))
Mar 14 08:22:26 sat-r220-04.lab.eng.rdu2.redhat.com pulp_streamer[29271]: nectar.downloaders.base:ERROR: (29271-90048) KeyError: u'response_code'
Mar 14 08:22:26 sat-r220-04.lab.eng.rdu2.redhat.com pulp_streamer[29271]: [-] 127.0.0.1 - - [14/Mar/2016:12:22:25 +0000] "GET /var/lib/pulp/content/units/distribution/49/da05e1e526ca2cefec9985aea4c85ba2d1a04e167ecf1acbe50b0063fc30c9/images/pxeboot/vmlinuz HTTP/1.1" 200 - "-" "Wget/1.17.1 (linux-gnu)"

Note the ConnectionError was caused by "[Errno 2] No such file or directory". After some research (see notes below) this is due to https://github.com/kennethreitz/requests/issues/2863.

The plan is to stop Nectar from writing temporary certificates to disk and then deleting them, as this will not work even if requests #2863 got fixed. Instead, they should be stored permanently on disk (/var/lib/pulp/pki or similar). In addition to this, a second issue has been filed to add a work-around to the streamer (https://pulp.plan.io/issues/1788). Note that the problem of stale CAs, client certificates, and client keys will still be present on all our downloading components, but it is unlikely they will experience a problem.


Files

Also available in: Atom PDF