Issue #1033
closedError during build_session does not propagate to importer
Description
An error risen in threaded.py:build_session
doesn't propagate out of a worker thread causing a repo sync to get stuck.
Actually, I have just a single reproducer using pulp-admin rpm repo create ... --proxy-host
without specifying protocol but other errors manifesting during a Requests session build might have similar effect.
The root cause of the error can be seen in downloaders/threaded.py:HTTPThreadedDownloader.worker
method, line 102 (7 here):
def worker(self, queue):
"""
:param queue: queue of DownloadRequest instances
:type queue: WorkerQueue
"""
try:
session = build_session(self.config)
while True:
request = queue.get()
if request is None or self.is_canceled:
session.close()
break
self._fetch(request, session)
except:
_logger.exception('Unhandled Exception in Worker Thread [%s]' % threading.currentThread().ident)
I'm attaching a hot-fix that calls self.cancel()
in case of an exception risen in the block to at least propagates back to the importer so that user doesn't have to witness the repo sync progress wheel spinning forever.
Anyway, best would be to introduce a self.error()
method so that proper information is propagated to the Downloader caller whenever a general non-url/-report error happens.
Reproducer steps:
[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo create --repo-id zoo --feed https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/ --proxy-host ip-172-31-26-58.eu-central-1.compute.internal --proxy-port 3128 --proxy-user squid --proxy-pass squid
[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo
[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
Synchronizing Repository [zoo]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata... FOREVER!
[\]
Having applied the patch:
[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
Synchronizing Repository [zoo]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[\]
... failed
repomd.xml has not been downloaded
Task Failed
Importer indicated a failed response
[root@ip-172-31-24-137 ~]#
Version:
python-nectar-1.3.1-2.el6.noarch
See also: Issue #1032
Files
1033 - Error during build_session does not propagate to importer.
closes #1033 https://pulp.plan.io/issues/1033