Issue #1033
closedError during build_session does not propagate to importer
Description
An error risen in threaded.py:build_session
doesn't propagate out of a worker thread causing a repo sync to get stuck.
Actually, I have just a single reproducer using pulp-admin rpm repo create ... --proxy-host
without specifying protocol but other errors manifesting during a Requests session build might have similar effect.
The root cause of the error can be seen in downloaders/threaded.py:HTTPThreadedDownloader.worker
method, line 102 (7 here):
def worker(self, queue):
"""
:param queue: queue of DownloadRequest instances
:type queue: WorkerQueue
"""
try:
session = build_session(self.config)
while True:
request = queue.get()
if request is None or self.is_canceled:
session.close()
break
self._fetch(request, session)
except:
_logger.exception('Unhandled Exception in Worker Thread [%s]' % threading.currentThread().ident)
I'm attaching a hot-fix that calls self.cancel()
in case of an exception risen in the block to at least propagates back to the importer so that user doesn't have to witness the repo sync progress wheel spinning forever.
Anyway, best would be to introduce a self.error()
method so that proper information is propagated to the Downloader caller whenever a general non-url/-report error happens.
Reproducer steps:
[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo create --repo-id zoo --feed https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/ --proxy-host ip-172-31-26-58.eu-central-1.compute.internal --proxy-port 3128 --proxy-user squid --proxy-pass squid
[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo
[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
Synchronizing Repository [zoo]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata... FOREVER!
[\]
Having applied the patch:
[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
Synchronizing Repository [zoo]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[\]
... failed
repomd.xml has not been downloaded
Task Failed
Importer indicated a failed response
[root@ip-172-31-24-137 ~]#
Version:
python-nectar-1.3.1-2.el6.noarch
See also: Issue #1032
Files
Updated by jortel@redhat.com over 9 years ago
- Priority changed from Normal to High
- Severity changed from 2. Medium to 3. High
- Triaged changed from No to Yes
Updated by ipanova@redhat.com about 9 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to ipanova@redhat.com
- Target Release - Nectar set to master
Updated by ipanova@redhat.com about 9 years ago
- Status changed from ASSIGNED to POST
Added by ipanova@redhat.com about 9 years ago
Updated by ipanova@redhat.com about 9 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset 73ef2cf16eb94de511d4812f5ffeeb8660a58fcc.
Updated by rbarlow almost 9 years ago
- Target Release - Nectar changed from master to 1.4.4
Updated by rbarlow almost 9 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
1033 - Error during build_session does not propagate to importer.
closes #1033 https://pulp.plan.io/issues/1033