Project

Profile

Help

Issue #1033

closed

Error during build_session does not propagate to importer

Added by mkovacik@redhat.com over 9 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Start date:
Due date:
Estimated time:
Severity:
3. High
Version - Nectar:
1.3.1
Platform Release:
Target Release - Nectar:
1.4.4
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

An error risen in threaded.py:build_session doesn't propagate out of a worker thread causing a repo sync to get stuck.
Actually, I have just a single reproducer using pulp-admin rpm repo create ... --proxy-host without specifying protocol but other errors manifesting during a Requests session build might have similar effect.
The root cause of the error can be seen in downloaders/threaded.py:HTTPThreadedDownloader.worker method, line 102 (7 here):

def worker(self, queue):
    """
    :param queue:       queue of DownloadRequest instances
    :type  queue:       WorkerQueue
    """
    try:
        session = build_session(self.config)

        while True:
            request = queue.get()

            if request is None or self.is_canceled:
                session.close()
                break

            self._fetch(request, session)

    except:
        _logger.exception('Unhandled Exception in Worker Thread [%s]' % threading.currentThread().ident)

I'm attaching a hot-fix that calls self.cancel() in case of an exception risen in the block to at least propagates back to the importer so that user doesn't have to witness the repo sync progress wheel spinning forever.
Anyway, best would be to introduce a self.error() method so that proper information is propagated to the Downloader caller whenever a general non-url/-report error happens.

Reproducer steps:
[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo create --repo-id zoo --feed https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/ --proxy-host ip-172-31-26-58.eu-central-1.compute.internal --proxy-port 3128 --proxy-user squid --proxy-pass squid
[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo

[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata... FOREVER!
[\]

Having applied the patch:

[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[\]
... failed

repomd.xml has not been downloaded

Task Failed

Importer indicated a failed response

[root@ip-172-31-24-137 ~]#

Version:
python-nectar-1.3.1-2.el6.noarch

See also: Issue #1032


Files

threaded.py.patch (403 Bytes) threaded.py.patch mkovacik@redhat.com, 06/05/2015 03:46 PM

Also available in: Atom PDF