Project

Profile

Help

Issue #1033

closed

Error during build_session does not propagate to importer

Added by mkovacik@redhat.com over 9 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Start date:
Due date:
Estimated time:
Severity:
3. High
Version - Nectar:
1.3.1
Platform Release:
Target Release - Nectar:
1.4.4
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

An error risen in threaded.py:build_session doesn't propagate out of a worker thread causing a repo sync to get stuck.
Actually, I have just a single reproducer using pulp-admin rpm repo create ... --proxy-host without specifying protocol but other errors manifesting during a Requests session build might have similar effect.
The root cause of the error can be seen in downloaders/threaded.py:HTTPThreadedDownloader.worker method, line 102 (7 here):

def worker(self, queue):
    """
    :param queue:       queue of DownloadRequest instances
    :type  queue:       WorkerQueue
    """
    try:
        session = build_session(self.config)

        while True:
            request = queue.get()

            if request is None or self.is_canceled:
                session.close()
                break

            self._fetch(request, session)

    except:
        _logger.exception('Unhandled Exception in Worker Thread [%s]' % threading.currentThread().ident)

I'm attaching a hot-fix that calls self.cancel() in case of an exception risen in the block to at least propagates back to the importer so that user doesn't have to witness the repo sync progress wheel spinning forever.
Anyway, best would be to introduce a self.error() method so that proper information is propagated to the Downloader caller whenever a general non-url/-report error happens.

Reproducer steps:
[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo create --repo-id zoo --feed https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/ --proxy-host ip-172-31-26-58.eu-central-1.compute.internal --proxy-port 3128 --proxy-user squid --proxy-pass squid
[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo

[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata... FOREVER!
[\]

Having applied the patch:

[root@ip-172-31-24-137 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[\]
... failed

repomd.xml has not been downloaded

Task Failed

Importer indicated a failed response

[root@ip-172-31-24-137 ~]#

Version:
python-nectar-1.3.1-2.el6.noarch

See also: Issue #1032


Files

threaded.py.patch (403 Bytes) threaded.py.patch mkovacik@redhat.com, 06/05/2015 03:46 PM
Actions #1

Updated by jortel@redhat.com over 9 years ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes
Actions #2

Updated by ipanova@redhat.com about 9 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ipanova@redhat.com
  • Target Release - Nectar set to master
Actions #3

Updated by ipanova@redhat.com about 9 years ago

  • Status changed from ASSIGNED to POST

Added by ipanova@redhat.com about 9 years ago

Revision 73ef2cf1 | View on GitHub

1033 - Error during build_session does not propagate to importer.

closes #1033 https://pulp.plan.io/issues/1033

Actions #4

Updated by ipanova@redhat.com about 9 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #5

Updated by rbarlow almost 9 years ago

  • Target Release - Nectar changed from master to 1.4.4
Actions #6

Updated by rbarlow almost 9 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #7

Updated by bmbouter over 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF