Project

Profile

Help

Issue #2988

Exception when raising a user-Defined Exception that has a custom __init__ signature

Added by bmbouter about 1 month ago. Updated 5 days ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
Severity:
2. Medium
Version:
Platform Release:
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 3
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No

Description

Consider this diff applied onto a sync task in core:

[bmbouter@localhost pulp]$ git diff platform/pulpcore/app/tasks/importer.py
diff --git a/platform/pulpcore/app/tasks/importer.py b/platform/pulpcore/app/tasks/importer.py
index 0ef18a8..bab5d76 100644
--- a/platform/pulpcore/app/tasks/importer.py
+++ b/platform/pulpcore/app/tasks/importer.py
@@ -51,6 +51,12 @@ def delete(repo_name, importer_name):
     models.Importer.objects.filter(name=importer_name, repository__name=repo_name).delete()

+class SomeError(Exception):
+    def __init__(self, url):
+        self.url = url
+        super().__init__()
+
+
 @shared_task(base=UserFacingTask)
 def sync(repo_name, importer_name):
     """ 
@@ -72,4 +78,5 @@ def sync(repo_name, importer_name):

     with storage.working_dir_context() as working_dir:
         importer.working_dir = working_dir
+        raise SomeError('asdf')
         importer.sync()

When you call a sync() task, you'll experience the following exception:

Aug 22 17:22:03 pulp3 celery[22903]: [2017-08-22 17:22:03,336: ERROR/ForkPoolWorker-1] Task failed : [65f60bc5-6433-4dda-a5c0-22b22c82ceb0]
Aug 22 17:22:03 pulp3 celery[22903]: [2017-08-22 17:22:03,342: ERROR/ForkPoolWorker-1] Task pulpcore.app.tasks.importer.sync[65f60bc5-6433-4dda-a5c0-22b22c82ceb0] raised unexpected: SomeError()
Aug 22 17:22:03 pulp3 celery[22903]: Traceback (most recent call last):
Aug 22 17:22:03 pulp3 celery[22903]:   File "/home/vagrant/.virtualenvs/pulp/lib/python3.5/site-packages/celery/app/trace.py", line 374, in trace_task
Aug 22 17:22:03 pulp3 celery[22903]:     R = retval = fun(*args, **kwargs)
Aug 22 17:22:03 pulp3 celery[22903]:   File "/home/vagrant/devel/pulp/platform/pulpcore/tasking/tasks.py", line 272, in __call__
Aug 22 17:22:03 pulp3 celery[22903]:     return super(UserFacingTask, self).__call__(*args, **kwargs)
Aug 22 17:22:03 pulp3 celery[22903]:   File "/home/vagrant/.virtualenvs/pulp/lib/python3.5/site-packages/celery/app/trace.py", line 629, in __protected_call__
Aug 22 17:22:03 pulp3 celery[22903]:     return self.run(*args, **kwargs)
Aug 22 17:22:03 pulp3 celery[22903]:   File "/home/vagrant/devel/pulp/platform/pulpcore/app/tasks/importer.py", line 81, in sync
Aug 22 17:22:03 pulp3 celery[22903]:     raise SomeError('asdf')
Aug 22 17:22:03 pulp3 celery[22903]: pulpcore.app.tasks.importer.SomeError
Aug 22 17:22:03 pulp3 celery[22903]: [2017-08-22 17:22:03,344: ERROR/MainProcess] Task handler raised error: <MaybeEncodingError: Error sending result: ''(1, <ExceptionInfo: SomeError()>, None)''. Reason: ''PicklingError("Can\'t pickle <class \'pulpcore.app.tasks.importer.SomeError\'>: it\'s not the same object as pulpcore.app.tasks.importer.SomeError",)''.>
Aug 22 17:22:03 pulp3 celery[22903]: Traceback (most recent call last):
Aug 22 17:22:03 pulp3 celery[22903]:   File "/home/vagrant/.virtualenvs/pulp/lib/python3.5/site-packages/billiard/pool.py", line 362, in workloop
Aug 22 17:22:03 pulp3 celery[22903]:     put((READY, (job, i, result, inqW_fd)))
Aug 22 17:22:03 pulp3 celery[22903]:   File "/home/vagrant/.virtualenvs/pulp/lib/python3.5/site-packages/billiard/queues.py", line 366, in put
Aug 22 17:22:03 pulp3 celery[22903]:     self.send_payload(ForkingPickler.dumps(obj))
Aug 22 17:22:03 pulp3 celery[22903]:   File "/home/vagrant/.virtualenvs/pulp/lib/python3.5/site-packages/billiard/reduction.py", line 56, in dumps
Aug 22 17:22:03 pulp3 celery[22903]:     cls(buf, protocol).dump(obj)
Aug 22 17:22:03 pulp3 celery[22903]: billiard.pool.MaybeEncodingError: Error sending result: ''(1, <ExceptionInfo: SomeError()>, None)''. Reason: ''PicklingError("Can\'t pickle <class \'pulpcore.app.tasks.importer.SomeError\'>: it\'s not the same object as pulpcore.app.tasks.importer.SomeError",)''.

However with this diff:

[bmbouter@localhost pulp]$ git diff platform/pulpcore/app/tasks/importer.py
diff --git a/platform/pulpcore/app/tasks/importer.py b/platform/pulpcore/app/tasks/importer.py
index 0ef18a8..023d0c3 100644
--- a/platform/pulpcore/app/tasks/importer.py
+++ b/platform/pulpcore/app/tasks/importer.py
@@ -51,6 +51,12 @@ def delete(repo_name, importer_name):
     models.Importer.objects.filter(name=importer_name, repository__name=repo_name).delete()

+class SomeError(Exception):
+    def __init__(self):
+        self.url = 'this does not matter'
+        super().__init__()
+
+
 @shared_task(base=UserFacingTask)
 def sync(repo_name, importer_name):
     """ 
@@ -72,4 +78,5 @@ def sync(repo_name, importer_name):

     with storage.working_dir_context() as working_dir:
         importer.working_dir = working_dir
+        raise SomeError()
         importer.sync()

When you call the sync() task you'll receive this traceback:

Aug 22 17:26:23 pulp3 celery[23021]: [2017-08-22 17:26:23,720: ERROR/ForkPoolWorker-1] Task failed : [7b33c9a3-d13f-45cf-a510-c16f3b18deeb]
Aug 22 17:26:23 pulp3 celery[23021]: [2017-08-22 17:26:23,724: ERROR/ForkPoolWorker-1] Task pulpcore.app.tasks.importer.sync[7b33c9a3-d13f-45cf-a510-c16f3b18deeb] raised unexpected: SomeError()
Aug 22 17:26:23 pulp3 celery[23021]: Traceback (most recent call last):
Aug 22 17:26:23 pulp3 celery[23021]:   File "/home/vagrant/.virtualenvs/pulp/lib/python3.5/site-packages/celery/app/trace.py", line 374, in trace_task
Aug 22 17:26:23 pulp3 celery[23021]:     R = retval = fun(*args, **kwargs)
Aug 22 17:26:23 pulp3 celery[23021]:   File "/home/vagrant/devel/pulp/platform/pulpcore/tasking/tasks.py", line 272, in __call__
Aug 22 17:26:23 pulp3 celery[23021]:     return super(UserFacingTask, self).__call__(*args, **kwargs)
Aug 22 17:26:23 pulp3 celery[23021]:   File "/home/vagrant/.virtualenvs/pulp/lib/python3.5/site-packages/celery/app/trace.py", line 629, in __protected_call__
Aug 22 17:26:23 pulp3 celery[23021]:     return self.run(*args, **kwargs)
Aug 22 17:26:23 pulp3 celery[23021]:   File "/home/vagrant/devel/pulp/platform/pulpcore/app/tasks/importer.py", line 81, in sync
Aug 22 17:26:23 pulp3 celery[23021]:     raise SomeError()
Aug 22 17:26:23 pulp3 celery[23021]: pulpcore.app.tasks.importer.SomeError

History

#1 Updated by bmbouter about 1 month ago

I think this is an upstream Celery bug. I commented on a bug that looks very similar here: https://github.com/celery/celery/issues/3758#issuecomment-324091432

#2 Updated by ttereshc 28 days ago

This issue can't be triaged without more info/discussion.
Do we want to workaround this issue in Pulp or we are going to wait for Celery upstream fix?

#3 Updated by bmbouter 18 days ago

I want to outline two next steps that should be done for this ticket.

1) I think we need to reproduce this outside of Pulp and file upstream. Then we can let upstream identify it as a duplicate or not. I don't think we should lead the effort to resolve this in upstream Celery.

2) We should workaround the issue in the meantime while we track the status of the upstream resolution. There are two workarounds I can think of. I prefer option A.

option a) Use a factory to generate the exceptions. This is how we did it in Pulp2. It's an uncommon pattern but I think it will workaround the issue because the exception class definitions themselves won't have custom init signatures. We should not do this long-term.

option b) We can continue to define concrete exceptions but add all the necessary info as attributes by whoever instatiates the exception. This is effectively option A only with the data binding happening all over the codebase instead of in one place. I don't recommend this, but I include it for completeness.

NOTE1: All of ^ is built on the assumption that we want concrete exception classes. I think we do. If others think we don't please post here.

NOTE2: I believe switching the exception style will be backwards incompatible for the plugin API since the way they would instantiate or subclass one of the exceptions provided by the plugin API would change. With the plugin API not at 1.0 yet we can make these changes still, post release. Just an FYI related to this.

#4 Updated by ttereshc 13 days ago

  • Triaged changed from No to Yes

#5 Updated by mhrivnak 5 days ago

  • Sprint/Milestone set to Sprint 26

Please register to edit this issue

Also available in: Atom PDF