Refactor #460
closed
Use celery.signals.worker_process_init to start the database connection
100%
Description
We attempted to do this in 2.4.0, but we ran into a few different issues. One of them was that for some reason our Celery tasks weren't being discovered by the workers. It was decided that this was too large of a change to go in so late in the 2.4.0 release cycle, and that we would do it the right way in a future release (perhaps 2.5.0, or 3.0.0).
Here is the pull request that first introduced the change that we would like to reintroduce in the future:
https://github.com/pulp/pulp/pull/1019
That change introduced two issues:
1) The qpid monkey patching no longer happened early enough in the worker startup, and so kombu would not work. I fixed this by moving the qpid import in initialization.py to be the first import, which revealed a second problem:
2) The Celery workers no longer discovered our tasks for some reason. I never learned why this happened before we decided to back out the change and revisit it later.
Here is the PR that removed the change:
https://github.com/pulp/pulp/pull/1023
+ This bug was cloned from Bugzilla Bug #1113192 +
.
Ensure that tests share a single DB conenction.
Many of our tests independently connected to the database, and they did not always ensure that they were connecting to the test database. This commit fixes the tests so they all use the test database.
Additionally, this commit refactors all of the entry points such that the connection is no longer started as a side effect of importing a module. Most of our code already worked this way, but this commit refactors the Celery workers to start the database connection using a signal instead of through an import.
This commit also makes it an error for the connection to be initialized more than once.
https://pulp.plan.io/issues/131 https://pulp.plan.io/issues/460
fixes #131 fixes #460