Task #2318
closed
Ensure thread-safety of ProgressReport models
Status:
CLOSED - CURRENTRELEASE
Description
Django models changes can be thread safe[0], but some care needs to be taken. We need the Progressreport models specifically to be thread safe.
This likely involves 2 things:
1. rewriting the implementaiton to be thread safe
2. Using documentation which identifies the thread-safety concern for plugin writers updating models themselves from many threads.
[0]: http://stackoverflow.com/questions/26234819/are-django-models-thread-safe#26234946
It is likely just the incrementing feature that needs to be thread-safe. The pattern will probably be:
- create a progress object
- create a Queue or similar
- create some threads and give them a reference to the queue, and probably the incrementor that comes with the progress object
- each thread runs in a loop: grab a job from the queue, do it, call queue.task_done(), and increment the progress object
- the main thread calls queue.join() and waits for the work to finish
It's worth considering if there's a way to either wrap or somehow hook into the queue's task_done method, but that's optional. There may not be a reasonable way to do so.
Yes item 1 is likely as simple as the increment() method only being updated. I think most of the work is in the docs.
- Groomed changed from No to Yes
- Sprint/Milestone set to 29
- Status changed from NEW to ASSIGNED
- Assignee set to dkliban@redhat.com
- Description updated (diff)
- Tags Documentation added
After some lunchtime conversation, it was confirmed that this is a documentation only change and that the docstrings should recommend the user have all threads share the same instance of ProgressBar. I don't think ProgressSpinner and ProgressReport need updates since updating those frequently will almost never occur.
I'm setting the documentation tag and rewriting the story some.
- Status changed from ASSIGNED to POST
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
- Sprint/Milestone deleted (
29)
- Sprint/Milestone set to 3.0.0
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Also available in: Atom
PDF
Adds docs about using ProgressBar with threads
closes #2318 https://pulp.plan.io/issues/2318