As a user I can track progress of the task group with a task group progress report
GroupProgressReport will be a similar model as ProgressReport.
Each group progress report will have a message, code, done, total, and relation to the TaskGroup. Plugin writers will create these objects to show progress of work that is expected to be completed by the tasks in the group.
Tasks that belong to the TaskGroup will update the progress report. All group progress reports needs to be created in advance so a Task can find the appropriate one, by code or message and update it.
Tasks will need to handle logic and figure out what exactly they need to update in the group progress report. For example a task in the migration plugin called
complex repo migration will create a repo version, publication and distribution. That means, the task will update 3 group progress reports.
To avoid race conditions / cache invalidation issues, this pattern needs to be used so that operations are performed directly inside the database
.update(done=F('done') + 1)
Question: How do we figure out 'total' per each group report?
Alternative Solution (a modification of the earlier "progressreport aggregation" strategy): on the TaskGroup serializer add another field called
progress_report. It will query the db and look for tasks that belong to the group and aggregate the results by task 'code'. For example group will have 4 syncing tasks, each task has code 'sync', in the aggregated report there will be total of 4 syncing repos and based on the each tasks status the done will be calculated. This implementation is limited to the name of the task, and it does not have that much flexibility in case tasks creates more resources. For example: a task in the migration plugin called
complex repo migration will create a repo version, publication and distribution. However in the progress report field it will only record that '1 complex repo migration done'.
Updated by bmbouter over 2 years ago
I believe users have needs to understand the overall progress of a group of tasks and distinctly see the progress of a single task. To the extent that is true, the design to have TaskGroupProgress be totally new objects that don't aggregate will force plugin writers to intentionally think about what progress is reported at each level. This is an application of the explicit is better than implicit design principal. The aggregation design would work in most cases, and be easier for plugin writers, but I don't think as strong for users.
For the question on figuring out 'total', generally the plugin writer sets it based on the workload they understand. I don't expect GroupProgressReport to make this race condition free in all cases. Here are three scenarios I think about regarding how various workloads could handle this depending on various needs.
There is no race condition on 'total'. The 'total' is known at some point and is only ever set once. No one but the user ever reads it.
There is only a write-read race condition on 'total'. In this case the writer sets it, calls save() and other tasks that read it, they can't be guaranteed 'total' is up to date unless it's set once. In the case it's set multiple times, they would need to have some sort of synchronization but TaskGroupProgress would not handle this for them in any way. I think this is also unlikely to be needed.
There is a write-write race condition. In this case multiple processes are writing to 'total' based on portions of the work they are discovering. In this case the F() values are the way. Or someone can use a database transaction and handle the transaction-failed errors when one saves and the other doesn't.
Overall I don't think TaskGroupProgress needs to provide much except a F() based implementation for implementing the 'done' count because that's the one that is likeliest to be incremented across multiple processes.
Updated by bmbouter over 2 years ago
The TaskGroupProgress seems reasonable to me. I'm guessing we'll probably have to use F() to update totals.
The 'total' is known at some point and is only ever set once.
Could you give more information about how this would work?
Sure. The import/export example I think is this case actually. IIRC, for imports, the first task to run reads the archive to import and determines how many/which repos need updating and dispatches one "sub-task" for each of them. The first task after reading the number of repos that need updating would set
total when it is known. The sub-tasks work through the work but do not modify
Updated by ttereshc over 2 years ago
I agree that we might find a way to set the
total only once for the current use cases.
I'm not entirely sure about the migration plugin because some of its work can be identified only during the migration itself but there is a chance that all those items are happening before subtasks for more well defined scope are dispatched.
+1 to start with F() for
And if we have a use case, we can add it for