https://pulp.plan.io/https://pulp.plan.io/favicon.ico2020-05-29T12:52:50ZPulpPulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=573442020-05-29T12:52:50Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/57344/diff?detail_id=57934">diff</a>)</li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=573452020-05-29T13:25:43Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/57345/diff?detail_id=57935">diff</a>)</li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=573532020-05-29T14:06:34Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/57353/diff?detail_id=57941">diff</a>)</li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=573542020-05-29T14:10:00Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/57354/diff?detail_id=57942">diff</a>)</li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=573622020-05-29T15:01:36Zdalleydalley@redhat.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/57362/diff?detail_id=57955">diff</a>)</li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=573772020-05-29T20:20:54Zbmbouterbmbouter@redhat.com
<ul></ul><p>I believe users have needs to understand the overall progress of a group of tasks and distinctly see the progress of a single task. To the extent that is true, the design to have TaskGroupProgress be totally new objects that don't aggregate will force plugin writers to intentionally think about what progress is reported at each level. This is an application of the explicit is better than implicit design principal. The aggregation design would work in most cases, and be easier for plugin writers, but I don't think as strong for users.</p>
<p>For the question on figuring out 'total', generally the plugin writer sets it based on the workload they understand. I don't expect GroupProgressReport to make this race condition free in all cases. Here are three scenarios I think about regarding how various workloads could handle this depending on various needs.</p>
<ul>
<li>
<p>There is no race condition on 'total'. The 'total' is known at some point and is only ever set once. No one but the user ever reads it.</p>
</li>
<li>
<p>There is only a write-read race condition on 'total'. In this case the writer sets it, calls save() and other tasks that read it, they can't be guaranteed 'total' is up to date unless it's set once. In the case it's set multiple times, they would need to have some sort of synchronization but TaskGroupProgress would not handle this for them in any way. I think this is also unlikely to be needed.</p>
</li>
<li>
<p>There is a write-write race condition. In this case multiple processes are writing to 'total' based on portions of the work they are discovering. In this case the F() values are the way. Or someone can use a database transaction and handle the transaction-failed errors when one saves and the other doesn't.</p>
</li>
</ul>
<p>Overall I don't think TaskGroupProgress needs to provide much except a F() based implementation for implementing the 'done' count because that's the one that is likeliest to be incremented across multiple processes.</p> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=576272020-06-04T13:41:50Zdaviddavis
<ul></ul><p>The TaskGroupProgress seems reasonable to me. I'm guessing we'll probably have to use F() to update totals.</p>
<blockquote>
<p>The 'total' is known at some point and is only ever set once.</p>
</blockquote>
<p>Could you give more information about how this would work?</p> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=576782020-06-05T16:02:01Zbmbouterbmbouter@redhat.com
<ul></ul><p>daviddavis wrote:</p>
<blockquote>
<p>The TaskGroupProgress seems reasonable to me. I'm guessing we'll probably have to use F() to update totals.</p>
<blockquote>
<p>The 'total' is known at some point and is only ever set once.</p>
</blockquote>
<p>Could you give more information about how this would work?</p>
</blockquote>
<p>Sure. The import/export example I think is this case actually. IIRC, for imports, the first task to run reads the archive to import and determines how many/which repos need updating and dispatches one "sub-task" for each of them. The first task after reading the number of repos that need updating would set <code>total</code> when it is known. The sub-tasks work through the work but do not modify <code>total</code> again.</p> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=577492020-06-08T17:25:33Zttereshcttereshc@redhat.com
<ul></ul><p>I agree that we might find a way to set the <code>total</code> only once for the current use cases.
I'm not entirely sure about the migration plugin because some of its work can be identified only during the migration itself but there is a chance that all those items are happening before subtasks for more well defined scope are dispatched.</p>
<p>+1 to start with F() for <code>done</code>.
And if we have a use case, we can add it for <code>total</code> later.</p> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=578182020-06-09T10:51:30Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/57818/diff?detail_id=58370">diff</a>)</li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=578192020-06-09T10:53:03Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Blocks</strong> <i><a class="issue tracker-3 status-11 priority-6 priority-default closed" href="/issues/6769">Story #6769</a>: As a user, I can track the progress of pulp2->pulp3 migrations</i> added</li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=578232020-06-09T11:06:49Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/57823/diff?detail_id=58376">diff</a>)</li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=578242020-06-09T11:07:09Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Groomed</strong> changed from <i>No</i> to <i>Yes</i></li><li><strong>Sprint</strong> set to <i>Sprint 74</i></li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=580512020-06-11T20:27:38Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 74</i> to <i>Sprint 75</i></li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=581802020-06-16T18:59:57Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>ASSIGNED</i></li><li><strong>Assignee</strong> set to <i>ipanova@redhat.com</i></li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=588262020-06-26T16:04:26Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 75</i> to <i>Sprint 76</i></li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=588732020-06-29T14:30:17Zipanova@redhat.comipanova@redhat.com
<ul></ul><p><a href="https://github.com/pulp/pulpcore/pull/755" class="external">https://github.com/pulp/pulpcore/pull/755</a></p> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=589062020-06-30T08:15:58Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Status</strong> changed from <i>ASSIGNED</i> to <i>MODIFIED</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>Applied in changeset <a class="changeset" title="Add GroupProgressReport model and serializer. closes #6858 https://pulp.plan.io/issues/6858" href="https://pulp.plan.io/projects/pulp/repository/pulpcore/revisions/104aed7b099305457d55d47accf72c0c3693a84a">pulpcore|104aed7b099305457d55d47accf72c0c3693a84a</a>.</p> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=592012020-07-08T19:00:49Zfao89
<ul><li><strong>Sprint/Milestone</strong> set to <i>3.5.0</i></li></ul> Pulp - Story #6858: As a user I can track progress of the task group with a task group progress reporthttps://pulp.plan.io/issues/6858?journal_id=592572020-07-09T15:06:12Zpulpbot
<ul><li><strong>Status</strong> changed from <i>MODIFIED</i> to <i>CLOSED - CURRENTRELEASE</i></li></ul>