https://pulp.plan.io/https://pulp.plan.io/favicon.ico2019-01-04T21:21:14ZPulpPulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=331032019-01-04T21:21:14Zbmbouterbmbouter@redhat.com
<ul><li><strong>Subject</strong> changed from <i>Stages API could deadlock when "discovering" content</i> to <i>Stages API could deadlock when "discovering" content due to minsize</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/33103/diff?detail_id=33848">diff</a>)</li></ul> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=331042019-01-04T21:31:38Zbmbouterbmbouter@redhat.com
<ul></ul><p><a class="user active" href="https://pulp.plan.io/users/13826">mdellweg</a> had an idea that the DeclarativeContent could be marked somehow for it to not be batched. This solves the issue by disabling batching without having to pass options to all of the stages.</p>
<p>Another option is to introduce a timer to Stage.batches. Overall I think that would be an important safety feature to avoiding deadlock. For example we could add a <code>max_timer=1000</code> where max_timer is the maximum number of milliseconds to hold > 0 items before returning the batch even if minsize is not met. Maybe plugin writers would need to override this option also?</p> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=331142019-01-07T09:28:36Zmdellweg
<ul></ul><p>In my last version of a proposed solution, i used the tagged DeclarativeContent objects to stop waiting for the current batch to fill. So the would not circumvent the batching completely, just cutting them shorter.</p>
<p>And there is another solution i can think of:<br>
We could introduce, like `None` as the pipeline ends marker, a `flush` marker that thaws all pending batches on the way, but not finish the loop. This would, however, require that the DCs do not reorder from step to step.</p> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=331392019-01-08T15:44:46Zbmbouterbmbouter@redhat.com
<ul></ul><p>mdellweg wrote:</p>
<blockquote>
<p>In my last version of a proposed solution, i used the tagged DeclarativeContent objects to stop waiting for the current batch to fill. So the would not circumvent the batching completely, just cutting them shorter.</p>
</blockquote>
<p>I imagined tagging all DeclarativeContent, but you're identifying you would just tag some of them. Can you describe that more? How do you know which DeclarativeContent items could cause deadlock?</p>
<blockquote>
<p>And there is another solution i can think of:<br>
We could introduce, like `None` as the pipeline ends marker, a `flush` marker that thaws all pending batches on the way, but not finish the loop. This would, however, require that the DCs do not reorder from step to step.</p>
</blockquote>
<p>We have some stages that do reordering now so I already have that assumption as a property of the Stages API.</p> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=331402019-01-08T15:51:04ZCodeHeeler
<ul><li><strong>Triaged</strong> changed from <i>No</i> to <i>Yes</i></li></ul> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=331912019-01-11T11:38:45Zmdellweg
<ul></ul><p>bmbouter wrote:</p>
<blockquote>
<p>I imagined tagging all DeclarativeContent, but you're identifying you would just tag some of them. Can you describe that more? How do you know which DeclarativeContent items could cause deadlock?</p>
</blockquote>
<p>I think it is as simple as "Can this content unit give raise to more content units? Then make it prioritized."<br>
In the case of Debian Repositories, the discovery chain has 3 Levels: Release -> PackageIndex -> Package. So I give the priority Flag to all but the last Level. Since the hugely overwhelming part of content units are leafs in this picture, this i not a big performance concern.</p>
<p>If you only have one content type, and you never know in advance, whether it can produce more, you are out of luck here. Then I could think of a kind of barrier (flush marker), that gets inserted whenever the first stage has nothing more to add, and you must make sure, no content unit get's overtaken by that barrier (It might help performance a little if the barrier was semipermeable).</p> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=332132019-01-11T20:11:47Zamacdona@redhat.comaustin@redhat.com
<ul><li><strong>Blocks</strong> <i><a class="issue tracker-4 status-11 priority-6 priority-default closed" href="/issues/4173">Refactor #4173</a>: Change the multilayered design to use Futures to handle nested content</i> added</li></ul> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=332312019-01-14T10:58:57Zmdellweg
<ul></ul><p>Before changing the flow control of the pipeline, i would like to change the plugin api in a way, that the abstracts the interface to the queues away from plugin writers.</p>
<p>RFC: <a href="https://github.com/mdellweg/pulpcore-plugin/tree/refactor_stages" class="external">https://github.com/mdellweg/pulpcore-plugin/tree/refactor_stages</a></p> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=335492019-01-30T13:25:55Zbherring
<ul><li><strong>Related to</strong> <i><a class="issue tracker-5 status-13 priority-8 priority-highest closed" href="/issues/4359">Test #4359</a>: 2.18.1 Testing</i> added</li></ul> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=335512019-01-30T13:26:22Zbherring
<ul><li><strong>Related to</strong> deleted (<i><a class="issue tracker-5 status-13 priority-8 priority-highest closed" href="/issues/4359">Test #4359</a>: 2.18.1 Testing</i>)</li></ul> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=337092019-02-06T21:16:44Zbmbouterbmbouter@redhat.com
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>POST</i></li><li><strong>Assignee</strong> set to <i>mdellweg</i></li></ul><p>PR available at: <a href="https://github.com/pulp/pulpcore-plugin/pull/48" class="external">https://github.com/pulp/pulpcore-plugin/pull/48</a></p> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=337622019-02-08T15:39:20Zmdellweg
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/33762/diff?detail_id=34480">diff</a>)</li></ul> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=337752019-02-08T17:39:27Zmdellweg
<ul><li><strong>Status</strong> changed from <i>POST</i> to <i>MODIFIED</i></li></ul><p>Applied in changeset commit:pulpcore-plugin|42f17e72a0b399c7eb6c61e9722da12021a278e6.</p> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=413942019-04-25T16:44:56Zdaviddavis
<ul><li><strong>Sprint/Milestone</strong> set to <i>3.0.0</i></li></ul> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=421472019-04-26T20:32:38Zbmbouterbmbouter@redhat.com
<ul><li><strong>Tags</strong> deleted (<del><i>Pulp 3</i></del>)</li></ul> Pulp - Issue #4296: Stages API could deadlock when "discovering" content due to minsizehttps://pulp.plan.io/issues/4296?journal_id=505742019-12-13T17:10:09Zbmbouterbmbouter@redhat.com
<ul><li><strong>Status</strong> changed from <i>MODIFIED</i> to <i>CLOSED - CURRENTRELEASE</i></li></ul>