https://pulp.plan.io/https://pulp.plan.io/favicon.ico2015-11-16T17:58:03ZPulpDocker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=70572015-11-16T17:58:03Zrbarlow
<ul><li><strong>Project</strong> changed from <i>Pulp</i> to <i>Docker Support</i></li><li><strong>Priority</strong> changed from <i>Normal</i> to <i>High</i></li><li><strong>Version - Docker</strong> set to <i>master</i></li><li><strong>Target Release - Docker</strong> set to <i>2.0.0</i></li><li><strong>Tags</strong> <i>Easy Fix</i> added</li></ul><p>It's likely that deleting these lines will solve this issue:</p>
<p><a href="https://github.com/pulp/pulp_docker/blob/docker_v2_api/plugins/pulp_docker/plugins/distributors/publish_steps.py#L135-L137" class="external">https://github.com/pulp/pulp_docker/blob/docker_v2_api/plugins/pulp_docker/plugins/distributors/publish_steps.py#L135-L137</a></p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=70842015-11-18T11:31:18Ztwaughtwaugh@redhat.com
<ul></ul><p>The suggested change didn't fix the problem.<br>
/v2/<repo>/manifests/latest from Crane disagrees with the Docker Distribution instance after a 'sync' and 'publish'.</p>
<p>Removing those lines did change which 'latest' I saw, but it still wasn't the right one. :-)</p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=71352015-11-20T15:45:57Zmhrivnakmhrivnak@redhat.com
<ul><li><strong>Triaged</strong> changed from <i>No</i> to <i>Yes</i></li></ul> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=71472015-11-20T18:39:56Zrbarlow
<ul><li><strong>Tags</strong> deleted (<del><i>Easy Fix</i></del>)</li></ul><p>twaugh wrote:</p>
<blockquote>
<p>The suggested change didn't fix the problem.<br>
/v2/<repo>/manifests/latest from Crane disagrees with the Docker Distribution instance after a 'sync' and 'publish'.</p>
</blockquote>
<p>You will need to also remove the images from the repository and then resync and republish. The code change prevents the data from ending up in this state in the future, but it will not fix data that is already in this state.</p>
<p>This code will not be our long term fix for the issue, but I do believe it will get you past your immediate hurdle. The problem that this code does not address is what would happen if the feed URL had "rolled back" a Manifest (i.e., an older manifest that had been latest and was replaced became latest again). To make that work we are going to need to track the tags in a more formal way than we are doing at this time.</p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=71602015-11-24T13:38:48Zttomecekttomecek@redhat.com
<ul></ul><p>rbarlow wrote:</p>
<blockquote>
<p>You will need to also remove the images from the repository and then resync and republish.</p>
</blockquote>
<p>I really don't understand why the operation is called sync then. As I understand synchronization it should be an operation which makes content of two repositories identical -- something like `rsync --delete`. Why should be users responsible for managing the actual synchronization?</p>
<p>The way I would solve this is to make the process configurable:</p>
<ul>
<li>add an option to control overwriting</li>
<li>add an option to delete content</li>
</ul> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=71672015-11-24T15:51:39Zmhrivnakmhrivnak@redhat.com
<ul></ul><p>I understand how that term could be interpreted differently. Perhaps a different word than "sync" should have been chosen, but the choice was made years ago, and we're not in a position right now to change it.</p>
<p>Pulp's sync workflow will continue to be additive by default. When new content appears in the remote feed, pulp will add it locally. When remote content disappears, pulp will retain that content. This is an important behavior that our users depend on.</p>
<p>Some content types have additional options to control and modify this behavior, and we're happy to consider adding something like that to the docker sync.</p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=71692015-11-24T19:07:07Zttomecekttomecek@redhat.com
<ul></ul><p>mhrivnak wrote:</p>
<blockquote>
<p>Pulp's sync workflow will continue to be additive by default. When new content appears in the remote feed, pulp will add it locally. When remote content disappears, pulp will retain that content. This is an important behavior that our users depend on.</p>
</blockquote>
<p>Okay, I understand that if this feature is in pulp for years.</p>
<p>It would be really beneficial for us to have those two options implemented. Especially the content overwriting -- the reason why Tim created this issue.</p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=72482015-12-03T15:55:02Zacarter@redhat.comacarter@redhat.com
<ul></ul><p>Checking in on this issue. Can you confirm that the changes requested will be prioritized or let us know if we need to have more conversations? This will be a blocker for Chain Rebuilds which is required by product management asap.</p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=72532015-12-03T20:14:07Zrbarlow
<ul></ul><p><a href="mailto:acarter@redhat.com" class="email">acarter@redhat.com</a> wrote:</p>
<blockquote>
<p>Checking in on this issue. Can you confirm that the changes requested will be prioritized or let us know if we need to have more conversations?</p>
</blockquote>
<p>It's high on my priority list. I will probably get to it in the next two weeks. In the meantime I believe the deleting those lines will work around the issue, though I will likely solve the problem a different way.</p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=75262015-12-21T15:50:34Zrbarlow
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>ASSIGNED</i></li><li><strong>Assignee</strong> set to <i>rbarlow</i></li></ul><p>I have developed a plan to solve this issue. I will introduce a new type of Unit called Tag that will have three attributes: repo_id, name, manifest, and version. version will be 2 (representing Docker v2) for now. The repo_id and name fields will have a uniqueness constraint so that we can guarantee that a Tag only exists once per repository. The manifest will be a Manifest digest.</p>
<p>During a sync, whatever tags are found in the remote repository will either get created in the local repository (if missing), or the digests will be updated to reflect the remote digests (if already present). Any Tag objects in the local repository that are not found in the remote repository will remain for now (to be consistent with the defaults of other Pulp plugins, which do not remove data during syncs). We can consider adding a feature later to remove missing Tags, Manifests, and Blobs, similar to pulp_rpm's --remove-missing feature.</p>
<p>When users perform copy operations, it is the user's choice to copy a Tag, a Manifest, or a Blob. If the user wishes to promote a Tag from one repository to another they would simply use Pulp's copy feature. The Manifest referenced by the Tag will also automatically be copied, as well as any Blobs that it references. Any Manifest copied will recursively copy referenced Blobs as it does today, but it will not copy any tags. Copying a Blob will not copy any Manifests at all, since the Blob is the lowest level Unit in Docker v2.</p>
<p>Another advantage of this design is that it will allow users to upload Tag units, which is an ability that was missing with our Docker v1 support. This means that existing Manifests in Pulp can be tagged by the user creating a Tag unit in the repository, which will only require specifying a name, a Manifest digest, and the repository that the user wishes the Tag to be uploaded to.</p>
<p>I believe our users will appreciate the flexibility of this design as it allows them explicit control over the tags which was not possible with pulp_docker < 2.0.</p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=75292015-12-21T17:44:52Ztwaughtwaugh@redhat.com
<ul></ul><p>This sounds like the right direction to me.</p>
<p>With the ability to upload Tag units, why would the user need to specify a name for the tag? Just because pulp_docker doesn't parse the Manifest? They won't want to create a Tag with a name different from what the Manifest says is its tag.</p>
<p>Also, to clarify: the Tag units will only be for V2 repositories?</p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=75342015-12-21T20:21:06Zrbarlow
<ul></ul><p>twaugh wrote:</p>
<blockquote>
<p>With the ability to upload Tag units, why would the user need to specify a name for the tag? Just because pulp_docker doesn't parse the Manifest? They won't want to create a Tag with a name different from what the Manifest says is its tag.</p>
</blockquote>
<p>Hahaha, that is a great point! I'll make it so you just need to give the digest that you want to tag, and the Tag will automatically get the name from it. My excuse is that I didn't have my coffee yet this morning when I wrote this.</p>
<blockquote>
<p>Also, to clarify: the Tag units will only be for V2 repositories?</p>
</blockquote>
<p>Correct, we do not wish to alter the v1 workflow if possible.</p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=75902015-12-23T20:12:28Zrbarlow
<ul><li><strong>Related to</strong> <i><a class="issue tracker-3 status-11 priority-6 priority-default closed" href="/issues/1460">Story #1460</a>: As a user, I can upload Tag units</i> added</li></ul> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=78032016-01-11T18:15:25Zrbarlow
<ul><li><strong>Status</strong> changed from <i>ASSIGNED</i> to <i>MODIFIED</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>Applied in changeset <a class="changeset" title="Add a formal Tag Unit model to track repository tags. This commit creates a new Tag model that i..." href="https://pulp.plan.io/projects/pulp_docker/repository/pulp_docker/revisions/91e09ca00d50db8866aa7bf4e955946c137d7fe4">91e09ca00d50db8866aa7bf4e955946c137d7fe4</a>.</p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=78062016-01-11T18:24:34Zrbarlow
<ul><li><strong>Status</strong> changed from <i>MODIFIED</i> to <i>ASSIGNED</i></li></ul><p>We need to do this again for the docker-v2-support branch, so I'm sending it back to assigned.</p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=80702016-01-21T20:08:34Zrbarlow
<ul><li><strong>Status</strong> changed from <i>ASSIGNED</i> to <i>MODIFIED</i></li></ul> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=80712016-01-21T20:09:00Zrbarlow
<ul></ul><p>I moved this back to modified, as we agreed that this would only be done in upstream pulp_docker.</p> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=88102016-02-11T20:16:51Zrbarlow
<ul><li><strong>Status</strong> changed from <i>MODIFIED</i> to <i>5</i></li></ul> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=109992016-04-26T16:16:02Zrbarlow
<ul><li><strong>Status</strong> changed from <i>5</i> to <i>CLOSED - CURRENTRELEASE</i></li><li><strong>Platform Release</strong> set to <i>2.8.0</i></li></ul> Docker Support - Issue #1375: 'latest' tag out-dated in Crane Docker Registry V2 APIhttps://pulp.plan.io/issues/1375?journal_id=394542019-04-15T20:41:16Zbmbouterbmbouter@redhat.com
<ul><li><strong>Tags</strong> <i>Pulp 2</i> added</li></ul>