Project

Profile

Help

Issue #1375

'latest' tag out-dated in Crane Docker Registry V2 API

Added by twaugh about 4 years ago. Updated 4 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Severity:
2. Medium
Version - Docker:
master
Platform Release:
2.8.0
Blocks Release:
Target Release - Docker:
2.0.0
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

I've observed /v2/<repo>/manifests/latest listing out-dated blobs compared to the equivalent tag on the 'distribution' used for sync.

I had previously created several manifests for the same set of blobs, with tags including '1.0', 'latest', and a unique tag. After this, a pulp sync was started.

After the sync finished, more manifests were created, this time for a new set of blobs, with tags including '1.0', 'latest', and another unique tag.

After another pulp sync was finished, /v2/<repo>/manifests/latest from Crane was showing the older manifest (incorrect), although /v2/<repo>/manifests/1.0 was showing the new manifest (correct).


Related issues

Related to Docker Support - Story #1460: As a user, I can upload Tag units CLOSED - CURRENTRELEASE Actions

Associated revisions

Revision ac41d4fd View on GitHub
Added by rbarlow about 4 years ago

Pass the updated Mongoengine units to Importer.remove_units().

The Docker Importer implements the remove_units() method, but the
platform was still passing the old RepoAssociation objects instead
of the proper Mongoengine units. This commit detects converted
plugins and passes them the new units.

https://pulp.plan.io/issues/1375

re #1375

Revision ac41d4fd View on GitHub
Added by rbarlow about 4 years ago

Pass the updated Mongoengine units to Importer.remove_units().

The Docker Importer implements the remove_units() method, but the
platform was still passing the old RepoAssociation objects instead
of the proper Mongoengine units. This commit detects converted
plugins and passes them the new units.

https://pulp.plan.io/issues/1375

re #1375

Revision 91e09ca0 View on GitHub
Added by rbarlow about 4 years ago

Add a formal Tag Unit model to track repository tags.

This commit creates a new Tag model that is used to mark which Manifests are tagged in a repository. Users
can upload, synchronize, and associate tags which allows them greater control over tags than was afforded in
the v1 code.

https://pulp.plan.io/issues/1375

fixes #1375

Revision 91e09ca0 View on GitHub
Added by rbarlow about 4 years ago

Add a formal Tag Unit model to track repository tags.

This commit creates a new Tag model that is used to mark which Manifests are tagged in a repository. Users
can upload, synchronize, and associate tags which allows them greater control over tags than was afforded in
the v1 code.

https://pulp.plan.io/issues/1375

fixes #1375

Revision 91e09ca0 View on GitHub
Added by rbarlow about 4 years ago

Add a formal Tag Unit model to track repository tags.

This commit creates a new Tag model that is used to mark which Manifests are tagged in a repository. Users
can upload, synchronize, and associate tags which allows them greater control over tags than was afforded in
the v1 code.

https://pulp.plan.io/issues/1375

fixes #1375

Revision 91e09ca0 View on GitHub
Added by rbarlow about 4 years ago

Add a formal Tag Unit model to track repository tags.

This commit creates a new Tag model that is used to mark which Manifests are tagged in a repository. Users
can upload, synchronize, and associate tags which allows them greater control over tags than was afforded in
the v1 code.

https://pulp.plan.io/issues/1375

fixes #1375

History

#1 Updated by rbarlow about 4 years ago

  • Project changed from Pulp to Docker Support
  • Priority changed from Normal to High
  • Version - Docker set to master
  • Target Release - Docker set to 2.0.0
  • Tags Easy Fix added

#2 Updated by twaugh about 4 years ago

The suggested change didn't fix the problem.
/v2/<repo>/manifests/latest from Crane disagrees with the Docker Distribution instance after a 'sync' and 'publish'.

Removing those lines did change which 'latest' I saw, but it still wasn't the right one. :-)

#3 Updated by mhrivnak about 4 years ago

  • Triaged changed from No to Yes

#4 Updated by rbarlow about 4 years ago

  • Tags deleted (Easy Fix)

twaugh wrote:

The suggested change didn't fix the problem.
/v2/<repo>/manifests/latest from Crane disagrees with the Docker Distribution instance after a 'sync' and 'publish'.

You will need to also remove the images from the repository and then resync and republish. The code change prevents the data from ending up in this state in the future, but it will not fix data that is already in this state.

This code will not be our long term fix for the issue, but I do believe it will get you past your immediate hurdle. The problem that this code does not address is what would happen if the feed URL had "rolled back" a Manifest (i.e., an older manifest that had been latest and was replaced became latest again). To make that work we are going to need to track the tags in a more formal way than we are doing at this time.

#5 Updated by ttomecek about 4 years ago

rbarlow wrote:

You will need to also remove the images from the repository and then resync and republish.

I really don't understand why the operation is called sync then. As I understand synchronization it should be an operation which makes content of two repositories identical -- something like `rsync --delete`. Why should be users responsible for managing the actual synchronization?

The way I would solve this is to make the process configurable:

  • add an option to control overwriting
  • add an option to delete content

#6 Updated by mhrivnak about 4 years ago

I understand how that term could be interpreted differently. Perhaps a different word than "sync" should have been chosen, but the choice was made years ago, and we're not in a position right now to change it.

Pulp's sync workflow will continue to be additive by default. When new content appears in the remote feed, pulp will add it locally. When remote content disappears, pulp will retain that content. This is an important behavior that our users depend on.

Some content types have additional options to control and modify this behavior, and we're happy to consider adding something like that to the docker sync.

#7 Updated by ttomecek about 4 years ago

mhrivnak wrote:

Pulp's sync workflow will continue to be additive by default. When new content appears in the remote feed, pulp will add it locally. When remote content disappears, pulp will retain that content. This is an important behavior that our users depend on.

Okay, I understand that if this feature is in pulp for years.

It would be really beneficial for us to have those two options implemented. Especially the content overwriting -- the reason why Tim created this issue.

#8 Updated by acarter@redhat.com about 4 years ago

Checking in on this issue. Can you confirm that the changes requested will be prioritized or let us know if we need to have more conversations? This will be a blocker for Chain Rebuilds which is required by product management asap.

#9 Updated by rbarlow about 4 years ago

wrote:

Checking in on this issue. Can you confirm that the changes requested will be prioritized or let us know if we need to have more conversations?

It's high on my priority list. I will probably get to it in the next two weeks. In the meantime I believe the deleting those lines will work around the issue, though I will likely solve the problem a different way.

#10 Updated by rbarlow about 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to rbarlow

I have developed a plan to solve this issue. I will introduce a new type of Unit called Tag that will have three attributes: repo_id, name, manifest, and version. version will be 2 (representing Docker v2) for now. The repo_id and name fields will have a uniqueness constraint so that we can guarantee that a Tag only exists once per repository. The manifest will be a Manifest digest.

During a sync, whatever tags are found in the remote repository will either get created in the local repository (if missing), or the digests will be updated to reflect the remote digests (if already present). Any Tag objects in the local repository that are not found in the remote repository will remain for now (to be consistent with the defaults of other Pulp plugins, which do not remove data during syncs). We can consider adding a feature later to remove missing Tags, Manifests, and Blobs, similar to pulp_rpm's --remove-missing feature.

When users perform copy operations, it is the user's choice to copy a Tag, a Manifest, or a Blob. If the user wishes to promote a Tag from one repository to another they would simply use Pulp's copy feature. The Manifest referenced by the Tag will also automatically be copied, as well as any Blobs that it references. Any Manifest copied will recursively copy referenced Blobs as it does today, but it will not copy any tags. Copying a Blob will not copy any Manifests at all, since the Blob is the lowest level Unit in Docker v2.

Another advantage of this design is that it will allow users to upload Tag units, which is an ability that was missing with our Docker v1 support. This means that existing Manifests in Pulp can be tagged by the user creating a Tag unit in the repository, which will only require specifying a name, a Manifest digest, and the repository that the user wishes the Tag to be uploaded to.

I believe our users will appreciate the flexibility of this design as it allows them explicit control over the tags which was not possible with pulp_docker < 2.0.

#11 Updated by twaugh about 4 years ago

This sounds like the right direction to me.

With the ability to upload Tag units, why would the user need to specify a name for the tag? Just because pulp_docker doesn't parse the Manifest? They won't want to create a Tag with a name different from what the Manifest says is its tag.

Also, to clarify: the Tag units will only be for V2 repositories?

#12 Updated by rbarlow about 4 years ago

twaugh wrote:

With the ability to upload Tag units, why would the user need to specify a name for the tag? Just because pulp_docker doesn't parse the Manifest? They won't want to create a Tag with a name different from what the Manifest says is its tag.

Hahaha, that is a great point! I'll make it so you just need to give the digest that you want to tag, and the Tag will automatically get the name from it. My excuse is that I didn't have my coffee yet this morning when I wrote this.

Also, to clarify: the Tag units will only be for V2 repositories?

Correct, we do not wish to alter the v1 workflow if possible.

#13 Updated by rbarlow about 4 years ago

  • Related to Story #1460: As a user, I can upload Tag units added

#14 Updated by rbarlow about 4 years ago

  • Status changed from ASSIGNED to MODIFIED
  • % Done changed from 0 to 100

#15 Updated by rbarlow about 4 years ago

  • Status changed from MODIFIED to ASSIGNED

We need to do this again for the docker-v2-support branch, so I'm sending it back to assigned.

#16 Updated by rbarlow about 4 years ago

  • Status changed from ASSIGNED to MODIFIED

#17 Updated by rbarlow about 4 years ago

I moved this back to modified, as we agreed that this would only be done in upstream pulp_docker.

#18 Updated by rbarlow almost 4 years ago

  • Status changed from MODIFIED to ON_QA

#19 Updated by rbarlow over 3 years ago

  • Status changed from ON_QA to CLOSED - CURRENTRELEASE
  • Platform Release set to 2.8.0

#20 Updated by bmbouter 9 months ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF