Project

Profile

Help

Issue #914

Repo sync broken

Added by mkovacik@redhat.com over 5 years ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Start date:
Due date:
Estimated time:
Severity:
3. High
Version - Docker:
1.0.0
Platform Release:
Target Release - Docker:
1.0.1
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

The repo synchronization is broken in pulp-docker-plugins-1.0.0-1.fc21.noarch
The Registry Tags API has changed recently --- see attached patch that fixes the issue for me

How reproducible:
Always

Steps to reproduce:
- pulp-admin docker repo create --repo-id=busybox --feed=https://index.docker.io --upstream-name=busybox
- pulp-admin docker repo sync run --repo-id=busybox

Actual result:
Sync task fails with an IOError NOT FOUND

Expected result:
Sync task finishes OK

Notes:
Breaks pulp automation test case: https://github.com/RedHatQE/pulp-automation/blob/master/tests/general_tests/test_20_docker_repo_cud.py

registry.py.diff (925 Bytes) registry.py.diff mkovacik@redhat.com, 04/24/2015 01:43 AM

Associated revisions

Revision 8081b301 View on GitHub
Added by mhrivnak over 5 years ago

The API for fetching tags from a remote registry or index changed. This uses the new API.

closes #914

Revision 8081b301 View on GitHub
Added by mhrivnak over 5 years ago

The API for fetching tags from a remote registry or index changed. This uses the new API.

closes #914

Revision 8081b301 View on GitHub
Added by mhrivnak over 5 years ago

The API for fetching tags from a remote registry or index changed. This uses the new API.

closes #914

Revision 8081b301 View on GitHub
Added by mhrivnak over 5 years ago

The API for fetching tags from a remote registry or index changed. This uses the new API.

closes #914

History

#1 Updated by mhrivnak over 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to mhrivnak
  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes

#2 Updated by mhrivnak over 5 years ago

Thanks for catching this. I confirmed the problem and solution. It's extra-confusing because right now, getting tags for the busybox repo from either index.docker.io, or registry-1.docker.io, both work but return different image IDs. Maybe the former has stale data that they're no longer updating?

Have you been able to find other repositories that are broken in this way? I was able to sync "pulp/worker" just fine, so it seems not all of them have the same mismatch of tag data.

#3 Updated by mkovacik@redhat.com over 5 years ago

mhrivnak wrote:

Thanks for catching this. I confirmed the problem and solution. It's extra-confusing because right now, getting tags for the busybox repo from either index.docker.io, or registry-1.docker.io, both work but return different image IDs. Maybe the former has stale data that they're no longer updating?

It seems the index url https://index.docker.io/v1/*/ is now being used just to get the Authorization Token. All other request are supposed to go through the endpoints specified in the X-Docker-Endpoints header and have to be authorized //1.
At least that works for now.

Have you been able to find other repositories that are broken in this way? I was able to sync "pulp/worker" just fine, so it seems not all of them have the same mismatch of tag data.

My Public personal repo is affected https://registry.hub.docker.com/repos/dparalen/ and the patch works for me in this case.
I'm able to sync "pulp/worker" having applied the patch.
Maybe only newer and library repos are affected, can't tell.

Cheers,
milan

//1 looking into some other projects implementation, https://github.com/CenturyLinkLabs/docker-reg-client ; some trial&error with curl ; following docker CLI tool logs ; googlin'

#4 Updated by mhrivnak over 5 years ago

  • Version changed from 2.6.1 Beta to 1.0.0

#5 Updated by mhrivnak over 5 years ago

  • Status changed from ASSIGNED to POST
  • Target Release - Docker set to 1.0.1

#6 Updated by mhrivnak over 5 years ago

  • Status changed from POST to MODIFIED

#7 Updated by mhrivnak over 5 years ago

  • % Done changed from 0 to 100

#8 Updated by bmbouter over 5 years ago

  • Version deleted (1.0.0)
  • Version - Docker set to 1.0.0

#9 Updated by dkliban@redhat.com over 5 years ago

  • Status changed from MODIFIED to 5

#11 Updated by mkovacik@redhat.com over 5 years ago

  • Status changed from 5 to 6

Verified with:

  • Red Hat Enterprise Linux Server release 6.6 (Santiago)
  • pulp-server-2.6.2-0.2.beta.el6.noarch
  • pulp-docker-plugins-1.0.1-0.2.beta.el6.noarch
[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo create --repo-id busybox --upstream-name busybox --feed https://index.docker.io
Repository [busybox] successfully created

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo sync run --repo-id busybox
+----------------------------------------------------------------------+
                   Synchronizing Repository [busybox]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Retrieving metadata
[/]
... completed

Copying units already in pulp
[-]
... completed

Downloading remote files
[==================================================] 100%
23 of 23 items
... completed

Saving images and tags
[-]
... completed

Task Succeeded

Publishing Image Files.
[==================================================] 100%
9 of 9 items
... completed

Making files available via web.
[-]
... completed

Task Succeeded

[root@ip-172-31-20-145 ~]# pulp-admin  -u admin -p admin docker repo delete --repo-id pulp-automation
This command may be exited via ctrl+c without affecting the request.

[\]
Running...

Repository [pulp-automation] successfully deleted

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo create --repo-id hello-world --upstream-name hello-world --feed https://index.docker.io
Repository [hello-world] successfully created

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo sync run --repo-id hello-world
+----------------------------------------------------------------------+
                 Synchronizing Repository [hello-world]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Retrieving metadata
[|]
... completed

Copying units already in pulp
[-]
... completed

Downloading remote files
[==================================================] 100%
5 of 5 items
... completed

Saving images and tags
[-]
... completed

Task Succeeded

Publishing Image Files.
[==================================================] 100%
2 of 2 items
... completed

Making files available via web.
[-]
... completed

Task Succeeded

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo create --repo-id pulp-automation --feed=https://index.docker.io --upstream-name dparalen/pulp-automation
Repository [pulp-automation] successfully created

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo sync run --repo-id hello-world
+----------------------------------------------------------------------+
                 Synchronizing Repository [hello-world]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Retrieving metadata
[-]
... completed

Copying units already in pulp
[-]
... completed

Downloading remote files
[-]
... completed

Saving images and tags
[-]
... completed

Task Succeeded

Publishing Image Files.
[==================================================] 100%
2 of 2 items
... completed

Making files available via web.
[-]
... completed

Task Succeeded

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo sync run --repo-id pulp-automation
+----------------------------------------------------------------------+
               Synchronizing Repository [pulp-automation]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Retrieving metadata
[/]
... completed

Copying units already in pulp
[-]
... completed

Downloading remote files
[==================================================] 100%
53 of 53 items
... completed

Saving images and tags
[-]
... completed

Task Succeeded

Publishing Image Files.
[==================================================] 100%
18 of 18 items
... completed

Making files available via web.
[-]
... completed

Task Succeeded

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo create --repo-id none --feed=https://index.docker.io --upstream-name none
Repository [none] successfully created

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo sync run --repo-id none
+----------------------------------------------------------------------+
                    Synchronizing Repository [none]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Retrieving metadata
[-]
... failed
NOT FOUND

Task Failed

NOT FOUND

[root@ip-172-31-20-145 ~]#

#12 Updated by dkliban@redhat.com about 5 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE

#15 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF