Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #914

closed

Repo sync broken

Added by mkovacik@redhat.com over 9 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
mhrivnak
Start date:
Due date:
Estimated time:
Severity:
3. High
Version - Docker:
1.0.0
Platform Release:
Target Release - Docker:
1.0.1
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

The repo synchronization is broken in pulp-docker-plugins-1.0.0-1.fc21.noarch
The Registry Tags API has changed recently --- see attached patch that fixes the issue for me

How reproducible:
Always

Steps to reproduce:
- pulp-admin docker repo create --repo-id=busybox --feed=https://index.docker.io --upstream-name=busybox
- pulp-admin docker repo sync run --repo-id=busybox

Actual result:
Sync task fails with an IOError NOT FOUND

Expected result:
Sync task finishes OK

Notes:
Breaks pulp automation test case: https://github.com/RedHatQE/pulp-automation/blob/master/tests/general_tests/test_20_docker_repo_cud.py

Files

registry.py.diff (925 Bytes) registry.py.diff mkovacik@redhat.com, 04/24/2015 01:43 AM
Actions
Copy link
 #1

Updated by mhrivnak over 9 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to mhrivnak
  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes
Actions
Copy link
 #2

Updated by mhrivnak over 9 years ago

Thanks for catching this. I confirmed the problem and solution. It's extra-confusing because right now, getting tags for the busybox repo from either index.docker.io, or registry-1.docker.io, both work but return different image IDs. Maybe the former has stale data that they're no longer updating?

Have you been able to find other repositories that are broken in this way? I was able to sync "pulp/worker" just fine, so it seems not all of them have the same mismatch of tag data.

Actions
Copy link
 #3

Updated by mkovacik@redhat.com over 9 years ago

mhrivnak wrote:

Thanks for catching this. I confirmed the problem and solution. It's extra-confusing because right now, getting tags for the busybox repo from either index.docker.io, or registry-1.docker.io, both work but return different image IDs. Maybe the former has stale data that they're no longer updating?

It seems the index url https://index.docker.io/v1/*/ is now being used just to get the Authorization Token. All other request are supposed to go through the endpoints specified in the X-Docker-Endpoints header and have to be authorized //1.
At least that works for now.

Have you been able to find other repositories that are broken in this way? I was able to sync "pulp/worker" just fine, so it seems not all of them have the same mismatch of tag data.

My Public personal repo is affected https://registry.hub.docker.com/repos/dparalen/ and the patch works for me in this case.
I'm able to sync "pulp/worker" having applied the patch.
Maybe only newer and library repos are affected, can't tell.

Cheers,
milan

//1 looking into some other projects implementation, https://github.com/CenturyLinkLabs/docker-reg-client ; some trial&error with curl ; following docker CLI tool logs ; googlin'

Actions
Copy link
 #4

Updated by mhrivnak over 9 years ago

  • Version changed from 2.6.1 Beta to 1.0.0
Actions
Copy link
 #5

Updated by mhrivnak over 9 years ago

  • Status changed from ASSIGNED to POST
  • Target Release - Docker set to 1.0.1

Added by mhrivnak over 9 years ago

Revision 8081b301 | View on GitHub

The API for fetching tags from a remote registry or index changed. This uses the new API.

closes #914

Added by mhrivnak over 9 years ago

Revision 8081b301 | View on GitHub

The API for fetching tags from a remote registry or index changed. This uses the new API.

closes #914

Added by mhrivnak over 9 years ago

Revision 8081b301 | View on GitHub

The API for fetching tags from a remote registry or index changed. This uses the new API.

closes #914

Added by mhrivnak over 9 years ago

Revision 8081b301 | View on GitHub

The API for fetching tags from a remote registry or index changed. This uses the new API.

closes #914

Actions
Copy link
 #6

Updated by mhrivnak over 9 years ago

  • Status changed from POST to MODIFIED
Actions
Copy link
 #7

Updated by mhrivnak over 9 years ago

  • % Done changed from 0 to 100
Actions
Copy link
 #8

Updated by bmbouter over 9 years ago

  • Version deleted (1.0.0)
  • Version - Docker set to 1.0.0
Actions
Copy link
 #9

Updated by dkliban@redhat.com over 9 years ago

  • Status changed from MODIFIED to 5
Actions
Copy link
 #11

Updated by mkovacik@redhat.com over 9 years ago

  • Status changed from 5 to 6

Verified with:

  • Red Hat Enterprise Linux Server release 6.6 (Santiago)
  • pulp-server-2.6.2-0.2.beta.el6.noarch
  • pulp-docker-plugins-1.0.1-0.2.beta.el6.noarch
[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo create --repo-id busybox --upstream-name busybox --feed https://index.docker.io
Repository [busybox] successfully created

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo sync run --repo-id busybox
+----------------------------------------------------------------------+
                   Synchronizing Repository [busybox]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Retrieving metadata
[/]
... completed

Copying units already in pulp
[-]
... completed

Downloading remote files
[==================================================] 100%
23 of 23 items
... completed

Saving images and tags
[-]
... completed

Task Succeeded

Publishing Image Files.
[==================================================] 100%
9 of 9 items
... completed

Making files available via web.
[-]
... completed

Task Succeeded

[root@ip-172-31-20-145 ~]# pulp-admin  -u admin -p admin docker repo delete --repo-id pulp-automation
This command may be exited via ctrl+c without affecting the request.

[\]
Running...

Repository [pulp-automation] successfully deleted

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo create --repo-id hello-world --upstream-name hello-world --feed https://index.docker.io
Repository [hello-world] successfully created

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo sync run --repo-id hello-world
+----------------------------------------------------------------------+
                 Synchronizing Repository [hello-world]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Retrieving metadata
[|]
... completed

Copying units already in pulp
[-]
... completed

Downloading remote files
[==================================================] 100%
5 of 5 items
... completed

Saving images and tags
[-]
... completed

Task Succeeded

Publishing Image Files.
[==================================================] 100%
2 of 2 items
... completed

Making files available via web.
[-]
... completed

Task Succeeded

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo create --repo-id pulp-automation --feed=https://index.docker.io --upstream-name dparalen/pulp-automation
Repository [pulp-automation] successfully created

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo sync run --repo-id hello-world
+----------------------------------------------------------------------+
                 Synchronizing Repository [hello-world]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Retrieving metadata
[-]
... completed

Copying units already in pulp
[-]
... completed

Downloading remote files
[-]
... completed

Saving images and tags
[-]
... completed

Task Succeeded

Publishing Image Files.
[==================================================] 100%
2 of 2 items
... completed

Making files available via web.
[-]
... completed

Task Succeeded

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo sync run --repo-id pulp-automation
+----------------------------------------------------------------------+
               Synchronizing Repository [pulp-automation]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Retrieving metadata
[/]
... completed

Copying units already in pulp
[-]
... completed

Downloading remote files
[==================================================] 100%
53 of 53 items
... completed

Saving images and tags
[-]
... completed

Task Succeeded

Publishing Image Files.
[==================================================] 100%
18 of 18 items
... completed

Making files available via web.
[-]
... completed

Task Succeeded

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo create --repo-id none --feed=https://index.docker.io --upstream-name none
Repository [none] successfully created

[root@ip-172-31-20-145 ~]# pulp-admin -u admin -p admin docker repo sync run --repo-id none
+----------------------------------------------------------------------+
                    Synchronizing Repository [none]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Retrieving metadata
[-]
... failed
NOT FOUND

Task Failed

NOT FOUND

[root@ip-172-31-20-145 ~]#
Actions
Copy link
 #12

Updated by dkliban@redhat.com over 9 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE
Actions
Copy link
 #15

Updated by bmbouter over 5 years ago

  • Tags Pulp 2 added
Actions
Send by e-mail Copy link

Also available in: Atom PDF