Project

Profile

Help

Issue #3960

Add support to handle pagination for tags/list endpoint

Added by ipanova@redhat.com 12 months ago. Updated 4 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Severity:
2. Medium
Version - Docker:
Platform Release:
2.17.1
Blocks Release:
Target Release - Docker:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 42

Description

During sync we query list of tags present in the repo. [0] [1]

Pagination support is available for Tags API, we need update our sync code to cope with this change.
We'd need to check the `Link` header if it is present in the response headers.
For more doc check [2]


1st request:
$ curl -i https://quay.io/v2/coreos/etcd/tags/list
HTTP/2 200 
server: nginx/1.14.0
date: Fri, 31 Aug 2018 13:13:56 GMT
content-type: application/json
content-length: 602
link: </v2/coreos/etcd/tags/list?next_page=gAAAAABbiT8UvBiyJvd3H5d7FS3azkmxqQuknl_2kKlKkbLpj_vGL_xv26HMBevZIVKY67vwpbKUo_Zact9vC6mjFd9KHGFGpkUHGc2jqGtd5aWpGbaIfqA%3D&n=50>; rel="next" 
x-frame-options: DENY
strict-transport-security: max-age=63072000; preload

{"name":"coreos/etcd","tags":["v0.4.6","v0.5.0_alpha.0","v0.5.0_alpha.1","v0.5.0_alpha.2","v0.5.0_alpha.3","test","v0.5.0_alpha.4","v0.5.0_alpha.5","v2.0.0_rc.1","v2.0.0","v2.0.3","v2.0.4","v2.0.5","v0.4.8","v2.0.6","v2.0.7","v2.0.8","v2.0.9","v2.0.10","v2.1.0-alpha.0","v2.0.11","v2.0.12","v2.1.0-alpha.1","v2.0.13","v2.1.0-rc.0","v2.1.1","v2.2.0-alpha.0","v2.2.0-alpha.1","v2.1.2","v2.2.0-rc.0","v2.1.3","v2.2.0","v2.2.1","v2.3.0-alpha.0","v2.2.2","v2.2.3","v2.2.4","v2.2.5","v2.3.0-alpha.1","v2.3.0","v2.3.1","v2.3.2","v2.3.3","v3.0.0-beta.0","v2.3.4","v2.3.5","v2.3.6","v2.3.7","v3.0.0","v3.0.1"]}

2. check if Link is present. Then follow it until the Link disappears from the headers

$ curl -i 'https://quay.io/v2/coreos/etcd/tags/list?next_page=gAAAAABbiT8UvBiyJvd3H5d7FS3azkmxqQuknl_2kKlKkbLpj_vGL_xv26HMBevZIVKY67vwpbKUo_Zact9vC6mjFd9KHGFGpkUHGc2jqGtd5aWpGbaIfqA%3D&n=50'
HTTP/2 200 
server: nginx/1.14.0
date: Fri, 31 Aug 2018 13:15:36 GMT
content-type: application/json
content-length: 623
link: </v2/coreos/etcd/tags/list?next_page=gAAAAABbiT94xBPZc4VUD9_hx1bmHtfl1Xduxm811b1m06WiwW2mOHErnpkbtZ7Ag7ehYVPVAf3LfS2s-FVSrkCXhUqNLoduaGJaBorCUzp7NPlybCickC8%3D&n=50>; rel="next" 
x-frame-options: DENY
strict-transport-security: max-age=63072000; preload

{"name":"coreos/etcd","tags":["v3.0.2","v3.0.3","v3.0.4","v3.0.5","v3.0.6","v3.0.7","v3.0.8","v3.0.9","v3.1.0-alpha.0","v3.0.10","v3.1.0-alpha.1","v3.0.11","v3.0.12","v3.1.0-rc.0","v3.0.13","v3.0.14","v3.0.15","v3.1.0-rc.1","v3.0.16","v3.0.17","v3.1.0","v2.3.8","v3.1.1","v3.1.2","v3.1.3","v3.1.4","v3.1.5","v3.1.6","v3.1.7","v3.2.0-rc.0","v3.2.0-rc.0-arm64","v3.2.0-rc.0-ppc64le","v3.1.8","v3.2.0-rc.1","v3.2.0-rc.1-arm64","v3.2.0-rc.1-ppc64le","v3.1.9","v3.2.0","v3.2.0-ppc64le","v3.2.1","v3.2.1-arm64","v3.2.1-ppc64le","v3.2.2","v3.2.2-arm64","v3.2.2-ppc64le","v3.0","v3.1.10","v3.2.3","v3.2.3-arm64","v3.2.3-ppc64le"]}

NOTE if registry does not support pagination, the Link header will be absent.
[0] https://github.com/pulp/pulp_docker/blob/2-master/plugins/pulp_docker/plugins/importers/sync.py#L224
[1] https://github.com/pulp/pulp_docker/blob/2-master/plugins/pulp_docker/plugins/registry.py#L450
[2] https://docs.docker.com/registry/spec/api/#listing-image-tags


Related issues

Duplicated by Docker Support - Issue #4408: Pulp crane not returning pagination info for tags/list CLOSED - DUPLICATE Actions

Associated revisions

Revision 0c5d1368 View on GitHub
Added by ipanova@redhat.com 12 months ago

Add support to handle pagination for tags/list endpoint.

closes #3960
https://pulp.plan.io/issues/3960

Revision 0c5d1368 View on GitHub
Added by ipanova@redhat.com 12 months ago

Add support to handle pagination for tags/list endpoint.

closes #3960
https://pulp.plan.io/issues/3960

Revision ca083c7c View on GitHub
Added by ipanova@redhat.com 11 months ago

Add support to handle pagination for tags/list endpoint.

closes #3960
https://pulp.plan.io/issues/3960

(cherry picked from commit 0c5d13680b2154622ebedb63874f6b187d614f2c)

Revision ca083c7c View on GitHub
Added by ipanova@redhat.com 11 months ago

Add support to handle pagination for tags/list endpoint.

closes #3960
https://pulp.plan.io/issues/3960

(cherry picked from commit 0c5d13680b2154622ebedb63874f6b187d614f2c)

History

#2 Updated by ipanova@redhat.com 12 months ago

  • Description updated (diff)

#3 Updated by daviddavis 12 months ago

  • Sprint set to Sprint 42

#4 Updated by daviddavis 12 months ago

  • Triaged changed from No to Yes

#5 Updated by ipanova@redhat.com 12 months ago

  • Assignee set to ipanova@redhat.com
  • Priority changed from Normal to High

#6 Updated by ipanova@redhat.com 12 months ago

  • Status changed from NEW to POST

#7 Updated by ipanova@redhat.com 12 months ago

  • Sprint/Milestone set to 2.18.0

#8 Updated by ipanova@redhat.com 12 months ago

  • Status changed from POST to MODIFIED

#9 Updated by ipanova@redhat.com 12 months ago

  • Sprint/Milestone deleted (2.18.0)

#10 Updated by dkliban@redhat.com 11 months ago

  • Platform Release set to 2.17.1

#12 Updated by ttereshc 11 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

#13 Updated by ipanova@redhat.com 5 months ago

  • Duplicated by Issue #4408: Pulp crane not returning pagination info for tags/list added

#14 Updated by bmbouter 4 months ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF