Project

Profile

Help

Issue #3960

closed

Add support to handle pagination for tags/list endpoint

Added by ipanova@redhat.com over 6 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version - Docker:
Platform Release:
2.17.1
Target Release - Docker:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 42
Quarter:

Description

During sync we query list of tags present in the repo. [0] [1]

Pagination support is available for Tags API, we need update our sync code to cope with this change.
We'd need to check the `Link` header if it is present in the response headers.
For more doc check [2]


1st request:
$ curl -i https://quay.io/v2/coreos/etcd/tags/list
HTTP/2 200 
server: nginx/1.14.0
date: Fri, 31 Aug 2018 13:13:56 GMT
content-type: application/json
content-length: 602
link: </v2/coreos/etcd/tags/list?next_page=gAAAAABbiT8UvBiyJvd3H5d7FS3azkmxqQuknl_2kKlKkbLpj_vGL_xv26HMBevZIVKY67vwpbKUo_Zact9vC6mjFd9KHGFGpkUHGc2jqGtd5aWpGbaIfqA%3D&n=50>; rel="next"
x-frame-options: DENY
strict-transport-security: max-age=63072000; preload

{"name":"coreos/etcd","tags":["v0.4.6","v0.5.0_alpha.0","v0.5.0_alpha.1","v0.5.0_alpha.2","v0.5.0_alpha.3","test","v0.5.0_alpha.4","v0.5.0_alpha.5","v2.0.0_rc.1","v2.0.0","v2.0.3","v2.0.4","v2.0.5","v0.4.8","v2.0.6","v2.0.7","v2.0.8","v2.0.9","v2.0.10","v2.1.0-alpha.0","v2.0.11","v2.0.12","v2.1.0-alpha.1","v2.0.13","v2.1.0-rc.0","v2.1.1","v2.2.0-alpha.0","v2.2.0-alpha.1","v2.1.2","v2.2.0-rc.0","v2.1.3","v2.2.0","v2.2.1","v2.3.0-alpha.0","v2.2.2","v2.2.3","v2.2.4","v2.2.5","v2.3.0-alpha.1","v2.3.0","v2.3.1","v2.3.2","v2.3.3","v3.0.0-beta.0","v2.3.4","v2.3.5","v2.3.6","v2.3.7","v3.0.0","v3.0.1"]}

2. check if Link is present. Then follow it until the Link disappears from the headers

$ curl -i 'https://quay.io/v2/coreos/etcd/tags/list?next_page=gAAAAABbiT8UvBiyJvd3H5d7FS3azkmxqQuknl_2kKlKkbLpj_vGL_xv26HMBevZIVKY67vwpbKUo_Zact9vC6mjFd9KHGFGpkUHGc2jqGtd5aWpGbaIfqA%3D&n=50'
HTTP/2 200 
server: nginx/1.14.0
date: Fri, 31 Aug 2018 13:15:36 GMT
content-type: application/json
content-length: 623
link: </v2/coreos/etcd/tags/list?next_page=gAAAAABbiT94xBPZc4VUD9_hx1bmHtfl1Xduxm811b1m06WiwW2mOHErnpkbtZ7Ag7ehYVPVAf3LfS2s-FVSrkCXhUqNLoduaGJaBorCUzp7NPlybCickC8%3D&n=50>; rel="next"
x-frame-options: DENY
strict-transport-security: max-age=63072000; preload

{"name":"coreos/etcd","tags":["v3.0.2","v3.0.3","v3.0.4","v3.0.5","v3.0.6","v3.0.7","v3.0.8","v3.0.9","v3.1.0-alpha.0","v3.0.10","v3.1.0-alpha.1","v3.0.11","v3.0.12","v3.1.0-rc.0","v3.0.13","v3.0.14","v3.0.15","v3.1.0-rc.1","v3.0.16","v3.0.17","v3.1.0","v2.3.8","v3.1.1","v3.1.2","v3.1.3","v3.1.4","v3.1.5","v3.1.6","v3.1.7","v3.2.0-rc.0","v3.2.0-rc.0-arm64","v3.2.0-rc.0-ppc64le","v3.1.8","v3.2.0-rc.1","v3.2.0-rc.1-arm64","v3.2.0-rc.1-ppc64le","v3.1.9","v3.2.0","v3.2.0-ppc64le","v3.2.1","v3.2.1-arm64","v3.2.1-ppc64le","v3.2.2","v3.2.2-arm64","v3.2.2-ppc64le","v3.0","v3.1.10","v3.2.3","v3.2.3-arm64","v3.2.3-ppc64le"]}

NOTE if registry does not support pagination, the Link header will be absent.
[0] https://github.com/pulp/pulp_docker/blob/2-master/plugins/pulp_docker/plugins/importers/sync.py#L224
[1] https://github.com/pulp/pulp_docker/blob/2-master/plugins/pulp_docker/plugins/registry.py#L450
[2] https://docs.docker.com/registry/spec/api/#listing-image-tags


Related issues

Has duplicate Docker Support - Issue #4408: Pulp crane not returning pagination info for tags/listCLOSED - DUPLICATEActions
Actions #2

Updated by ipanova@redhat.com over 6 years ago

  • Description updated (diff)
Actions #3

Updated by daviddavis over 6 years ago

  • Sprint set to Sprint 42
Actions #4

Updated by daviddavis over 6 years ago

  • Triaged changed from No to Yes
Actions #5

Updated by ipanova@redhat.com over 6 years ago

  • Assignee set to ipanova@redhat.com
  • Priority changed from Normal to High
Actions #6

Updated by ipanova@redhat.com over 6 years ago

  • Status changed from NEW to POST

Added by ipanova@redhat.com over 6 years ago

Revision 0c5d1368 | View on GitHub

Add support to handle pagination for tags/list endpoint.

closes #3960 https://pulp.plan.io/issues/3960

Added by ipanova@redhat.com over 6 years ago

Revision 0c5d1368 | View on GitHub

Add support to handle pagination for tags/list endpoint.

closes #3960 https://pulp.plan.io/issues/3960

Actions #7

Updated by ipanova@redhat.com over 6 years ago

  • Sprint/Milestone set to 2.18.0
Actions #8

Updated by ipanova@redhat.com over 6 years ago

  • Status changed from POST to MODIFIED
Actions #9

Updated by ipanova@redhat.com over 6 years ago

  • Sprint/Milestone deleted (2.18.0)
Actions #10

Updated by dkliban@redhat.com over 6 years ago

  • Platform Release set to 2.17.1

Added by ipanova@redhat.com over 6 years ago

Revision ca083c7c | View on GitHub

Add support to handle pagination for tags/list endpoint.

closes #3960 https://pulp.plan.io/issues/3960

(cherry picked from commit 0c5d13680b2154622ebedb63874f6b187d614f2c)

Added by ipanova@redhat.com over 6 years ago

Revision ca083c7c | View on GitHub

Add support to handle pagination for tags/list endpoint.

closes #3960 https://pulp.plan.io/issues/3960

(cherry picked from commit 0c5d13680b2154622ebedb63874f6b187d614f2c)

Actions #12

Updated by ttereshc about 6 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #13

Updated by ipanova@redhat.com over 5 years ago

  • Has duplicate Issue #4408: Pulp crane not returning pagination info for tags/list added
Actions #14

Updated by bmbouter over 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF