Project

Profile

Help

Test #4755

closed

Test support to handle pagination for tags/list endpoint during sync

Added by kersom almost 5 years ago. Updated about 2 years ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Version:
Platform Release:
Tags:
Sprint:
Quarter:

Description

Ticket moved to GitHub: "pulp/pulpcore/1823":https://github.com/pulp/pulpcore/issues/1823


During sync we query list of tags present in the repo.

Pagination support is available for Tags API, we need update our sync code to cope with this change.
We'd need to check the `Link` header if it is present in the response headers.
For more docs check [0]
For Pulp2 implementation check [1]


1st request:
$ curl -i https://quay.io/v2/coreos/etcd/tags/list
HTTP/2 200 
server: nginx/1.14.0
date: Fri, 31 Aug 2018 13:13:56 GMT
content-type: application/json
content-length: 602
link: </v2/coreos/etcd/tags/list?next_page=gAAAAABbiT8UvBiyJvd3H5d7FS3azkmxqQuknl_2kKlKkbLpj_vGL_xv26HMBevZIVKY67vwpbKUo_Zact9vC6mjFd9KHGFGpkUHGc2jqGtd5aWpGbaIfqA%3D&n=50>; rel="next"
x-frame-options: DENY
strict-transport-security: max-age=63072000; preload

{"name":"coreos/etcd","tags":["v0.4.6","v0.5.0_alpha.0","v0.5.0_alpha.1","v0.5.0_alpha.2","v0.5.0_alpha.3","test","v0.5.0_alpha.4","v0.5.0_alpha.5","v2.0.0_rc.1","v2.0.0","v2.0.3","v2.0.4","v2.0.5","v0.4.8","v2.0.6","v2.0.7","v2.0.8","v2.0.9","v2.0.10","v2.1.0-alpha.0","v2.0.11","v2.0.12","v2.1.0-alpha.1","v2.0.13","v2.1.0-rc.0","v2.1.1","v2.2.0-alpha.0","v2.2.0-alpha.1","v2.1.2","v2.2.0-rc.0","v2.1.3","v2.2.0","v2.2.1","v2.3.0-alpha.0","v2.2.2","v2.2.3","v2.2.4","v2.2.5","v2.3.0-alpha.1","v2.3.0","v2.3.1","v2.3.2","v2.3.3","v3.0.0-beta.0","v2.3.4","v2.3.5","v2.3.6","v2.3.7","v3.0.0","v3.0.1"]}

2. check if Link is present. Then follow it until the Link disappears from the headers

$ curl -i 'https://quay.io/v2/coreos/etcd/tags/list?next_page=gAAAAABbiT8UvBiyJvd3H5d7FS3azkmxqQuknl_2kKlKkbLpj_vGL_xv26HMBevZIVKY67vwpbKUo_Zact9vC6mjFd9KHGFGpkUHGc2jqGtd5aWpGbaIfqA%3D&n=50'
HTTP/2 200 
server: nginx/1.14.0
date: Fri, 31 Aug 2018 13:15:36 GMT
content-type: application/json
content-length: 623
link: </v2/coreos/etcd/tags/list?next_page=gAAAAABbiT94xBPZc4VUD9_hx1bmHtfl1Xduxm811b1m06WiwW2mOHErnpkbtZ7Ag7ehYVPVAf3LfS2s-FVSrkCXhUqNLoduaGJaBorCUzp7NPlybCickC8%3D&n=50>; rel="next"
x-frame-options: DENY
strict-transport-security: max-age=63072000; preload

{"name":"coreos/etcd","tags":["v3.0.2","v3.0.3","v3.0.4","v3.0.5","v3.0.6","v3.0.7","v3.0.8","v3.0.9","v3.1.0-alpha.0","v3.0.10","v3.1.0-alpha.1","v3.0.11","v3.0.12","v3.1.0-rc.0","v3.0.13","v3.0.14","v3.0.15","v3.1.0-rc.1","v3.0.16","v3.0.17","v3.1.0","v2.3.8","v3.1.1","v3.1.2","v3.1.3","v3.1.4","v3.1.5","v3.1.6","v3.1.7","v3.2.0-rc.0","v3.2.0-rc.0-arm64","v3.2.0-rc.0-ppc64le","v3.1.8","v3.2.0-rc.1","v3.2.0-rc.1-arm64","v3.2.0-rc.1-ppc64le","v3.1.9","v3.2.0","v3.2.0-ppc64le","v3.2.1","v3.2.1-arm64","v3.2.1-ppc64le","v3.2.2","v3.2.2-arm64","v3.2.2-ppc64le","v3.0","v3.1.10","v3.2.3","v3.2.3-arm64","v3.2.3-ppc64le"]}

NOTE if registry does not support pagination, the Link header will be absent.
[0] https://docs.docker.com/registry/spec/api/#listing-image-tags
[1] https://github.com/pulp/pulp_docker/pull/262

Notes from triage:
Sync from quay should do it.


Related issues

Related to Container Support - Task #4168: Add support to handle pagination for tags/list endpoint during syncCLOSED - CURRENTRELEASEipanova@redhat.com

Actions
Actions #1

Updated by kersom almost 5 years ago

  • Related to Task #4168: Add support to handle pagination for tags/list endpoint during sync added
Actions #2

Updated by kersom almost 5 years ago

  • Description updated (diff)
Actions #3

Updated by kersom almost 5 years ago

  • Status changed from MODIFIED to NEW
Actions #4

Updated by pulpbot about 2 years ago

  • Description updated (diff)
  • Status changed from NEW to CLOSED - DUPLICATE

Also available in: Atom PDF