Project

Profile

Help

Task #4168

Add support to handle pagination for tags/list endpoint during sync

Added by ipanova@redhat.com about 1 year ago. Updated 7 months ago.

Status:
MODIFIED
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Platform Release:
Blocks Release:
Target Release - Docker:
Backwards Incompatible:
No
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 51

Description

During sync we query list of tags present in the repo.

Pagination support is available for Tags API, we need update our sync code to cope with this change.
We'd need to check the `Link` header if it is present in the response headers.
For more docs check [0]
For Pulp2 implementation check [1]


1st request:
$ curl -i https://quay.io/v2/coreos/etcd/tags/list
HTTP/2 200 
server: nginx/1.14.0
date: Fri, 31 Aug 2018 13:13:56 GMT
content-type: application/json
content-length: 602
link: </v2/coreos/etcd/tags/list?next_page=gAAAAABbiT8UvBiyJvd3H5d7FS3azkmxqQuknl_2kKlKkbLpj_vGL_xv26HMBevZIVKY67vwpbKUo_Zact9vC6mjFd9KHGFGpkUHGc2jqGtd5aWpGbaIfqA%3D&n=50>; rel="next" 
x-frame-options: DENY
strict-transport-security: max-age=63072000; preload

{"name":"coreos/etcd","tags":["v0.4.6","v0.5.0_alpha.0","v0.5.0_alpha.1","v0.5.0_alpha.2","v0.5.0_alpha.3","test","v0.5.0_alpha.4","v0.5.0_alpha.5","v2.0.0_rc.1","v2.0.0","v2.0.3","v2.0.4","v2.0.5","v0.4.8","v2.0.6","v2.0.7","v2.0.8","v2.0.9","v2.0.10","v2.1.0-alpha.0","v2.0.11","v2.0.12","v2.1.0-alpha.1","v2.0.13","v2.1.0-rc.0","v2.1.1","v2.2.0-alpha.0","v2.2.0-alpha.1","v2.1.2","v2.2.0-rc.0","v2.1.3","v2.2.0","v2.2.1","v2.3.0-alpha.0","v2.2.2","v2.2.3","v2.2.4","v2.2.5","v2.3.0-alpha.1","v2.3.0","v2.3.1","v2.3.2","v2.3.3","v3.0.0-beta.0","v2.3.4","v2.3.5","v2.3.6","v2.3.7","v3.0.0","v3.0.1"]}

2. check if Link is present. Then follow it until the Link disappears from the headers

$ curl -i 'https://quay.io/v2/coreos/etcd/tags/list?next_page=gAAAAABbiT8UvBiyJvd3H5d7FS3azkmxqQuknl_2kKlKkbLpj_vGL_xv26HMBevZIVKY67vwpbKUo_Zact9vC6mjFd9KHGFGpkUHGc2jqGtd5aWpGbaIfqA%3D&n=50'
HTTP/2 200 
server: nginx/1.14.0
date: Fri, 31 Aug 2018 13:15:36 GMT
content-type: application/json
content-length: 623
link: </v2/coreos/etcd/tags/list?next_page=gAAAAABbiT94xBPZc4VUD9_hx1bmHtfl1Xduxm811b1m06WiwW2mOHErnpkbtZ7Ag7ehYVPVAf3LfS2s-FVSrkCXhUqNLoduaGJaBorCUzp7NPlybCickC8%3D&n=50>; rel="next" 
x-frame-options: DENY
strict-transport-security: max-age=63072000; preload

{"name":"coreos/etcd","tags":["v3.0.2","v3.0.3","v3.0.4","v3.0.5","v3.0.6","v3.0.7","v3.0.8","v3.0.9","v3.1.0-alpha.0","v3.0.10","v3.1.0-alpha.1","v3.0.11","v3.0.12","v3.1.0-rc.0","v3.0.13","v3.0.14","v3.0.15","v3.1.0-rc.1","v3.0.16","v3.0.17","v3.1.0","v2.3.8","v3.1.1","v3.1.2","v3.1.3","v3.1.4","v3.1.5","v3.1.6","v3.1.7","v3.2.0-rc.0","v3.2.0-rc.0-arm64","v3.2.0-rc.0-ppc64le","v3.1.8","v3.2.0-rc.1","v3.2.0-rc.1-arm64","v3.2.0-rc.1-ppc64le","v3.1.9","v3.2.0","v3.2.0-ppc64le","v3.2.1","v3.2.1-arm64","v3.2.1-ppc64le","v3.2.2","v3.2.2-arm64","v3.2.2-ppc64le","v3.0","v3.1.10","v3.2.3","v3.2.3-arm64","v3.2.3-ppc64le"]}

NOTE if registry does not support pagination, the Link header will be absent.
[0] https://docs.docker.com/registry/spec/api/#listing-image-tags
[1] https://github.com/pulp/pulp_docker/pull/262


Related issues

Related to Pulp - Test #4755: Test support to handle pagination for tags/list endpoint during sync NEW Actions

Associated revisions

Revision 03d1bc3c View on GitHub
Added by ipanova@redhat.com 7 months ago

Add support to handle pagination for tags/list endpoint during sync

closes #4168
https://pulp.plan.io/issues/4168

Revision 03d1bc3c View on GitHub
Added by ipanova@redhat.com 7 months ago

Add support to handle pagination for tags/list endpoint during sync

closes #4168
https://pulp.plan.io/issues/4168

History

#1 Updated by ipanova@redhat.com 8 months ago

  • Sprint set to Sprint 51

#2 Updated by ipanova@redhat.com 7 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ipanova@redhat.com

#3 Updated by ipanova@redhat.com 7 months ago

  • Status changed from ASSIGNED to POST

#4 Updated by ipanova@redhat.com 7 months ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#5 Updated by bmbouter 7 months ago

  • Tags deleted (Pulp 3)

#6 Updated by kersom 7 months ago

  • Related to Test #4755: Test support to handle pagination for tags/list endpoint during sync added

Please register to edit this issue

Also available in: Atom PDF