Project

Profile

Help

Task #4168

Add support to handle pagination for tags/list endpoint during sync

Added by ipanova@redhat.com over 1 year ago. Updated about 1 month ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Platform Release:
Blocks Release:
Backwards Incompatible:
No
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 51

Description

During sync we query list of tags present in the repo.

Pagination support is available for Tags API, we need update our sync code to cope with this change.
We'd need to check the `Link` header if it is present in the response headers.
For more docs check [0]
For Pulp2 implementation check [1]


1st request:
$ curl -i https://quay.io/v2/coreos/etcd/tags/list
HTTP/2 200 
server: nginx/1.14.0
date: Fri, 31 Aug 2018 13:13:56 GMT
content-type: application/json
content-length: 602
link: </v2/coreos/etcd/tags/list?next_page=gAAAAABbiT8UvBiyJvd3H5d7FS3azkmxqQuknl_2kKlKkbLpj_vGL_xv26HMBevZIVKY67vwpbKUo_Zact9vC6mjFd9KHGFGpkUHGc2jqGtd5aWpGbaIfqA%3D&n=50>; rel="next"
x-frame-options: DENY
strict-transport-security: max-age=63072000; preload

{"name":"coreos/etcd","tags":["v0.4.6","v0.5.0_alpha.0","v0.5.0_alpha.1","v0.5.0_alpha.2","v0.5.0_alpha.3","test","v0.5.0_alpha.4","v0.5.0_alpha.5","v2.0.0_rc.1","v2.0.0","v2.0.3","v2.0.4","v2.0.5","v0.4.8","v2.0.6","v2.0.7","v2.0.8","v2.0.9","v2.0.10","v2.1.0-alpha.0","v2.0.11","v2.0.12","v2.1.0-alpha.1","v2.0.13","v2.1.0-rc.0","v2.1.1","v2.2.0-alpha.0","v2.2.0-alpha.1","v2.1.2","v2.2.0-rc.0","v2.1.3","v2.2.0","v2.2.1","v2.3.0-alpha.0","v2.2.2","v2.2.3","v2.2.4","v2.2.5","v2.3.0-alpha.1","v2.3.0","v2.3.1","v2.3.2","v2.3.3","v3.0.0-beta.0","v2.3.4","v2.3.5","v2.3.6","v2.3.7","v3.0.0","v3.0.1"]}

2. check if Link is present. Then follow it until the Link disappears from the headers

$ curl -i 'https://quay.io/v2/coreos/etcd/tags/list?next_page=gAAAAABbiT8UvBiyJvd3H5d7FS3azkmxqQuknl_2kKlKkbLpj_vGL_xv26HMBevZIVKY67vwpbKUo_Zact9vC6mjFd9KHGFGpkUHGc2jqGtd5aWpGbaIfqA%3D&n=50'
HTTP/2 200 
server: nginx/1.14.0
date: Fri, 31 Aug 2018 13:15:36 GMT
content-type: application/json
content-length: 623
link: </v2/coreos/etcd/tags/list?next_page=gAAAAABbiT94xBPZc4VUD9_hx1bmHtfl1Xduxm811b1m06WiwW2mOHErnpkbtZ7Ag7ehYVPVAf3LfS2s-FVSrkCXhUqNLoduaGJaBorCUzp7NPlybCickC8%3D&n=50>; rel="next"
x-frame-options: DENY
strict-transport-security: max-age=63072000; preload

{"name":"coreos/etcd","tags":["v3.0.2","v3.0.3","v3.0.4","v3.0.5","v3.0.6","v3.0.7","v3.0.8","v3.0.9","v3.1.0-alpha.0","v3.0.10","v3.1.0-alpha.1","v3.0.11","v3.0.12","v3.1.0-rc.0","v3.0.13","v3.0.14","v3.0.15","v3.1.0-rc.1","v3.0.16","v3.0.17","v3.1.0","v2.3.8","v3.1.1","v3.1.2","v3.1.3","v3.1.4","v3.1.5","v3.1.6","v3.1.7","v3.2.0-rc.0","v3.2.0-rc.0-arm64","v3.2.0-rc.0-ppc64le","v3.1.8","v3.2.0-rc.1","v3.2.0-rc.1-arm64","v3.2.0-rc.1-ppc64le","v3.1.9","v3.2.0","v3.2.0-ppc64le","v3.2.1","v3.2.1-arm64","v3.2.1-ppc64le","v3.2.2","v3.2.2-arm64","v3.2.2-ppc64le","v3.0","v3.1.10","v3.2.3","v3.2.3-arm64","v3.2.3-ppc64le"]}

NOTE if registry does not support pagination, the Link header will be absent.
[0] https://docs.docker.com/registry/spec/api/#listing-image-tags
[1] https://github.com/pulp/pulp_docker/pull/262


Related issues

Related to Pulp - Test #4755: Test support to handle pagination for tags/list endpoint during sync NEW Actions

Associated revisions

Revision 03d1bc3c View on GitHub
Added by ipanova@redhat.com 11 months ago

Add support to handle pagination for tags/list endpoint during sync

closes #4168 https://pulp.plan.io/issues/4168

Revision 03d1bc3c View on GitHub
Added by ipanova@redhat.com 11 months ago

Add support to handle pagination for tags/list endpoint during sync

closes #4168 https://pulp.plan.io/issues/4168

Revision 03d1bc3c View on GitHub
Added by ipanova@redhat.com 11 months ago

Add support to handle pagination for tags/list endpoint during sync

closes #4168 https://pulp.plan.io/issues/4168

Revision 03d1bc3c View on GitHub
Added by ipanova@redhat.com 11 months ago

Add support to handle pagination for tags/list endpoint during sync

closes #4168 https://pulp.plan.io/issues/4168

History

#1 Updated by ipanova@redhat.com 11 months ago

  • Sprint set to Sprint 51

#2 Updated by ipanova@redhat.com 11 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ipanova@redhat.com

#3 Updated by ipanova@redhat.com 11 months ago

  • Status changed from ASSIGNED to POST

#4 Updated by ipanova@redhat.com 11 months ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

Applied in changeset commit:pulp_docker|03d1bc3c0548bfc7b963a8c553bd7d55770e9566.

#5 Updated by bmbouter 10 months ago

  • Tags deleted (Pulp 3)

#6 Updated by kersom 10 months ago

  • Related to Test #4755: Test support to handle pagination for tags/list endpoint during sync added

#7 Updated by ipanova@redhat.com 3 months ago

  • Project changed from Docker Support to Container Support

#8 Updated by ipanova@redhat.com 3 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

#9 Updated by ipanova@redhat.com about 1 month ago

  • Sprint/Milestone set to 1.0.0

Please register to edit this issue

Also available in: Atom PDF