Fails to pull images from AWS ecr to pulp with unknown blob error
We are trying to sync Amazon ECR container registry images to pulp
Followed pulp workflow:
- created the repo
- created remote --> which points to ECR registry, pull policy is "on_demand"
- Synced remote to repo
- created distribution.
All the API calls were successful.
- when docker pulls API is executed we are getting an unknown blob error.
# docker pull <pulp_server>/quay.io/maprtech/csi-nfsplugin:126.96.36.199 188.8.131.52: Pulling from quay.io/maprtech/csi-nfsplugin d9e72d058dc5: Pulling fs layer cca21acb641a: Pulling fs layer 8f3bec2e0ffe: Downloading d4889023e8ff: Waiting 6fce7d7675ec: Waiting 342937d06bfb: Waiting fb8dcb3732a4: Waiting 2bc41d47dd50: Waiting c2a40f5494a9: Waiting unknown blob*
When the repo is deleted, partially downloaded blobs are not deleted/ cleaned up. And when we recreate the repo, remote, and resync, it points to the same corrupted or partially downloaded blobs. This results in an unknown blob error when Docker pull command is executed.
Amazon ECR token required to pull the images will expire in 11 hrs, Is there any efficient way to rotate the token in pulp? We used PUT remote API call to update the AWS ECR token and did a resync to the repo, the issue remains the same.
Let me know if you need more information.