Test #4128
closed
Test sync of a repository that is missing blobs
Added by amacdona@redhat.com over 5 years ago. Updated about 5 years ago.
Description
This test will require the creation of a new fixture for pulp_docker that is a corrupted repository. This fixture is primarily intended to test https://pulp.plan.io/issues/2849 (pulp 2) but could also be useful for testing pulp 3.
The "corrupted repository" can be a fully published non-corrupt repository that is simply missing one or more Images referenced by or or more of the Image Manifests.
Related issues
Updated by amacdona@redhat.com over 5 years ago
- Project changed from Pulp to Docker Support
Updated by amacdona@redhat.com over 5 years ago
- Related to Issue #2849: Worker terminates abnormally while processing task added
Updated by amacdona@redhat.com over 5 years ago
I was able to reproduce the problem using a pulp-pulp sync.
Updated by bherring over 5 years ago
Verified Releaes¶
Platform Version: 2.19a1
Status¶
Works as Expected
Verification Log/Snippet¶
By moving the source of the symlink blob on the master_node, a 404 sync issue was created with crane.
Pulp successfully prevents a corrupted repo, as noted by the well documented source RM #2849.
[root@kvm-04-guest08 ~]# pulp-admin -u admin -p admin docker repo sync run --repo-id fixture
+----------------------------------------------------------------------+
Synchronizing Repository [fixture]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading manifests
[/]
... completed
Copying units already in pulp
[-]
... completed
Copying units already in pulp
[-]
... completed
Downloading remote files
[===================== ] 42%
159 of 375 items
... failed
Task Failed
Image download(s) from
http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:5b0d5902
6729b68570d99bc4f3f7c31a2e4f2a5736435641565d93e7c25bd2c3 failed. Sync task has
failed to prevent a corrupted repository.
[root@kvm-04-guest08 ~]#
And handles the failure as expected:
-- Logs begin at Mon 2018-12-03 07:37:52 EST, end at Mon 2018-12-03 13:35:00 EST. --
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download succeeded: http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:a923c1afe177096fdfe14e91f7d6615866fadc7f2843c064c6d7774db6d4c447.
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download succeeded: http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:b14df7f8f09f465786a6d40cc1ce5b0f584da00fb5fdb837f11a1cd14e537632.
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download succeeded: http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:5b45a4076feb26fda45e37de5bcc047be974e4bb85978d7f3713068282f11456.
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download succeeded: http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:61a60a4d80d102088680814e82dda06552ed4e27e838c7733b75372e111b4a2e.
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download failed: Download of http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:5b0d59026729b68570d99bc4f3f7c31a2e4f2a5736435641565d93e7c25bd2c3 failed with code 404: Not Found
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download succeeded: http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:a1d02d3b71d76a6a5944396b02b745b8c53fc66f001ec649f3a506c353aae23a.
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download of http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:3f46e1095de266d1af2b069542c947ebe417063bedd12d781dcd39c9f6759ed3 was cancelled
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download of http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:f40a81bb9a0e674c5f790a76d53d594271dd46fc6c191581e3f79b32972ad700 was cancelled
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download of http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:521aaaf5e96e9be460cf2f45f6496113cfc6a5c1a0dc3e63f57c97367a4b133f was cancelled
Dec 03 13:05:16 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: pulp.server.async.tasks:INFO: [4dd06e49] Task failed : [4dd06e49-7314-4f6d-86a3-05e21901d3e2] : Image download(s) from http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:5b0d59026729b68570d99bc4f3f7c31a2e4f2a5736435641565d93e7c25bd2c3 failed. Sync task has failed to prevent a corrupted repository.
Dec 03 13:05:16 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: celery.app.trace:INFO: [4dd06e49] Task pulp.server.managers.repo.sync.sync[4dd06e49-7314-4f6d-86a3-05e21901d3e2] raised expected: PulpCodedException()
Dec 03 13:05:16 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: celery.app.trace:INFO: [a5f819b1] Task pulp.server.async.tasks._release_resource[a5f819b1-5f25-483d-ab82-820ac1efcebc] succeeded in 0.00434369600043s: None
Notes¶
I will be adding all notes to #4129 as both have relatively the same verification and use the same manual test harness.
In short, our current automated harness currently does not include the concept or ability to do multiple pulp instances running to be test against in a system.
This is an improvement to feature testing we can and should look into for Pulp3.
Updated by bherring over 5 years ago
- Status changed from ASSIGNED to CLOSED - COMPLETE
No automated test added to the repo.
Manual verification only.
Discussed with Austin.
Updated by bherring over 5 years ago
- Related to Test #4259: As a user I can sync from a docker registry added
.