Project

Profile

Help

Test #4128

Test sync of a repository that is missing blobs

Added by amacdona@redhat.com about 1 year ago. Updated 8 months ago.

Status:
CLOSED - COMPLETE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Platform Release:
Tags:
Pulp 2
Sprint:

Description

This test will require the creation of a new fixture for pulp_docker that is a corrupted repository. This fixture is primarily intended to test https://pulp.plan.io/issues/2849 (pulp 2) but could also be useful for testing pulp 3.

The "corrupted repository" can be a fully published non-corrupt repository that is simply missing one or more Images referenced by or or more of the Image Manifests.


Related issues

Related to Docker Support - Issue #2849: Worker terminates abnormally while processing task CLOSED - CURRENTRELEASE Actions
Related to Pulp - Test #4259: As a user I can sync from a docker registry CLOSED - COMPLETE Actions

History

#1 Updated by amacdona@redhat.com about 1 year ago

  • Project changed from Pulp to Docker Support

#2 Updated by amacdona@redhat.com about 1 year ago

  • Related to Issue #2849: Worker terminates abnormally while processing task added

#3 Updated by amacdona@redhat.com about 1 year ago

I was able to reproduce the problem using a pulp-pulp sync.

https://pulp.plan.io/issues/2849#note-14

#4 Updated by bherring about 1 year ago

  • Status changed from NEW to ASSIGNED

#5 Updated by bherring about 1 year ago

  • Assignee set to bherring

#6 Updated by bherring about 1 year ago

Verified Releaes

Platform Version: 2.19a1

Status

Works as Expected

Verification Log/Snippet

By moving the source of the symlink blob on the master_node, a 404 sync issue was created with crane.

Pulp successfully prevents a corrupted repo, as noted by the well documented source RM #2849.

[root@kvm-04-guest08 ~]# pulp-admin -u admin -p admin docker repo sync run --repo-id fixture
+----------------------------------------------------------------------+
                   Synchronizing Repository [fixture]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading manifests
[/]
... completed

Copying units already in pulp
[-]
... completed

Copying units already in pulp
[-]
... completed

Downloading remote files
[=====================                             ] 42%
159 of 375 items
... failed

Task Failed

Image download(s) from
http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:5b0d5902
6729b68570d99bc4f3f7c31a2e4f2a5736435641565d93e7c25bd2c3 failed. Sync task has
failed to prevent a corrupted repository.

[root@kvm-04-guest08 ~]# 

And handles the failure as expected:

-- Logs begin at Mon 2018-12-03 07:37:52 EST, end at Mon 2018-12-03 13:35:00 EST. --
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download succeeded: http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:a923c1afe177096fdfe14e91f7d6615866fadc7f2843c064c6d7774db6d4c447.
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download succeeded: http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:b14df7f8f09f465786a6d40cc1ce5b0f584da00fb5fdb837f11a1cd14e537632.
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download succeeded: http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:5b45a4076feb26fda45e37de5bcc047be974e4bb85978d7f3713068282f11456.
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download succeeded: http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:61a60a4d80d102088680814e82dda06552ed4e27e838c7733b75372e111b4a2e.
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download failed: Download of http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:5b0d59026729b68570d99bc4f3f7c31a2e4f2a5736435641565d93e7c25bd2c3 failed with code 404: Not Found
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download succeeded: http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:a1d02d3b71d76a6a5944396b02b745b8c53fc66f001ec649f3a506c353aae23a.
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download of http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:3f46e1095de266d1af2b069542c947ebe417063bedd12d781dcd39c9f6759ed3 was cancelled
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download of http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:f40a81bb9a0e674c5f790a76d53d594271dd46fc6c191581e3f79b32972ad700 was cancelled
Dec 03 13:05:15 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: nectar.downloaders.threaded:INFO: Download of http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:521aaaf5e96e9be460cf2f45f6496113cfc6a5c1a0dc3e63f57c97367a4b133f was cancelled
Dec 03 13:05:16 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: pulp.server.async.tasks:INFO: [4dd06e49] Task failed : [4dd06e49-7314-4f6d-86a3-05e21901d3e2] : Image download(s) from http://kvm-04-guest09.rhts.eng.bos.redhat.com:5000/v2/test/blobs/sha256:5b0d59026729b68570d99bc4f3f7c31a2e4f2a5736435641565d93e7c25bd2c3 failed. Sync task has failed to prevent a corrupted repository.
Dec 03 13:05:16 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: celery.app.trace:INFO: [4dd06e49] Task pulp.server.managers.repo.sync.sync[4dd06e49-7314-4f6d-86a3-05e21901d3e2] raised expected: PulpCodedException()
Dec 03 13:05:16 kvm-04-guest08.rhts.eng.bos.redhat.com pulp[18366]: celery.app.trace:INFO: [a5f819b1] Task pulp.server.async.tasks._release_resource[a5f819b1-5f25-483d-ab82-820ac1efcebc] succeeded in 0.00434369600043s: None

Notes

I will be adding all notes to #4129 as both have relatively the same verification and use the same manual test harness.

In short, our current automated harness currently does not include the concept or ability to do multiple pulp instances running to be test against in a system.

This is an improvement to feature testing we can and should look into for Pulp3.

#7 Updated by bherring about 1 year ago

  • Status changed from ASSIGNED to CLOSED - COMPLETE

No automated test added to the repo.

Manual verification only.

Discussed with Austin.

#8 Updated by bherring 12 months ago

  • Related to Test #4259: As a user I can sync from a docker registry added

#9 Updated by bmbouter 8 months ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF