Project

Profile

Help

Issue #8631

closed

schema conversion adds an empty blob in the manifest but it is not present on the fs

Added by ipanova@redhat.com almost 3 years ago. Updated almost 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 96
Quarter:

Description

This leads to pulp3 to pulp2 sync failure. In the converted manifest an empty blob with digest sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 is present and pulp2 trys to fetch it unsuccessfully because pulp3 does not have it on the fs

Error: DKR1020: Image download(s) from https://kangae.example.com/v2/test_organizationupgrade1-test_product-foremanbusybox/blobs/sha256:a3ed95caeb02ffe68cdd9
fd84406680ae93d633cb16422d00e8a7c22955b46d4 failed. Sync task has failed to prevent a corrupted repository.
DKR1020: Image download(s) from https://kangae.example.com/v2/test_organizationupgrade1-library-composite-test_product-foremanbusybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 failed. Sync task has failed to prevent a corrupted repository.
DKR1020: Image download(s) from https://kangae.example.com/v2/test_organizationupgrade1-library-component_1-test_product-foremanbusybox/blobs/sha256:a3ed95ca
eb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 failed. Sync task has failed to prevent a corrupted repository.


when i try to curl one of these, i get:

{"errors":[{"code":"BLOB_UNKNOWN","message":"Blob not found.","detail":{"digest":"sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4"}}]}

and on the pulp2 box: Apr 27 00:22:33  pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://kangae.example.com/v2/test_org
anizationupgrade1-library-composite-test_product-foremanbusybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 failed with code 404: Not Found

https://github.com/pulp/pulp_container/blob/master/pulp_container/app/schema_convert.py#L173

use this details to sync a repo

            "upstream_name": "foreman/busybox-test",                                                                                                                                                                                  
            "url": "https://quay.io/",  

Related issues

Copied to Container Support - Backport #8654: Backport to 2.1.: schema conversion adds an empty blob in the manifest but it is not present on the fsCLOSED - CURRENTRELEASEipanova@redhat.com

Actions
Copied to Container Support - Backport #8699: Backport to 2.5: schema conversion adds an empty blob in the manifest but it is not present on the fsCLOSED - CURRENTRELEASEmdellweg

Actions
Actions #1

Updated by ipanova@redhat.com almost 3 years ago

We need to do similar but in python https://github.com/distribution/distribution/blob/6affafd1f030087d88f88841bf66a8abe2bf4d24/manifest/schema1/config_builder.go#L22 https://github.com/distribution/distribution/blob/6affafd1f030087d88f88841bf66a8abe2bf4d24/manifest/schema1/config_builder_test.go#L58


In [14]: x=[31, 139, 8, 0, 0, 9, 110, 136, 0, 255, 98, 24, 5, 163, 96, 20, 140, 88,0, 8, 0, 0, 255, 255, 46, 175, 181, 239, 0, 4, 0, 0,]

In [15]: arr=bytearray(x)

In [16]: import hashlib

In [17]: hashlib.sha256(arr).hexdigest()
Out[17]: 'a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4'
Actions #2

Updated by ipanova@redhat.com almost 3 years ago

  • Copied to Backport #8654: Backport to 2.1.: schema conversion adds an empty blob in the manifest but it is not present on the fs added
Actions #3

Updated by ipanova@redhat.com almost 3 years ago

  • Sprint set to Sprint 95
Actions #4

Updated by ipanova@redhat.com almost 3 years ago

  • Sprint/Milestone set to 2.6.0
Actions #5

Updated by ipanova@redhat.com almost 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ipanova@redhat.com
Actions #6

Updated by rchan almost 3 years ago

  • Sprint changed from Sprint 95 to Sprint 96
Actions #7

Updated by pulpbot almost 3 years ago

  • Status changed from ASSIGNED to POST

Added by ipanova@redhat.com almost 3 years ago

Revision c16e10f7 | View on GitHub

Create and return empty blob on the fly.

closes #8631

Added by ipanova@redhat.com almost 3 years ago

Revision c16e10f7 | View on GitHub

Create and return empty blob on the fly.

closes #8631

Actions #8

Updated by ipanova@redhat.com almost 3 years ago

  • Status changed from POST to MODIFIED
Actions #9

Updated by ipanova@redhat.com almost 3 years ago

  • Copied to Backport #8699: Backport to 2.5: schema conversion adds an empty blob in the manifest but it is not present on the fs added
Actions #10

Updated by pulpbot almost 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF