Issue #9599
Updated by lmjachky over 2 years ago
The error is triggered only after removing the reference to a repository within a distribution. Steps to reproduce the error:
1. Disable token authentication and sync a remote repository.
```
cat dockersync.bash
http POST http://localhost:24817/pulp/api/v3/repositories/container/container/ name=foo
export REPO_HREF=$(http :24817/pulp/api/v3/repositories/container/container/ | jq -r '.results[] | select(.name == "foo") | .pulp_href')
http POST http://localhost:24817/pulp/api/v3/remotes/container/container/ name='lubosmj/mirror' upstream_name='lubosmj/mirror' url='https://registry.hub.docker.com' policy='immediate' password="hesielkoNaDocker1337" username="lubosmj"
export REMOTE_HREF=$(http :24817/pulp/api/v3/remotes/container/container/ | jq -r '.results[] | select(.name == "lubosmj/mirror") | .pulp_href')
pulp container repository sync --name foo --remote lubosmj/mirror
pulp container distribution create --name name --base-path base-path --repository foo
```
```
bash dockersync.bash
HTTP/1.1 201 Created
Access-Control-Expose-Headers: Correlation-ID
Allow: GET, POST, HEAD, OPTIONS
Connection: close
Content-Length: 465
Content-Type: application/json
Correlation-ID: 01981a6e63f74f99bec6910b2a5b9d86
Date: Thu, 02 Dec 2021 14:29:40 GMT
Location: /pulp/api/v3/repositories/container/container/17dc4c9e-53e1-4ae5-82bc-c4d0632b9600/
Referrer-Policy: same-origin
Server: gunicorn
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
{
"description": null,
"latest_version_href": "/pulp/api/v3/repositories/container/container/17dc4c9e-53e1-4ae5-82bc-c4d0632b9600/versions/0/",
"name": "foo",
"pulp_created": "2021-12-02T14:29:39.928443Z",
"pulp_href": "/pulp/api/v3/repositories/container/container/17dc4c9e-53e1-4ae5-82bc-c4d0632b9600/",
"pulp_labels": {},
"remote": null,
"retain_repo_versions": null,
"versions_href": "/pulp/api/v3/repositories/container/container/17dc4c9e-53e1-4ae5-82bc-c4d0632b9600/versions/"
}
HTTP/1.1 201 Created
Access-Control-Expose-Headers: Correlation-ID
Allow: GET, POST, HEAD, OPTIONS
Connection: close
Content-Length: 614
Content-Type: application/json
Correlation-ID: 398410f0419246099448201306bb0842
Date: Thu, 02 Dec 2021 14:29:41 GMT
Location: /pulp/api/v3/remotes/container/container/9fc79087-758a-47c6-9832-04fd25191a62/
Referrer-Policy: same-origin
Server: gunicorn
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
{
"ca_cert": null,
"client_cert": null,
"connect_timeout": null,
"download_concurrency": null,
"exclude_tags": null,
"headers": null,
"include_tags": null,
"max_retries": null,
"name": "lubosmj/mirror",
"policy": "immediate",
"proxy_url": null,
"pulp_created": "2021-12-02T14:29:41.645134Z",
"pulp_href": "/pulp/api/v3/remotes/container/container/9fc79087-758a-47c6-9832-04fd25191a62/",
"pulp_labels": {},
"pulp_last_updated": "2021-12-02T14:29:41.645148Z",
"rate_limit": null,
"sock_connect_timeout": null,
"sock_read_timeout": null,
"tls_validation": true,
"total_timeout": null,
"upstream_name": "lubosmj/mirror",
"url": "https://registry.hub.docker.com"
}
Started background task /pulp/api/v3/tasks/7f9f6d2a-a51d-48b5-a919-61d8674f7d45/
...Done.
Started background task /pulp/api/v3/tasks/df191dd2-0a5f-4cd9-a5d8-0d2ba4c8a038/
.Done.
{
"pulp_href": "/pulp/api/v3/distributions/container/container/34cfda8e-b5b8-4aa3-a656-ac5adba3dd76/",
"repository": "/pulp/api/v3/repositories/container/container/17dc4c9e-53e1-4ae5-82bc-c4d0632b9600/",
"content_guard": "/pulp/api/v3/contentguards/container/content_redirect/f6615734-fa37-463e-942b-05310b246ae2/",
"base_path": "base-path",
"name": "name",
"pulp_labels": {},
"pulp_created": "2021-12-02T14:29:48.358690Z",
"repository_version": null,
"registry_path": "pulp3-source-fedora34.localhost.example.com/base-path",
"namespace": "/pulp/api/v3/pulp_container/namespaces/3cbed472-7734-43b7-8a31-6fbad955d2da/",
"private": false,
"description": null
}
```
2. Fetch one of the existing blobs.
```
http :24817/pulp/api/v3/content/container/blobs/?limit=1
HTTP/1.1 200 OK
Access-Control-Expose-Headers: Correlation-ID
Allow: GET, HEAD, OPTIONS
Connection: close
Content-Length: 480
Content-Type: application/json
Correlation-ID: 23eedc8c421d4f858e99d8b1bc7cd025
Date: Thu, 02 Dec 2021 14:30:53 GMT
Referrer-Policy: same-origin
Server: gunicorn
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
{
"count": 6,
"next": "http://localhost:24817/pulp/api/v3/content/container/blobs/?limit=1&offset=1",
"previous": null,
"results": [
{
"artifact": "/pulp/api/v3/artifacts/36d71c4f-f0ca-4a25-9bc8-2d7818d21281/",
"digest": "sha256:f7776189e4fa6a19c57087c0d1bbd413dbf6bc4c25547b2be2377a3b373b6676",
"media_type": "application/vnd.docker.container.image.v1+json",
"pulp_created": "2021-12-02T14:29:46.539410Z",
"pulp_href": "/pulp/api/v3/content/container/blobs/2bc41c2a-eca4-44c9-aadd-9ffac91756e2/"
}
]
}
http :24817/v2/base-path/blobs/sha256:f7776189e4fa6a19c57087c0d1bbd413dbf6bc4c25547b2be2377a3b373b6676 'Accept: application/vnd.docker.container.image.v1+json'
HTTP/1.1 302 Found
Access-Control-Expose-Headers: Correlation-ID
Allow: GET, HEAD, OPTIONS
Connection: close
Content-Length: 0
Content-Type: text/html; charset=utf-8
Correlation-ID: 461309c2d3474cdf886d0d23d19bc29a
Date: Thu, 02 Dec 2021 14:32:00 GMT
Docker-Distribution-Api-Version: registry/2.0
Location: http://pulp3-source-fedora34.localhost.example.com/pulp/container/base-path/blobs/sha256:f7776189e4fa6a19c57087c0d1bbd413dbf6bc4c25547b2be2377a3b373b6676?validate_token=d1ad8c80f9bbb71c2e682b658ec9097c9fd967a7f168c27ef998d1c43f3027a4:948980f81a848f4308098e51acbfb346d01bb7c71ee3ba1b43e7e07d9789afdf
Referrer-Policy: same-origin
Server: gunicorn
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
http http://pulp3-source-fedora34.localhost.example.com/pulp/container/base-path/blobs/sha256:f7776189e4fa6a19c57087c0d1bbd413dbf6bc4c25547b2be2377a3b373b6676?validate_token=d1ad8c80f9bbb71c2e682b658ec9097c9fd967a7f168c27ef998d1c43f3027a4:948980f81a848f4308098e51acbfb346d01bb7c71ee3ba1b43e7e07d9789afdf 'Accept: application/vnd.docker.container.image.v1+json'
HTTP/1.1 200 OK
Accept-Ranges: bytes
Connection: keep-alive
Content-Disposition: attachment; filename=776189e4fa6a19c57087c0d1bbd413dbf6bc4c25547b2be2377a3b373b6676
Content-Length: 1820
Content-Type: application/octet-stream
Date: Thu, 02 Dec 2021 14:32:35 GMT
Docker-Content-Digest: sha256:f7776189e4fa6a19c57087c0d1bbd413dbf6bc4c25547b2be2377a3b373b6676
Docker-Distribution-API-Version: registry/2.0
Etag: "16bcf67e171d7042-71c"
Last-Modified: Thu, 02 Dec 2021 14:29:46 GMT
Server: nginx/1.20.1
{"architecture":"amd64","config":{"Hostname":"d0e5d053b661","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Cmd":null,"ArgsEscaped":true,"Image":"sha256:e004b88433f9737b37e2d66721effa20ff1263082c8152882a1976c72a494a34","Volumes":null,"WorkingDir":"","Entrypoint":["mysql"],"OnBuild":[],"Labels":{}},"container":"82bea353549a807a6bbcd4d9dfcd490c194691bf30c99b75abed0693c9d27072","container_config":{"Hostname":"d0e5d053b661","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Cmd":["/bin/sh","-c","#(nop) ","ENTRYPOINT [\"mysql\"]"],"ArgsEscaped":true,"Image":"sha256:e004b88433f9737b37e2d66721effa20ff1263082c8152882a1976c72a494a34","Volumes":null,"WorkingDir":"","Entrypoint":["mysql"],"OnBuild":[],"Labels":{}},"created":"2020-01-27T14:35:45.362414698Z","docker_version":"1.13.1","history":[{"created":"2019-03-07T22:19:53.313789681Z","created_by":"/bin/sh -c #(nop) ADD file:aa17928040e31624cad9c7ed19ac277c5402c4b9ba39f834250affca40c4046e in / "},{"created":"2019-03-07T22:19:53.447205048Z","created_by":"/bin/sh -c #(nop) CMD [\"/bin/sh\"]","empty_layer":true},{"created":"2020-01-27T14:35:45.032879266Z","created_by":"/bin/sh -c apk add --no-cache mysql-client"},{"created":"2020-01-27T14:35:45.362414698Z","created_by":"/bin/sh -c #(nop) ENTRYPOINT [\"mysql\"]","empty_layer":true}],"os":"linux","rootfs":{"type":"layers","diff_ids":["sha256:3fc64803ca2de7279269048fe2b8b3c73d4536448c87c32375b2639ac168a48b","sha256:f8f387438ce6199d3c8dcd4e4daaaebea2eacb67c807090451805aa550fcc855"]}}
```
3. Remove the repository from the distribution.
```
http PATCH :24817/pulp/api/v3/distributions/container/container/34cfda8e-b5b8-4aa3-a656-ac5adba3dd76/ repository=""
HTTP/1.1 202 Accepted
Access-Control-Expose-Headers: Correlation-ID
Allow: GET, PUT, PATCH, DELETE, HEAD, OPTIONS
Connection: close
Content-Length: 67
Content-Type: application/json
Correlation-ID: fe0607bc5c0143ba8935b38dc52e5410
Date: Thu, 02 Dec 2021 14:33:00 GMT
Referrer-Policy: same-origin
Server: gunicorn
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
{
"task": "/pulp/api/v3/tasks/905ae1c1-ac14-42af-8dcb-7350323fa5e5/"
}
```
4. Try to fetch the same blob once again.
```
http http://pulp3-source-fedora34.localhost.example.com/pulp/container/base-path/blobs/sha256:f7776189e4fa6a19c57087c0d1bbd413dbf6bc4c25547b2be2377a3b373b6676?validate_token=d1ad8c80f9bbb71c2e682b658ec9097c9fd967a7f168c27ef998d1c43f3027a4:948980f81a848f4308098e51acbfb346d01bb7c71ee3ba1b43e7e07d9789afdf 'Accept: application/vnd.docker.container.image.v1+json'
HTTP/1.1 500 Internal Server Error
Connection: keep-alive
Content-Length: 55
Content-Type: text/plain; charset=utf-8
Date: Thu, 02 Dec 2021 14:33:16 GMT
Server: nginx/1.20.1
500 Internal Server Error
Server got itself in trouble
```
Traceback:
```
File "/usr/local/lib/pulp/lib64/python3.9/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
resp = await request_handler(request)
File "/usr/local/lib/pulp/lib64/python3.9/site-packages/aiohttp/web_app.py", line 504, in _handle
resp = await handler(request)
File "/usr/local/lib/pulp/lib64/python3.9/site-packages/aiohttp/web_middlewares.py", line 117, in impl
return await handler(request)
File "/home/vagrant/devel/pulpcore/pulpcore/content/authentication.py", line 41, in authenticate
return await handler(request)
File "/home/vagrant/devel/pulp_container/pulp_container/app/registry.py", line 220, in get_by_digest
content__in=await sync_to_async(repository_version.get_content)(),
AttributeError: 'NoneType' object has no attribute 'get_content'
```
.