Actions
Issue #8706
closed
Can not push docker images
Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:
Description
I installed centos/stream8 via vagrant and run pulp-installer (3.12.2) through ansible playbooks which worked pretty well last week. I could push and pull docker images via admin user. Today I run it again from a fresh centos/stream8. After I try to push the same docker images I get the error:
Error: error copying image to the remote destination: Error writing blob: Error uploading layer chunked: StatusCode: 400, <h1>Bad Request (400)</h1>
On pulp server side (/var/log/messages) I see the following errors:
May 6 12:10:53 localhost gunicorn[63707]: pulp [ff7ecdd9d83546caac1a113f6b8ce070]: django.request:WARNING: Unauthorized: /v2/
May 6 12:10:53 localhost gunicorn[63707]: pulp [ff7ecdd9d83546caac1a113f6b8ce070]: 127.0.0.1 - - [06/May/2021:12:10:53 +0000] "GET /v2/ HTTP/1.0" 401 106 "-" "libpod/2.2.1"
May 6 12:10:53 localhost gunicorn[63707]: pulp [1c18ad637af640afafbdca49108fb3fc]: pulpcore.deprecation:WARNING: BaseDistribution is deprecated and could be removed as early as pulpcore==3.13; use pulpcore.plugin.models.Distribution instead.
May 6 12:10:53 localhost gunicorn[63707]: pulp [1c18ad637af640afafbdca49108fb3fc]: 127.0.0.1 - admin [06/May/2021:12:10:53 +0000] "GET /token/?account=admin&scope=repository%3Acoredns-coredns%3Apull%2Cpush&service=pulp.test.com HTTP/1.0" 200 626 "https://pulp.test.com/token?account=admin&scope=repository%3Acoredns-coredns%3Apull%2Cpush&service=pulp.test.com" "libpod/2.2.1"
May 6 12:10:53 localhost gunicorn[63707]: pulp [40eb978fa7bc490cab34ae54a8765c72]: pulpcore.deprecation:WARNING: BaseDistribution is deprecated and could be removed as early as pulpcore==3.13; use pulpcore.plugin.models.Distribution instead.
May 6 12:10:53 localhost gunicorn[63707]: pulp [40eb978fa7bc490cab34ae54a8765c72]: 127.0.0.1 - admin [06/May/2021:12:10:53 +0000] "GET /token/?account=admin&scope=repository%3Acoredns-coredns%3Apull%2Cpush&service=pulp.test.com HTTP/1.0" 200 626 "https://pulp.test.com/token?account=admin&scope=repository%3Acoredns-coredns%3Apull%2Cpush&service=pulp.test.com" "libpod/2.2.1"
May 6 12:10:53 localhost gunicorn[63707]: pulp [ee303ad169a3406c82a077c9616f381b]: pulpcore.deprecation:WARNING: BaseDistribution is deprecated and could be removed as early as pulpcore==3.13; use pulpcore.plugin.models.Distribution instead.
May 6 12:10:53 localhost gunicorn[63707]: pulp [ee303ad169a3406c82a077c9616f381b]: django.request:WARNING: Not Found: /v2/coredns-coredns/blobs/sha256:225df95e717ceb672de0e45aa49f352eace21512240205972aca0fccc9612722
May 6 12:10:53 localhost gunicorn[63707]: pulp [ee303ad169a3406c82a077c9616f381b]: 127.0.0.1 - - [06/May/2021:12:10:53 +0000] "HEAD /v2/coredns-coredns/blobs/sha256:225df95e717ceb672de0e45aa49f352eace21512240205972aca0fccc9612722 HTTP/1.0" 404 158 "-" "libpod/2.2.1"
May 6 12:10:53 localhost gunicorn[63707]: pulp [15dfc37e12d04e3fa20fda53be4321ac]: pulpcore.deprecation:WARNING: BaseDistribution is deprecated and could be removed as early as pulpcore==3.13; use pulpcore.plugin.models.Distribution instead.
May 6 12:10:53 localhost gunicorn[63707]: pulp [15dfc37e12d04e3fa20fda53be4321ac]: django.request:WARNING: Not Found: /v2/coredns-coredns/blobs/sha256:85c53e1bd74e42c7decc4f7362eca51cd6cba15c227a75b3d90e8d01e8bb97e7
May 6 12:10:53 localhost gunicorn[63707]: pulp [15dfc37e12d04e3fa20fda53be4321ac]: 127.0.0.1 - - [06/May/2021:12:10:53 +0000] "HEAD /v2/coredns-coredns/blobs/sha256:85c53e1bd74e42c7decc4f7362eca51cd6cba15c227a75b3d90e8d01e8bb97e7 HTTP/1.0" 404 158 "-" "libpod/2.2.1"
May 6 12:10:53 localhost gunicorn[63707]: pulp [9fa28e323d6a4c6484a0217505c041c8]: pulpcore.deprecation:WARNING: BaseDistribution is deprecated and could be removed as early as pulpcore==3.13; use pulpcore.plugin.models.Distribution instead.
May 6 12:10:53 localhost gunicorn[63707]: pulp [9fa28e323d6a4c6484a0217505c041c8]: 127.0.0.1 - - [06/May/2021:12:10:53 +0000] "POST /v2/coredns-coredns/blobs/uploads/ HTTP/1.0" 202 0 "-" "libpod/2.2.1"
May 6 12:10:53 localhost gunicorn[63707]: pulp [9c8838fb915e48d98bd60204a3ff3cc9]: pulpcore.deprecation:WARNING: BaseDistribution is deprecated and could be removed as early as pulpcore==3.13; use pulpcore.plugin.models.Distribution instead.
May 6 12:10:53 localhost gunicorn[63707]: pulp [9c8838fb915e48d98bd60204a3ff3cc9]: 127.0.0.1 - - [06/May/2021:12:10:53 +0000] "POST /v2/coredns-coredns/blobs/uploads/ HTTP/1.0" 202 0 "-" "libpod/2.2.1"
May 6 12:10:53 localhost gunicorn[63707]: pulp [91fb0ab8f84549569cd87b53be57a8b5]: pulpcore.deprecation:WARNING: BaseDistribution is deprecated and could be removed as early as pulpcore==3.13; use pulpcore.plugin.models.Distribution instead.
May 6 12:10:53 localhost gunicorn[63707]: pulp [91fb0ab8f84549569cd87b53be57a8b5]: django.security.SuspiciousFileOperation:ERROR: Could not derive file name from ''
May 6 12:10:53 localhost gunicorn[63707]: pulp [91fb0ab8f84549569cd87b53be57a8b5]: django.request:WARNING: Bad Request: /v2/coredns-coredns/blobs/uploads/41a16969-8a47-475b-9847-5728bcbf4fe0
May 6 12:10:53 localhost gunicorn[63707]: pulp [91fb0ab8f84549569cd87b53be57a8b5]: 127.0.0.1 - - [06/May/2021:12:10:53 +0000] "PATCH /v2/coredns-coredns/blobs/uploads/41a16969-8a47-475b-9847-5728bcbf4fe0 HTTP/1.0" 400 26 "-" "libpod/2.2.1"
May 6 12:10:54 localhost gunicorn[63707]: pulp [4b0f79ce7df946bea6e2b97a6fa38a51]: pulpcore.deprecation:WARNING: BaseDistribution is deprecated and could be removed as early as pulpcore==3.13; use pulpcore.plugin.models.Distribution instead.
May 6 12:10:54 localhost gunicorn[63707]: pulp [4b0f79ce7df946bea6e2b97a6fa38a51]: django.security.SuspiciousFileOperation:ERROR: Could not derive file name from ''
May 6 12:10:54 localhost gunicorn[63707]: pulp [4b0f79ce7df946bea6e2b97a6fa38a51]: django.request:WARNING: Bad Request: /v2/coredns-coredns/blobs/uploads/34cc1ced-4ea4-4059-9636-f1229a72834a
May 6 12:10:54 localhost gunicorn[63707]: pulp [4b0f79ce7df946bea6e2b97a6fa38a51]: 127.0.0.1 - - [06/May/2021:12:10:54 +0000] "PATCH /v2/coredns-coredns/blobs/uploads/34cc1ced-4ea4-4059-9636-f1229a72834a HTTP/1.0" 400 26 "-" "libpod/2.2.1"
I logged in as admin user like I always did before without any issues. According to the documentation it should be possible to push as admin user and it also worked before. I really have no clue anymore what changed after the reinstallation.
Related issues
Updated by sli720 almost 3 years ago
- Copied from Issue #8705: Can not push docker images added
Updated by sli720 almost 3 years ago
ad. I used the following commands to push which worked one week ago but stopped working this week after reinstallation:
docker login pulp.test.com (with admin credentials)
docker pull rancher/coredns-coredns:1.8.3
docker tag rancher/coredns-coredns:1.8.3 pulp.test.com/coredns-coredns:1.8.3
docker push pulp.test.com/coredns-coredns:1.8.3
Could it be that the version of django or another package changed which is not compatible with pulp anymore? Is there a version binding in the ansible playbooks for thirdparty dependencies because I could not find it? Unfortunately I lost the old setup so I can not compare the versions.
Updated by sli720 almost 3 years ago
I found the package. It is Django. I downgraded it to 2.2.20. Everything above does not work.
Updated by ipanova@redhat.com almost 3 years ago
sli720 wrote:
I found the package. It is Django. I downgraded it to 2.2.20. Everything above does not work.
Hi thank you for the report, we are aware of the incompatibility issues with the Django 2.2.21 and actively working on solving that https://pulp.plan.io/issues/8691 . For now please downgrade to 2.2.20
Updated by ipanova@redhat.com almost 3 years ago
- Status changed from NEW to CLOSED - CURRENTRELEASE
Actions
.