Issue #4583
closedUnable to docker/podman pull from a docker-distribution
Description
Failure shows up in tests:
_____________________ PullContentTestCase.test_pull_image ______________________
self = <pulp_docker.tests.functional.api.test_pull_content.PullContentTestCase testMethod=test_pull_image>
def test_pull_image(self):
"""Verify that a client can pull the image from Pulp.
1. Using the RegistryClient pull the image from Pulp.
2. Pull the same image from remote registry.
3. Verify both images has the same checksum.
4. Ensure image is deleted after the test.
"""
registry = cli.RegistryClient(self.cfg)
registry.raise_if_unsupported(
unittest.SkipTest, 'Test requires podman/docker'
)
local_url = urljoin(
self.cfg.get_content_host_base_url(),
self.distribution['base_path']
)
> registry.pull(local_url)
pulp_docker/tests/functional/api/test_pull_content.py:151:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/pulp_smash/cli.py:965: in _dispatch_command
result = self._client.run(cmd, sudo=True)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/pulp_smash/cli.py:257: in run
return self.response_handler(completed_process)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/pulp_smash/cli.py:54: in code_handler
completed_proc.check_returncode()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = CompletedProcess(args=('sudo', 'podman', 'pull', 'localhost:8080/1d95b8a8-f9f6-43c2-bed2-48cb47023154'), returncode=12...b47023154": Invalid image name "localhost:8080/1d95b8a8-f9f6-43c2-bed2-48cb47023154", unknown transport "localhost"\n')
def check_returncode(self):
"""Raise an exception if ``returncode`` is non-zero.
Raise :class:`pulp_smash.exceptions.CalledProcessError` if
``returncode`` is non-zero.
Why not raise ``subprocess.CalledProcessError``? Because stdout and
stderr are not included when str() is called on a CalledProcessError
object. A typical message is::
"Command '('ls', 'foo')' returned non-zero exit status 2"
This information is valuable. One could still make
``subprocess.CalledProcessError`` work by overloading ``args``:
>>> if isinstance(args, (str, bytes)):
... custom_args = (args, stdout, stderr)
... else:
... custom_args = tuple(args) + (stdout, stderr)
>>> subprocess.CalledProcessError(args, returncode)
But this seems like a hack.
In addition, it's generally good for an application to raise expected
exceptions from its own namespace, so as to better abstract away
dependencies.
"""
if self.returncode != 0:
raise exceptions.CalledProcessError(
> self.args, self.returncode, self.stdout, self.stderr
)
E pulp_smash.exceptions.CalledProcessError: Command ('sudo', 'podman', 'pull', 'localhost:8080/1d95b8a8-f9f6-43c2-bed2-48cb47023154') returned non-zero exit status 125.
E
E stdout:
E
E stderr: Trying to pull docker://localhost:8080/1d95b8a8-f9f6-43c2-bed2-48cb47023154...time="2019-03-26T17:32:04Z" level=error msg="Error pulling image ref //localhost:8080/1d95b8a8-f9f6-43c2-bed2-48cb47023154:latest: Error determining manifest MIME type for docker://localhost:8080/1d95b8a8-f9f6-43c2-bed2-48cb47023154:latest: Error reading manifest latest in localhost:8080/1d95b8a8-f9f6-43c2-bed2-48cb47023154: received unexpected HTTP status: 500 Internal Server Error"
E Failed
E (0x178cce0,0xc420144720)
E Error: error pulling image "localhost:8080/1d95b8a8-f9f6-43c2-bed2-48cb47023154": Invalid image name "localhost:8080/1d95b8a8-f9f6-43c2-bed2-48cb47023154", unknown transport "localhost"
I was able to reproduce locally:
podman pull localhost:8080/foo
Trying to pull docker://localhost:8080/foo...Failed
(0x179ec00,0xc420270320)
Error: error pulling image "localhost:8080/foo": Invalid image name "localhost:8080/foo", unknown transport "localhost"
Docker pull doesnt work either:
docker pull localhost:8080/foo
Using default tag: latest
Error response from daemon: received unexpected HTTP status: 500 Internal Server Error
FYI:
cat /etc/docker/daemon.json
{
"insecure-registries" : ["localhost:8080"]
}
cat /etc/containers/registries.conf
[registries.search]
registries = ["docker.io", "registry.fedoraproject.org", "quay.io", "registry.access.redhat.com", "registry.centos.org"]
[registries.insecure]
registries = ["localhost:8080"]
[registries.block]
registries = []
http http://localhost:8000/pulp/api/v3/docker-distributions/
HTTP/1.1 200 OK
Allow: GET, POST, HEAD, OPTIONS
Connection: close
Content-Length: 394
Content-Type: application/json
Date: Tue, 26 Mar 2019 19:31:47 GMT
Server: gunicorn/19.9.0
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"count": 1,
"next": null,
"previous": null,
"results": [
{
"_created": "2019-03-26T19:13:58.403236Z",
"_href": "/pulp/api/v3/docker-distributions/e880d0f3-891b-4a90-a235-15c50df6370e/",
"base_path": "foo",
"content_guard": null,
"name": "baz",
"publication": "/pulp/api/v3/publications/6663b5f9-d8c5-4cc4-88e6-cab081e1d953/",
"publisher": null,
"registry_path": "localhost:8080/foo",
"remote": null,
"repository": null
}
]
}
Updated by amacdona@redhat.com over 5 years ago
Traceback in the log:
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: Traceback (most recent call last):
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/web_protocol.py", line 418, in start
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: resp = await task
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/web_app.py", line 458, in _handle
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: resp = await handler(request)
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: File "/home/vagrant/devel/pulp_docker/pulp_docker/app/registry.py", line 163, in get_tag
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: return await Registry.dispatch_tag(tag, response_headers)
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: File "/home/vagrant/devel/pulp_docker/pulp_docker/app/registry.py", line 202, in dispatch_tag
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: return await Registry._dispatch(artifact.file.name, response_headers)
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: File "/home/vagrant/devel/pulp_docker/pulp_docker/app/registry.py", line 101, in _dispatch
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: full_headers['Content-Length'] = os.path.getsize(path)
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: File "/usr/lib64/python3.6/genericpath.py", line 50, in getsize
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: return os.stat(filename).st_size
Mar 26 19:45:05 pulp3-source-fedora28.localhost.example.com gunicorn[978]: FileNotFoundError: [Errno 2] No such file or directory: 'artifact/06/1ca9704a714ee3e8b80523ec720c64f6209ad3f97c0ff7cb9ec7d19f15149f'
Updated by dkliban@redhat.com over 5 years ago
I am seeing the following traceback in gunicorn when I perform a docker pull:
[2019-03-26 19:41:57 +0000] [7369] [ERROR] Error handling request
Traceback (most recent call last):
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/web_protocol.py", line 275, in data_received
messages, upgraded, tail = self._request_parser.feed_data(data)
File "aiohttp/_http_parser.pyx", line 523, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: invalid HTTP method
[2019-03-26 19:41:57 +0000] [7369] [ERROR] Error handling request
Traceback (most recent call last):
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/web_protocol.py", line 275, in data_received
messages, upgraded, tail = self._request_parser.feed_data(data)
File "aiohttp/_http_parser.pyx", line 523, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: invalid HTTP method
[2019-03-26 19:41:57 +0000] [7369] [ERROR] Error handling request
Traceback (most recent call last):
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/web_protocol.py", line 418, in start
resp = await task
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/web_app.py", line 458, in _handle
resp = await handler(request)
File "/home/vagrant/devel/pulp_docker/pulp_docker/app/registry.py", line 163, in get_tag
return await Registry.dispatch_tag(tag, response_headers)
File "/home/vagrant/devel/pulp_docker/pulp_docker/app/registry.py", line 202, in dispatch_tag
return await Registry._dispatch(artifact.file.name, response_headers)
File "/home/vagrant/devel/pulp_docker/pulp_docker/app/registry.py", line 101, in _dispatch
full_headers['Content-Length'] = os.path.getsize(path)
File "/usr/lib64/python3.6/genericpath.py", line 50, in getsize
return os.stat(filename).st_size
FileNotFoundError: [Errno 2] No such file or directory: 'artifact/06/1ca9704a714ee3e8b80523ec720c64f6209ad3f97c0ff7cb9ec7d19f15149f'
[2019-03-26 19:41:57 +0000] [7370] [ERROR] Error handling request
Traceback (most recent call last):
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/web_protocol.py", line 275, in data_received
messages, upgraded, tail = self._request_parser.feed_data(data)
File "aiohttp/_http_parser.pyx", line 523, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: invalid HTTP method
Added by dkliban@redhat.com over 5 years ago
Added by dkliban@redhat.com over 5 years ago
Revision 3035c9a9 | View on GitHub
Problem: registry is using relative path to serve artifact
Solution: prepend the MEDIA_ROOT to the path before trying to send the artifcat
Added by dkliban@redhat.com over 5 years ago
Revision 3035c9a9 | View on GitHub
Problem: registry is using relative path to serve artifact
Solution: prepend the MEDIA_ROOT to the path before trying to send the artifcat
Added by dkliban@redhat.com over 5 years ago
Revision 3035c9a9 | View on GitHub
Problem: registry is using relative path to serve artifact
Solution: prepend the MEDIA_ROOT to the path before trying to send the artifcat
Updated by dkliban@redhat.com over 5 years ago
- Status changed from NEW to MODIFIED
Applied in changeset commit:pulp_docker|3035c9a931043ec0c165fed50b0674bfb7ef7d9d.
Updated by ipanova@redhat.com over 5 years ago
- Assignee set to dkliban@redhat.com
- Sprint set to Sprint 50
Updated by dkliban@redhat.com over 5 years ago
Applied in changeset 3035c9a931043ec0c165fed50b0674bfb7ef7d9d.
Updated by ipanova@redhat.com about 5 years ago
- Project changed from Docker Support to Container Support
Updated by ipanova@redhat.com almost 5 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Problem: registry is using relative path to serve artifact
Solution: prepend the MEDIA_ROOT to the path before trying to send the artifcat
fixes: #4583 https://pulp.plan.io/issues/4583