Issue #5253: Docker distributions fail when CONTENT_HOST is not set - Container Support - Pulp

Actions

Send by e-mail Copy link

Issue #5253

closed

Docker distributions fail when CONTENT_HOST is not set

Added by amacdona@redhat.com over 5 years ago. Updated almost 5 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Normal

Assignee:

lmjachky

Sprint/Milestone:

1.0.0

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Platform Release:

OS:

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Pulp 3 docker blocker

Sprint:

Sprint 61

Quarter:

Description

In a default pulplift environment, content_host is not included in the pulp_settings. In this environment, retrieving the task with a created_resource of a distribution fails.

This is covered by the functional tests, which fail out of the box on pulplift.

pulp_docker/tests/functional/api/test_crud_distributions.py::CRUDDockerDistributionsTestCase::test_01_create_distribution FAILED

========================================================================================== FAILURES ==========================================================================================
________________________________________________________________ CRUDDockerDistributionsTestCase.test_01_create_distribution _________________________________________________________________

self = <pulp_docker.tests.functional.api.test_crud_distributions.CRUDDockerDistributionsTestCase testMethod=test_01_create_distribution>

    def test_01_create_distribution(self):
        """Create a distribution."""
        body = gen_distribution()
        response_dict = self.client.post(
>           DOCKER_DISTRIBUTION_PATH, body
        )

E           requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http://localhost:24817/pulp/api/v3/tasks/e433b5ff-635b-45d3-a173-d61c3c177ac5/

Logged Traceback:

Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]: pulp: django.request:ERROR: Internal Server Error: /pulp/api/v3/tasks/e433b5ff-635b-45d3-a173-d61c3c177ac5/
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]: Traceback (most recent call last):
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = get_response(request)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = self.process_exception_by_middleware(e, request)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = wrapped_callback(request, *callback_args, **callback_kwargs)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     return view_func(*args, **kwargs)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/viewsets.py", line 114, in view
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     return self.dispatch(request, *args, **kwargs)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 497, in dispatch
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = self.handle_exception(exc)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 457, in handle_exception
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     self.raise_uncaught_exception(exc)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 468, in raise_uncaught_exception
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     raise exc
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 494, in dispatch
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = handler(request, *args, **kwargs)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/mixins.py", line 56, in retrieve
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     return Response(serializer.data)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 559, in data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     ret = super().data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 261, in data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     self._data = self.to_representation(self.instance)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 526, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     ret[field.field_name] = field.to_representation(attribute)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/relations.py", line 535, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     for value in iterable
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/relations.py", line 535, in <listcomp>
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     for value in iterable
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/home/vagrant/devel/pulpcore/pulpcore/app/serializers/task.py", line 31, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     return serializer.data.get('_href')
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 559, in data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     ret = super().data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 261, in data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     self._data = self.to_representation(self.instance)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 526, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     ret[field.field_name] = field.to_representation(attribute)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/home/vagrant/devel/pulp_docker/pulp_docker/app/serializers.py", line 111, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     host = self.context['request'].get_host()
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]: AttributeError: 'NoneType' object has no attribute 'get_host'
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]: 127.0.0.1 - admin [12/Aug/2019:19:44:43 +0000] "GET /pulp/api/v3/tasks/e433b5ff-635b-45d3-a173-d61c3c177ac5/ HTTP/1.1" 500 27 "-" "python-requests/2.22.0"

Workaround¶

Append `CONTENT_HOST=localhost:24816` to `/etc/pulp/settings.py`, and restart the services. Tests pass.

Actions

Copy link

Updated by dkliban@redhat.com over 5 years ago

Triaged changed from No to Yes
Sprint set to Sprint 57

Actions

Copy link

Updated by rchan about 5 years ago

Sprint changed from Sprint 57 to Sprint 58

Actions

Copy link

Updated by ipanova@redhat.com about 5 years ago

we are not passing anything into the context that's why it is None https://github.com/pulp/pulpcore/blob/master/pulpcore/app/viewsets/base.py#L336

however this would not solve the problem because request.get_host() would give us Out[1]: 'localhost:24817' instead of localhost:24816

Actions

Copy link

Updated by lmjachky about 5 years ago

It looks to me that the only usage of CONTENT_HOST is in the serializer which serializes the field "registry_path". This field is used for distributions. As @ipanova highlighted out, the method get_host() returns "localhost:24817" instead of "localhost:24816" by default. Therefore, the else statement in pulp_docker https://github.com/pulp/pulp_docker/blob/752698a4251072a2dbfbe4df4793fd3ea4d99fde/pulp_docker/app/serializers.py#L112-L113 is redundant. The workaround is to append the following setting to the file "pulplift/example.dev-config.yml":

pulp_settings:
  content_host: "localhost:24816"

If we allow users to specify CONTENT_HOST (https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html?highlight=content_host#content-host), I think that we should also check if a content server runs upon the same bind https://github.com/pulp/ansible-pulp/blob/95159929a74aa64536a0c76a0d999aa73f5b0b84/roles/pulp-content/templates/pulp-content-app.service.j2#L16. Would not this cause inconsistencies in the future? Like a different port number for registry_path (localhost:24816/foo) and a different port number for the handler (localhost:54321/foo).

Anyway, what are the differences between pulp_content_bind (pulplift/ansible-pulp), pulp_content_host (pulp_docker/.travis), and content_host ([pulpcore|pulp_docker]/.travis)? These values are used almost across all installers. And all of these variables store the same value. Is it possible to use one variable for that? Or at least, the existing variables should be documented in a more transparent way.

Actions

Copy link