Project

Profile

Help

Issue #5253

Docker distributions fail when CONTENT_HOST is not set

Added by amacdona@redhat.com about 1 year ago. Updated 9 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 3 docker blocker
Sprint:
Sprint 61
Quarter:

Description

In a default pulplift environment, content_host is not included in the pulp_settings. In this environment, retrieving the task with a created_resource of a distribution fails.

This is covered by the functional tests, which fail out of the box on pulplift.

pulp_docker/tests/functional/api/test_crud_distributions.py::CRUDDockerDistributionsTestCase::test_01_create_distribution FAILED

========================================================================================== FAILURES ==========================================================================================
________________________________________________________________ CRUDDockerDistributionsTestCase.test_01_create_distribution _________________________________________________________________

self = <pulp_docker.tests.functional.api.test_crud_distributions.CRUDDockerDistributionsTestCase testMethod=test_01_create_distribution>

    def test_01_create_distribution(self):
        """Create a distribution."""
        body = gen_distribution()
        response_dict = self.client.post(
>           DOCKER_DISTRIBUTION_PATH, body
        )

E           requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http://localhost:24817/pulp/api/v3/tasks/e433b5ff-635b-45d3-a173-d61c3c177ac5/

Logged Traceback:

Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]: pulp: django.request:ERROR: Internal Server Error: /pulp/api/v3/tasks/e433b5ff-635b-45d3-a173-d61c3c177ac5/
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]: Traceback (most recent call last):
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = get_response(request)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = self.process_exception_by_middleware(e, request)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = wrapped_callback(request, *callback_args, **callback_kwargs)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     return view_func(*args, **kwargs)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/viewsets.py", line 114, in view
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     return self.dispatch(request, *args, **kwargs)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 497, in dispatch
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = self.handle_exception(exc)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 457, in handle_exception
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     self.raise_uncaught_exception(exc)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 468, in raise_uncaught_exception
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     raise exc
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 494, in dispatch
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = handler(request, *args, **kwargs)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/mixins.py", line 56, in retrieve
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     return Response(serializer.data)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 559, in data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     ret = super().data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 261, in data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     self._data = self.to_representation(self.instance)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 526, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     ret[field.field_name] = field.to_representation(attribute)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/relations.py", line 535, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     for value in iterable
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/relations.py", line 535, in <listcomp>
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     for value in iterable
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/home/vagrant/devel/pulpcore/pulpcore/app/serializers/task.py", line 31, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     return serializer.data.get('_href')
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 559, in data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     ret = super().data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 261, in data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     self._data = self.to_representation(self.instance)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 526, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     ret[field.field_name] = field.to_representation(attribute)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/home/vagrant/devel/pulp_docker/pulp_docker/app/serializers.py", line 111, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     host = self.context['request'].get_host()
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]: AttributeError: 'NoneType' object has no attribute 'get_host'
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]: 127.0.0.1 - admin [12/Aug/2019:19:44:43 +0000] "GET /pulp/api/v3/tasks/e433b5ff-635b-45d3-a173-d61c3c177ac5/ HTTP/1.1" 500 27 "-" "python-requests/2.22.0"

Workaround

Append `CONTENT_HOST=localhost:24816` to `/etc/pulp/settings.py`, and restart the services. Tests pass.

Associated revisions

Revision e027cd3f View on GitHub
Added by bmbouter 12 months ago

Introduce CONTENT_ORIGIN setting

This also adds a check that the user specified CONTENT_ORIGIN at startup. Pulp refuses to start if it is not set with an error message for the user.

Required PR: https://github.com/PulpQE/pulp-smash/pull/1227 Required PR: https://github.com/pulp/ansible-pulp/pull/185 Required PR: https://github.com/pulp/pulp_file/pull/303

https://pulp.plan.io/issues/5629 re #5629

History

#1 Updated by dkliban@redhat.com about 1 year ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 57

#2 Updated by rchan about 1 year ago

  • Sprint changed from Sprint 57 to Sprint 58

#3 Updated by ipanova@redhat.com about 1 year ago

we are not passing anything into the context that's why it is None https://github.com/pulp/pulpcore/blob/master/pulpcore/app/viewsets/base.py#L336

however this would not solve the problem because request.get_host() would give us Out[1]: 'localhost:24817' instead of localhost:24816

#4 Updated by lmjachky about 1 year ago

It looks to me that the only usage of CONTENT_HOST is in the serializer which serializes the field "registry_path". This field is used for distributions. As @ipanova highlighted out, the method get_host() returns "localhost:24817" instead of "localhost:24816" by default. Therefore, the else statement in pulp_docker https://github.com/pulp/pulp_docker/blob/752698a4251072a2dbfbe4df4793fd3ea4d99fde/pulp_docker/app/serializers.py#L112-L113 is redundant. The workaround is to append the following setting to the file "pulplift/example.dev-config.yml":

pulp_settings:
  content_host: "localhost:24816"

If we allow users to specify CONTENT_HOST (https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html?highlight=content_host#content-host), I think that we should also check if a content server runs upon the same bind https://github.com/pulp/ansible-pulp/blob/95159929a74aa64536a0c76a0d999aa73f5b0b84/roles/pulp-content/templates/pulp-content-app.service.j2#L16. Would not this cause inconsistencies in the future? Like a different port number for registry_path (localhost:24816/foo) and a different port number for the handler (localhost:54321/foo).

Anyway, what are the differences between pulp_content_bind (pulplift/ansible-pulp), pulp_content_host (pulp_docker/.travis), and content_host ([pulpcore|pulp_docker]/.travis)? These values are used almost across all installers. And all of these variables store the same value. Is it possible to use one variable for that? Or at least, the existing variables should be documented in a more transparent way.

#5 Updated by rchan about 1 year ago

  • Sprint deleted (Sprint 58)

#6 Updated by ipanova@redhat.com about 1 year ago

  • Tags Pulp 3 docker blocker added

#7 Updated by lmjachky about 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to lmjachky

#8 Updated by ipanova@redhat.com about 1 year ago

  • Sprint set to Sprint 60

#9 Updated by rchan almost 1 year ago

  • Sprint changed from Sprint 60 to Sprint 61

#10 Updated by lmjachky 12 months ago

  • Status changed from ASSIGNED to POST

#11 Updated by ipanova@redhat.com 12 months ago

  • Status changed from POST to MODIFIED

#12 Updated by ipanova@redhat.com 11 months ago

  • Project changed from Docker Support to Container Support

#13 Updated by ipanova@redhat.com 11 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

#14 Updated by ipanova@redhat.com 9 months ago

  • Sprint/Milestone set to 1.0.0

Please register to edit this issue

Also available in: Atom PDF