Project

Profile

Help

Issue #5253

Docker distributions fail when CONTENT_HOST is not set

Added by amacdona@redhat.com 3 months ago. Updated 11 days ago.

Status:
MODIFIED
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Severity:
2. Medium
Version - Docker:
Platform Release:
Blocks Release:
Target Release - Docker:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 3 docker blocker
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 61

Description

In a default pulplift environment, content_host is not included in the pulp_settings. In this environment, retrieving the task with a created_resource of a distribution fails.

This is covered by the functional tests, which fail out of the box on pulplift.

pulp_docker/tests/functional/api/test_crud_distributions.py::CRUDDockerDistributionsTestCase::test_01_create_distribution FAILED

========================================================================================== FAILURES ==========================================================================================
________________________________________________________________ CRUDDockerDistributionsTestCase.test_01_create_distribution _________________________________________________________________

self = <pulp_docker.tests.functional.api.test_crud_distributions.CRUDDockerDistributionsTestCase testMethod=test_01_create_distribution>

    def test_01_create_distribution(self):
        """Create a distribution.""" 
        body = gen_distribution()
        response_dict = self.client.post(
>           DOCKER_DISTRIBUTION_PATH, body
        )

E           requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http://localhost:24817/pulp/api/v3/tasks/e433b5ff-635b-45d3-a173-d61c3c177ac5/

Logged Traceback:

Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]: pulp: django.request:ERROR: Internal Server Error: /pulp/api/v3/tasks/e433b5ff-635b-45d3-a173-d61c3c177ac5/
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]: Traceback (most recent call last):
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = get_response(request)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = self.process_exception_by_middleware(e, request)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = wrapped_callback(request, *callback_args, **callback_kwargs)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     return view_func(*args, **kwargs)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/viewsets.py", line 114, in view
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     return self.dispatch(request, *args, **kwargs)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 497, in dispatch
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = self.handle_exception(exc)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 457, in handle_exception
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     self.raise_uncaught_exception(exc)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 468, in raise_uncaught_exception
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     raise exc
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 494, in dispatch
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     response = handler(request, *args, **kwargs)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/mixins.py", line 56, in retrieve
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     return Response(serializer.data)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 559, in data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     ret = super().data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 261, in data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     self._data = self.to_representation(self.instance)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 526, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     ret[field.field_name] = field.to_representation(attribute)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/relations.py", line 535, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     for value in iterable
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/relations.py", line 535, in <listcomp>
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     for value in iterable
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/home/vagrant/devel/pulpcore/pulpcore/app/serializers/task.py", line 31, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     return serializer.data.get('_href')
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 559, in data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     ret = super().data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 261, in data
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     self._data = self.to_representation(self.instance)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 526, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     ret[field.field_name] = field.to_representation(attribute)
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:   File "/home/vagrant/devel/pulp_docker/pulp_docker/app/serializers.py", line 111, in to_representation
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]:     host = self.context['request'].get_host()
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]: AttributeError: 'NoneType' object has no attribute 'get_host'
Aug 12 19:44:43 pulp3-source-fedora29.localhost.example.com gunicorn[28726]: 127.0.0.1 - admin [12/Aug/2019:19:44:43 +0000] "GET /pulp/api/v3/tasks/e433b5ff-635b-45d3-a173-d61c3c177ac5/ HTTP/1.1" 500 27 "-" "python-requests/2.22.0" 

Workaround

Append `CONTENT_HOST=localhost:24816` to `/etc/pulp/settings.py`, and restart the services. Tests pass.

Associated revisions

Revision e027cd3f View on GitHub
Added by bmbouter 11 days ago

Introduce CONTENT_ORIGIN setting

This also adds a check that the user specified CONTENT_ORIGIN at
startup. Pulp refuses to start if it is not set with an error message
for the user.

Required PR: https://github.com/PulpQE/pulp-smash/pull/1227
Required PR: https://github.com/pulp/ansible-pulp/pull/185
Required PR: https://github.com/pulp/pulp_file/pull/303

https://pulp.plan.io/issues/5629
re #5629

History

#1 Updated by dkliban@redhat.com 3 months ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 57

#2 Updated by rchan 3 months ago

  • Sprint changed from Sprint 57 to Sprint 58

#3 Updated by ipanova@redhat.com 3 months ago

we are not passing anything into the context that's why it is None https://github.com/pulp/pulpcore/blob/master/pulpcore/app/viewsets/base.py#L336

however this would not solve the problem because request.get_host() would give us Out1: 'localhost:24817' instead of localhost:24816

#4 Updated by lmjachky 3 months ago

It looks to me that the only usage of CONTENT_HOST is in the serializer which serializes the field "registry_path". This field is used for distributions. As @ipanova highlighted out, the method get_host() returns "localhost:24817" instead of "localhost:24816" by default. Therefore, the else statement in pulp_docker https://github.com/pulp/pulp_docker/blob/752698a4251072a2dbfbe4df4793fd3ea4d99fde/pulp_docker/app/serializers.py#L112-L113 is redundant. The workaround is to append the following setting to the file "pulplift/example.dev-config.yml":

pulp_settings:
  content_host: "localhost:24816" 

If we allow users to specify CONTENT_HOST (https://docs.pulpproject.org/en/3.0/nightly/installation/configuration.html?highlight=content_host#content-host), I think that we should also check if a content server runs upon the same bind https://github.com/pulp/ansible-pulp/blob/95159929a74aa64536a0c76a0d999aa73f5b0b84/roles/pulp-content/templates/pulp-content-app.service.j2#L16. Would not this cause inconsistencies in the future? Like a different port number for registry_path (localhost:24816/foo) and a different port number for the handler (localhost:54321/foo).

Anyway, what are the differences between pulp_content_bind (pulplift/ansible-pulp), pulp_content_host (pulp_docker/.travis), and content_host ([pulpcore|pulp_docker]/.travis)? These values are used almost across all installers. And all of these variables store the same value. Is it possible to use one variable for that? Or at least, the existing variables should be documented in a more transparent way.

#5 Updated by rchan 2 months ago

  • Sprint deleted (Sprint 58)

#6 Updated by ipanova@redhat.com about 2 months ago

  • Tags Pulp 3 docker blocker added

#7 Updated by lmjachky about 1 month ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to lmjachky

#8 Updated by ipanova@redhat.com about 1 month ago

  • Sprint set to Sprint 60

#9 Updated by rchan 26 days ago

  • Sprint changed from Sprint 60 to Sprint 61

#10 Updated by lmjachky 15 days ago

  • Status changed from ASSIGNED to POST

#11 Updated by ipanova@redhat.com 11 days ago

  • Status changed from POST to MODIFIED

Please register to edit this issue

Also available in: Atom PDF