Project

Profile

Help

Issue #4340

"artifact" field causes serialization to blow up if the content units were lazily synced

Added by dalley 10 months ago. Updated 7 months ago.

Status:
MODIFIED
Priority:
Normal
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
Severity:
2. Medium
Platform Release:
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Target Release - File:
Sprint:
Sprint 48

Description

If you do a lazy sync and then hit up the http GET :8000/pulp/api/v3/content/file/files/ endpoint, you're greeted with a 500 error and the following traceback:


Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]: pulp: django.request:ERROR: Internal Server Error: /pulp/api/v3/content/file/files/
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]: Traceback (most recent call last):
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/options.py", line 564, in get_field
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     return self.fields_map[field_name]
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]: KeyError: 'artifact'
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]: During handling of the above exception, another exception occurred:
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]: Traceback (most recent call last):
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", line 635, in serializable_value
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     field = self._meta.get_field(field_name)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/options.py", line 566, in get_field
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     raise FieldDoesNotExist("%s has no field named '%s'" % (self.object_name, field_name))
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]: django.core.exceptions.FieldDoesNotExist: FileContent has no field named 'artifact'
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]: During handling of the above exception, another exception occurred:
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]: Traceback (most recent call last):
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     response = get_response(request)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/core/handlers/base.py", line 126, in _get_response
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     response = self.process_exception_by_middleware(e, request)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/core/handlers/base.py", line 124, in _get_response
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     response = wrapped_callback(request, *callback_args, **callback_kwargs)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     return view_func(*args, **kwargs)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/viewsets.py", line 116, in view
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     return self.dispatch(request, *args, **kwargs)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 495, in dispatch
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     response = self.handle_exception(exc)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 455, in handle_exception
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     self.raise_uncaught_exception(exc)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/views.py", line 492, in dispatch
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     response = handler(request, *args, **kwargs)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/mixins.py", line 45, in list
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     return self.get_paginated_response(serializer.data)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 765, in data
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     ret = super(ListSerializer, self).data
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 262, in data
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     self._data = self.to_representation(self.instance)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 683, in to_representation
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     self.child.to_representation(item) for item in iterable
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 683, in <listcomp>
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     self.child.to_representation(item) for item in iterable
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/serializers.py", line 514, in to_representation
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     attribute = field.get_attribute(instance)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rest_framework/relations.py", line 182, in get_attribute
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     value = attribute_instance.serializable_value(self.source_attrs[-1])
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/base.py", line 637, in serializable_value
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     return getattr(self, field_name)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_file/app/models.py", line 33, in artifact
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     return self._artifacts.get().pk
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     return getattr(self.get_queryset(), name)(*args, **kwargs)
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:   File "/usr/local/lib/pulp/lib64/python3.7/site-packages/django/db/models/query.py", line 399, in get
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]:     self.model._meta.object_name
Dec 20 18:18:08 p3.pulp.vm gunicorn[10767]: pulpcore.app.models.content.Artifact.DoesNotExist: Artifact matching query does not exist

The problem is that if you use lazy sync, there's no artifact, so this code will fail.

    @property
    def artifact(self):
        """ 
        Return the artifact id (there is only one for this content type).
        """ 
        return self._artifacts.get().pk

This problem is not specific to the file plugin, it's one which probably affects every plugin that copied this setter... which is all of them?

A simple fix to this would be to do something like this:

        x = self._artifacts.first()
        if x is None:
            return None
        return x.pk

Then, when the "artifact" has not yet been created, it would be listed as "null".


Related issues

Related to Pulp - Issue #4329: Not able to get URLs for the content types present in the repository version CLOSED - DUPLICATE Actions
Copied to File Support - Test #4463: Assert that accessing content endpoint does not raise 500 for lazy synced content CLOSED - COMPLETE Actions

History

#1 Updated by gmbnomis 10 months ago

  • Checklist item Ping cookbook plugin maintainer added

Just to confirm: I had the same issue in pulp_cookbook and solved it the way you propose above. The property looks as follows now (not pushed to GitHub yet):

    @property
    def artifact(self):
        try:
            return self._artifacts.get().pk
        except Artifact.DoesNotExist:
            return None

And I added a ping to the cookbook plugin maintainer to the checklist ;-)

#2 Updated by gmbnomis 10 months ago

  • Checklist item Ping cookbook plugin maintainer set to Done

#3 Updated by dalley 10 months ago

Thanks!

Long term, I would really like to get rid of this boilerplate entirely (somehow), but I decided to leave that discussion off the issue for the time being.

#4 Updated by dalley 10 months ago

  • Related to Issue #4329: Not able to get URLs for the content types present in the repository version added

#5 Updated by CodeHeeler 10 months ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 47

#6 Updated by rchan 10 months ago

  • Sprint changed from Sprint 47 to Sprint 48

#7 Updated by dalley 10 months ago

  • Tags Pulp 3 RC Blocker added

#8 Updated by dalley 10 months ago

  • Checklist item deleted (Resolve in pulp_file)
  • Checklist item deleted (Resolve in pulp_python)
  • Checklist item deleted (Resolve in pulp_ansible)
  • Checklist item deleted (Resolve in pulp_docker)
  • Checklist item deleted (Resolve in pulp_rpm)
  • Checklist item deleted (Resolve in plugin_template)
  • Checklist item deleted (Ping debian / gem plugin maintainers)
  • Checklist item deleted (Ping cookbook plugin maintainer)
  • Status changed from ASSIGNED to POST

This issue will be fixed by this PR as part of a larger effort to purge this boilerplate from the plugins.

https://github.com/pulp/pulp_file/pull/159

Story here with more details:

https://pulp.plan.io/issues/4366

Since the idea will be to get rid of this boilerplate altogether, I'm going to get rid of the checklist and consider this issue fixed as soon as the File PR is merged. The linked story has a checklist which I will use.

#9 Updated by dalley 10 months ago

  • Status changed from POST to MODIFIED

#10 Updated by bherring 9 months ago

  • Copied to Test #4463: Assert that accessing content endpoint does not raise 500 for lazy synced content added

#11 Updated by bmbouter 7 months ago

  • Tags deleted (Pulp 3, Pulp 3 RC Blocker)

Please register to edit this issue

Also available in: Atom PDF