Migration Plugin

There is an attempt to exclude pulp3_repository_version if it's not set.

This causes a failure if I try to limit the output to certain fields. Even if I ask to include the field itself.

[vagrant@pulp2-nightly-pulp3-source-centos7 ~]$ http ':/pulp/api/v3/pulp2content/?limit=1&fields=["pulp3_repository_version"]'
HTTP/1.1 500 Internal Server Error
Connection: close
Content-Length: 27
Content-Type: text/html; charset=UTF-8
Date: Fri, 11 Dec 2020 19:19:36 GMT
Server: gunicorn/20.0.4
Vary: Cookie
X-Frame-Options: SAMEORIGIN

<h1>Server Error (500)</h1>

Logs

Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: pulp: django.request:ERROR: Internal Server Error: /pulp/api/v3/pulp2content/
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: Traceback (most recent call last):
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: response = get_response(request)
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: response = self.process_exception_by_middleware(e, request)
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: response = wrapped_callback(request, *callback_args, **callback_kwargs)
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: return view_func(*args, **kwargs)
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/viewsets.py", line 114, in view
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: return self.dispatch(request, *args, **kwargs)
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/views.py", line 505, in dispatch
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: response = self.handle_exception(exc)
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/views.py", line 465, in handle_exception
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: self.raise_uncaught_exception(exc)
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: raise exc
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/views.py", line 502, in dispatch
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: response = handler(request, *args, **kwargs)
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/mixins.py", line 43, in list
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: return self.get_paginated_response(serializer.data)
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/serializers.py", line 760, in data
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: ret = super().data
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/serializers.py", line 260, in data
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: self._data = self.to_representation(self.instance)
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/serializers.py", line 678, in to_representation
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: self.child.to_representation(item) for item in iterable
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/serializers.py", line 678, in <listcomp>
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: self.child.to_representation(item) for item in iterable
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/serializers.py", line 172, in to_representation
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: del result['pulp3_repository_version']
Dec 11 19:08:34 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[32087]: KeyError: 'pulp3_repository_version'

It's easy to fix by checking if the key is there. However I wonder if there are more issues here. if I make a request using httpie witout any fields specified, I get the expected format, no pulp3_repository_version, because it's None.

$ http ':/pulp/api/v3/pulp2content/?limit=1'
HTTP/1.1 200 OK
Allow: GET, HEAD, OPTIONS
Connection: Keep-Alive
Content-Length: 586
Content-Type: application/json
Date: Fri, 11 Dec 2020 19:26:08 GMT
Keep-Alive: timeout=5, max=10000
Server: gunicorn/20.0.4
Vary: Accept,Cookie
X-Frame-Options: SAMEORIGIN

{
    "count": 266,
    "next": "http://127.0.0.1:24817/pulp/api/v3/pulp2content/?limit=1&offset=1",
    "previous": null,
    "results": [
        {
            "downloaded": true,
            "pulp2_content_type_id": "iso",
            "pulp2_id": "eedfce4b-d1d9-49b6-be5d-5d8c002ac3ab",
            "pulp2_last_updated": 1572546758,
            "pulp2_storage_path": "/var/lib/pulp/content/units/iso/6c/855d3e0d05e36b074280d7bac3ae14c5f3fcd5a0b50117332370bd614d72bd/5.iso",
            "pulp3_content": "/pulp/api/v3/content/file/files/6b9174ef-72ed-44d4-a61f-7458c7e2f1fd/",
            "pulp_created": "2020-12-11T18:29:43.890958Z",
            "pulp_href": "/pulp/api/v3/pulp2content/757d4bcf-ef54-4519-9a20-8b1d369f6798/"
        }
    ]
}

When I use bindings, I see it there with value None. Is it intentional? I thought that bindings are expected to produce the same result as API.

In [22]: pulp2content_api.list(ordering='pulp2_id', limit=1)
Out[22]: 
{'count': 266,
 'next': 'http://127.0.0.1:24817/pulp/api/v3/pulp2content/?limit=1&offset=1&ordering=pulp2_id',
 'previous': None,
 'results': [{'downloaded': False,
              'pulp2_content_type_id': 'iso',
              'pulp2_id': '006d116f-aaa0-498b-82d4-f97aeb9f340c',
              'pulp2_last_updated': 1572546754,
              'pulp2_storage_path': '/var/lib/pulp/content/units/iso/2d/ffc2fa86aff8ebe792baee3fa7df0daee50888db60d9bba32c05ed68901288',
              'pulp3_content': '/pulp/api/v3/content/file/files/f816fb17-320d-496e-a16d-a8b5e17686cd/',
              'pulp3_repository_version': None,
              'pulp_created': datetime.datetime(2020, 12, 11, 18, 29, 43, 886139, tzinfo=tzlocal()),
              'pulp_href': '/pulp/api/v3/pulp2content/8b94653f-9e74-4911-b258-178906a77cc5/'}]}

And a separate question, do we need to exclude it from the output at all? It seems that it would be better if API is consistent and always has certain fields present. Any reasons to exclude it?