Project

Profile

Help

Issue #6058

closed

re-migrating after deleting a pulp2 repo errors with: AttributeError: 'NoneType' object has no attribute 'publication_set'

Added by jsherril@redhat.com about 4 years ago. Updated over 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 65
Quarter:

Description

Here's what i did:

Created a migration plan to migrate 2 repositories:

{"plugins":\[{"type":"iso","repositories":\[{"name":"5b08c8a6-d3bb-41af-9cd4-cadf8fec2a55","repository_versions":\[{"pulp2_repository_id":"5b08c8a6-d3bb-41af-9cd4-cadf8fec2a55","pulp2_distributor_repository_ids":\["5b08c8a6-d3bb-41af-9cd4-cadf8fec2a55"\]}\],"pulp2_importer_repository_id":"5b08c8a6-d3bb-41af-9cd4-cadf8fec2a55"},{"name":"file_test-1-2","repository_versions":\[{"pulp2_repository_id":"1-file_test-v1_0-5b08c8a6-d3bb-41af-9cd4-cadf8fec2a55","pulp2_distributor_repository_ids":\["1-file_test-Library-5b08c8a6-d3bb-41af-9cd4-cadf8fec2a55"\]}\]}\]}\]}

Then deleted one of the repos in pulp2, and created a new plan to migrate the one that is remaining:

{"plugins":[{"type":"iso","repositories":[{"name":"5b08c8a6-d3bb-41af-9cd4-cadf8fec2a55","repository_versions":[{"pulp2_repository_id":"5b08c8a6-d3bb-41af-9cd4-cadf8fec2a55","pulp2_distributor_repository_ids":["5b08c8a6-d3bb-41af-9cd4-cadf8fec2a55"]}],"pulp2_importer_repository_id":"5b08c8a6-d3bb-41af-9cd4-cadf8fec2a55"}]}]}

I then tried to list from the pulp2repositories api, and i get an error:

pulp: django.request:ERROR: Internal Server Error: /pulp/api/v3/pulp2repositories/
Traceback (most recent call last):
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 71, in wrapped_view
return view_func(*args, **kwargs)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/viewsets.py", line 114, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/views.py", line 505, in dispatch
response = self.handle_exception(exc)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/views.py", line 465, in handle_exception
self.raise_uncaught_exception(exc)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
raise exc
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/views.py", line 502, in dispatch
response = handler(request, *args, **kwargs)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/mixins.py", line 43, in list
return self.get_paginated_response(serializer.data)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/serializers.py", line 757, in data
ret = super().data
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/serializers.py", line 261, in data
self._data = self.to_representation(self.instance)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/serializers.py", line 675, in to_representation
self.child.to_representation(item) for item in iterable
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/serializers.py", line 675, in <listcomp>
self.child.to_representation(item) for item in iterable
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/serializers.py", line 526, in to_representation
ret[field.field_name] = field.to_representation(attribute)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/fields.py", line 1873, in to_representation
return method(value)
File "/usr/local/lib/pulp/src/pulp/pulp_2to3_migration/app/serializers.py", line 184, in get_pulp3_publication_href
pub_list = obj.pulp3_repository_version.publication_set.all()
AttributeError: 'NoneType' object has no attribute 'publication_set'
Actions #1

Updated by dalley about 4 years ago

I can't seem to reproduce

(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 pulpcore]$ http GET :24817/pulp/api/v3/pulp2repositories/                                                                                                                                  
....
....

{
    "count": 2,
    "next": null,
    "previous": null,
    "results": [
        {
            "is_migrated": true,
            "not_in_plan": false,
            "pulp2_object_id": "5e2f60dc6e50c9325cda8f53",
            "pulp2_repo_id": "file",
            "pulp3_distribution_hrefs": [
                "/pulp/api/v3/distributions/file/file/a1fbff03-4095-4b03-944d-495042d76591/"
            ],
            "pulp3_publication_href": [
                "/pulp/api/v3/publications/file/file/b3029362-6ee9-4748-9f23-aa460cb50c4e/"
            ],
            "pulp3_remote_href": "/pulp/api/v3/remotes/file/file/595cc126-5042-47b6-af6b-7de4d8e4b596/",
            "pulp3_repository_version": "/pulp/api/v3/repositories/file/file/0ee9f786-1c4c-4529-bdeb-3739a4b262ff/versions/1/",
            "pulp_created": "2020-01-27T23:05:52.466059Z",
            "pulp_href": "/pulp/api/v3/pulp2repositories/3d5ea4c5-cb7b-47f1-b7b7-ef4579b8ec56/"
        },
        {
            "is_migrated": true,
            "not_in_plan": false,
            "pulp2_object_id": "5e2f60dc6e50c9325cda8f59",
            "pulp2_repo_id": "file-many",
            "pulp3_distribution_hrefs": [
                "/pulp/api/v3/distributions/file/file/143bc56d-3c10-4b08-a093-b5e126887f1e/"
            ],
            "pulp3_publication_href": [
                "/pulp/api/v3/publications/file/file/112b9ed5-93b1-4d77-9a51-d1987b46c579/"
            ],
            "pulp3_remote_href": "/pulp/api/v3/remotes/file/file/eed00d69-0bce-4ffc-a0fe-9dbfa1f21a4d/",
            "pulp3_repository_version": "/pulp/api/v3/repositories/file/file/c5450ac6-9382-4d34-9792-ef8c075c54b4/versions/1/",
            "pulp_created": "2020-01-27T23:05:52.415607Z",
            "pulp_href": "/pulp/api/v3/pulp2repositories/ced895c7-43a4-4cb4-bfd2-c0f6b3435bc4/"
        }
    ]
}

(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 pulpcore]$ pulp-admin iso repo delete --repo-id file-many
....

Repository [file-many] successfully deleted

(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 pulpcore]$ http POST :24817/pulp/api/v3/migration-plans/ plan='{"plugins": [{"type": "iso", "repositories": [{"name": "pulp3file", "pulp2_importer_repository_id": "file", "repository_vers
ions": [{"pulp2_repository_id": "file", "pulp2_distributor_repository_ids": ["file"]}]}]}]}'
....
....

(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 pulpcore]$ http POST :24817/pulp/api/v3/migration-plans/a1c5c759-6596-4dfa-a2af-fc19681f1015/run/
....
....

{
    "task": "/pulp/api/v3/tasks/8301c9b8-a0f6-4ee1-9076-95fbde3c62e1/"
}

(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 pulpcore]$ http GET :24817/pulp/api/v3/pulp2repositories/
....
....
{
    "count": 2,
    "next": null,
    "previous": null,
    "results": [
        {
            "is_migrated": true,
            "not_in_plan": true,
            "pulp2_object_id": "5e2f60dc6e50c9325cda8f59",
            "pulp2_repo_id": "file-many",
            "pulp3_distribution_hrefs": [],
            "pulp3_publication_href": [],
            "pulp3_remote_href": null,
            "pulp3_repository_version": "/pulp/api/v3/repositories/file/file/c5450ac6-9382-4d34-9792-ef8c075c54b4/versions/1/",
            "pulp_created": "2020-01-27T23:05:52.415607Z",
            "pulp_href": "/pulp/api/v3/pulp2repositories/ced895c7-43a4-4cb4-bfd2-c0f6b3435bc4/"
        },
        {
            "is_migrated": true,
            "not_in_plan": false,
            "pulp2_object_id": "5e2f60dc6e50c9325cda8f53",
            "pulp2_repo_id": "file",
            "pulp3_distribution_hrefs": [
                "/pulp/api/v3/distributions/file/file/1096a093-e3ea-4b94-a74b-6bf67e741992/"
            ],
            "pulp3_publication_href": [
                "/pulp/api/v3/publications/file/file/1d0fc161-6e06-438b-bee1-d365623838ab/"
            ],
            "pulp3_remote_href": "/pulp/api/v3/remotes/file/file/5e51aea5-48bd-46d5-9deb-1f5581f218f5/",
            "pulp3_repository_version": "/pulp/api/v3/repositories/file/file/0ee9f786-1c4c-4529-bdeb-3739a4b262ff/versions/1/",
            "pulp_created": "2020-01-27T23:05:52.466059Z",
            "pulp_href": "/pulp/api/v3/pulp2repositories/3d5ea4c5-cb7b-47f1-b7b7-ef4579b8ec56/"
        }
    ]
}
Actions #2

Updated by dalley about 4 years ago

Granted, I'm not sure that repository should still be there?

Actions #3

Updated by jsherril@redhat.com about 4 years ago

I will try to reproduce!

Actions #4

Updated by jsherril@redhat.com about 4 years ago

I suspect it might have had something to do with: https://pulp.plan.io/issues/6056

where by the 2nd repo was never fully migrated.

Actions #5

Updated by ipanova@redhat.com about 4 years ago

The only way i managed to reproduce this is to query pulp2repositories while task is still running. When we serialize the results the repo_version has not been created at that time that is why we get that traceback. After task successfully completes i get 200 on pulp2repositories api endpoint

$ http POST :24817/pulp/api/v3/migration-plans/43f46baf-9168-4f85-bfb7-a9ad1ea5f0ad/run/
HTTP/1.1 202 Accepted
Allow: POST, OPTIONS
Connection: close
Content-Length: 67
Content-Type: application/json
Date: Tue, 28 Jan 2020 22:09:34 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN

{
    "task": "/pulp/api/v3/tasks/7ded9d41-c338-4143-aad2-71f3f2127e41/"
}

(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 devel]$ http GET :24817/pulp/api/v3/pulp2repositories/
HTTP/1.1 500 Internal Server Error
Connection: close
Content-Length: 27
Content-Type: text/html
Date: Tue, 28 Jan 2020 22:09:36 GMT
Server: gunicorn/20.0.4
Vary: Cookie
X-Frame-Options: SAMEORIGIN

<h1>Server Error (500)</h1>

(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 devel]$ http GET :24817/pulp/api/v3/pulp2repositories/
HTTP/1.1 500 Internal Server Error
Connection: close
Content-Length: 27
Content-Type: text/html
Date: Tue, 28 Jan 2020 22:09:37 GMT
Server: gunicorn/20.0.4
Vary: Cookie
X-Frame-Options: SAMEORIGIN

<h1>Server Error (500)</h1>

(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 devel]$ pjournal 
-- Logs begin at Tue 2020-01-28 15:21:34 UTC. --
Jan 28 22:09:37 pulp2-nightly-pulp3-source-centos7.fluffy.example.com gunicorn[11859]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/serializers.py", line 675, in <listcomp>
Jan 28 22:09:37 pulp2-nightly-pulp3-source-centos7.fluffy.example.com gunicorn[11859]: self.child.to_representation(item) for item in iterable
Jan 28 22:09:37 pulp2-nightly-pulp3-source-centos7.fluffy.example.com gunicorn[11859]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/serializers.py", line 526, in to_representation
Jan 28 22:09:37 pulp2-nightly-pulp3-source-centos7.fluffy.example.com gunicorn[11859]: ret[field.field_name] = field.to_representation(attribute)
Jan 28 22:09:37 pulp2-nightly-pulp3-source-centos7.fluffy.example.com gunicorn[11859]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/fields.py", line 1873, in to_representation
Jan 28 22:09:37 pulp2-nightly-pulp3-source-centos7.fluffy.example.com gunicorn[11859]: return method(value)
Jan 28 22:09:37 pulp2-nightly-pulp3-source-centos7.fluffy.example.com gunicorn[11859]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/serializers.py", line 184, in get_pulp3_publication_href
Jan 28 22:09:37 pulp2-nightly-pulp3-source-centos7.fluffy.example.com gunicorn[11859]: pub_list = obj.pulp3_repository_version.publication_set.all()
Jan 28 22:09:37 pulp2-nightly-pulp3-source-centos7.fluffy.example.com gunicorn[11859]: AttributeError: 'NoneType' object has no attribute 'publication_set'
Jan 28 22:09:37 pulp2-nightly-pulp3-source-centos7.fluffy.example.com gunicorn[11859]: 127.0.0.1 - admin [28/Jan/2020:22:09:37 +0000] "GET /pulp/api/v3/pulp2repositories/ HTTP/1.1" 500 27 "-" "HTTPie/1.0.3"
Jan 28 22:09:40 pulp2-nightly-pulp3-source-centos7.fluffy.example.com rq[11856]: pulp: rq.worker:INFO: 11856@pulp2-nightly-pulp3-source-centos7.fluffy.example.com: Job OK (7ded9d41-c338-4143-aad2-71f3f2127e41)
Jan 28 22:09:40 pulp2-nightly-pulp3-source-centos7.fluffy.example.com rq[11856]: pulp: rq.worker:INFO: 11856@pulp2-nightly-pulp3-source-centos7.fluffy.example.com: b053f51d-d2cc-4b43-9b31-5458e7a93362
Jan 28 22:09:40 pulp2-nightly-pulp3-source-centos7.fluffy.example.com rq[11856]: pulp: rq.worker:INFO: 11856@pulp2-nightly-pulp3-source-centos7.fluffy.example.com: Job OK (b053f51d-d2cc-4b43-9b31-5458e7a93362)
^[[A^[[B^C
(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 devel]$ http GET :24817/pulp/api/v3/pulp2repositories/
HTTP/1.1 200 OK
Allow: GET, HEAD, OPTIONS
Connection: close
Content-Length: 2667
Content-Type: application/json
Date: Tue, 28 Jan 2020 22:09:48 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN

{
    "count": 4,
    "next": null,
    "previous": null,
    "results": [
        {
            "is_migrated": true,
            "not_in_plan": false,
            "pulp2_object_id": "5e1c932ec998ac367cd60d85",
            "pulp2_repo_id": "file2",
            "pulp3_distribution_hrefs": [
                "/pulp/api/v3/distributions/file/file/2eef2d4f-f4ca-4c61-ac08-1f42089d2293/"
            ],
            "pulp3_publication_href": [
                "/pulp/api/v3/publications/file/file/0607d4ea-ce79-4ec5-b94d-8389b0f5066d/"
            ],
            "pulp3_remote_href": "/pulp/api/v3/remotes/file/file/7549b885-f930-4475-9c0c-a37956960f3c/",
            "pulp3_repository_version": "/pulp/api/v3/repositories/file/file/bfc30d9b-1dc7-4d3a-9801-602b2d00fe4a/versions/1/",
            "pulp_created": "2020-01-28T22:09:34.824798Z",
            "pulp_href": "/pulp/api/v3/pulp2repositories/d6fd4544-17a4-4aab-a633-25ad7bd4c88c/"
        },
        {
       <snip>

@jsherril can we close this out? in case your issue was caused by https://pulp.plan.io/issues/6058#note-4 that would mean that the task failed and with failed task we cannot guarantee serialization of the data since it has not been migrated fully/correctly.

Actions #6

Updated by jsherril@redhat.com about 4 years ago

Yes, i do think that this only happens when a migration failure occurs. However i think we need to either:

1) ensure that 'dirty' data that causes a 500 on error can never get created (or gets cleaned up)
2) relax the rendering of the pulp2repositories api to handle this situation
3) don't return those pulp2repositories as part of the api

i've seen this multiple times, essentially whenever a migration fails for whatever reason, it leaves the pulp2Repository api returning a 500 ISE, which i think it should never do.

Thoughts?

Actions #7

Updated by ipanova@redhat.com about 4 years ago

  • Description updated (diff)
  • Status changed from NEW to POST
  • Assignee set to ipanova@redhat.com
  • Triaged changed from No to Yes
  • Sprint set to Sprint 65
Actions #8

Updated by ipanova@redhat.com about 4 years ago

@justin i eliminated the 500 ISE error, but we still are displaying repos that did not fully migrate( for example publication has not been created for it, you will just need null on the publication href)

Actions #9

Updated by ttereshc about 4 years ago

  • Description updated (diff)

pre'd the description

Added by ipanova@redhat.com about 4 years ago

Revision 5785ff67 | View on GitHub

Fix rendering of the pulp2repositories after a failed migration.

closes #6058

Added by ipanova@redhat.com about 4 years ago

Revision 5785ff67 | View on GitHub

Fix rendering of the pulp2repositories after a failed migration.

closes #6058

Added by ipanova@redhat.com about 4 years ago

Revision 5785ff67 | View on GitHub

Fix rendering of the pulp2repositories after a failed migration.

closes #6058

Actions #10

Updated by ipanova@redhat.com about 4 years ago

  • Status changed from POST to MODIFIED
Actions #13

Updated by ttereshc about 4 years ago

  • Sprint/Milestone set to 0.1.0
Actions #14

Updated by ttereshc about 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #15

Updated by ggainey almost 4 years ago

  • Tags Katello added
  • Tags deleted (Katello-P2)

Also available in: Atom PDF