Project

Profile

Help

Issue #7876

NoneType' object has no attribute 'pk'

Added by adam.winberg@smhi.se 10 days ago. Updated 6 days ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

On a fresh pulp3 installation, running consecutive 2to3 migrations results in following error (first migration ran without error):

Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: Traceback (most recent call last):
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:   File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:     rv = job.perform()
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:   File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:     self._result = self._execute()
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:   File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:     return self.func(*self.args, **self.kwargs)
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 139, in migrate_from_pulp2
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:     migrate_content(plan)
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 36, in migrate_content
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:     plugin.migrator.migrate_content_to_pulp3()
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 140, in migrate_content_to_pulp3
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:     loop.run_until_complete(dm.create())
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:   File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:     return future.result()
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 88, in create
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:     await pipeline
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:   File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:     await asyncio.gather(*futures)
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:   File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:     await self.run()
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:   File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 244, in run
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:     RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch))
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:   File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 296, in _needed_remote_artifacts
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]:     if remote_artifact.remote_id == d_artifact.remote.pk:
Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: AttributeError: 'NoneType' object has no attribute 'pk'

The first couple of migrations resulted in #7842. Next day, new migration attempt and now I get this, which is produced earlier in the migration process than the previous error.

Running in an rpm-based installation on RHEL8:

python3-pulp-rpm-3.7.0-1.el8.noarch
python3-pulpcore-3.7.3-1.el8.noarch
 python3-pulp-2to3-migration-0.5.1-1.el8.noarch

History

#1 Updated by ttereshc 6 days ago

Hi Adam,

Do you have a large pulp2 setup? Many repositories? I think this issue depends on pulp2 data and it would be helpful to know some details, so we can reproduce the problem.

How do you get content in pulp2? Is it usually sync? On_demand? Do you upload your own content into pulp2? Do you copy content between repositories? Is it possible that there are repositories without any feed in the importer?

#2 Updated by ttereshc 6 days ago

FWIW, I haven't tracked it down yet. I think it might be an issue in pulpcore for the case when the same content is migrated with the remote and without.

#3 Updated by adam.winberg@smhi.se 6 days ago

We have about 80 repos, where 50 of them have a feed-url, and 38 of those are synced on a regular basis. The other 12 are synced manually when needed. And the repos without a feed-url are used as frozen repo images, which we copy rpm/erratas to from the synced repos in a more controlled manner. For example we have the RHEL8 baseos repo which we sync every day, and once a week we copy important erratas from that repo to our 'frozen-baseos' repo which is then used to serve our clients.

We very seldom upload our own content, it is almost all synced from external repos.

Some outputs from migration:

    {
      "code": "migrating.rpm.content",
      "done": 153807,
      "message": "Migrating rpm content to Pulp 3 rpm",
      "state": "completed",
      "suffix": null,
      "total": 153807
    },

    {
      "code": "migrating.rpm.content",
      "done": 90653,
      "message": "Migrating rpm content to Pulp 3 erratum",
      "state": "completed",
      "suffix": null,
      "total": 90653 
    },

Please register to edit this issue

Also available in: Atom PDF