Project

Profile

Help

Issue #8862

‘NoneType’ object has no attribute ‘url’ when migrating distribution tree content

Added by dalley 10 days ago. Updated 9 days ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Quarter:

Description

https://community.theforeman.org/t/katello-nonetype-object-has-no-attribute-url/23761/3

May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: c5f35d84-3b21-41de-a63f-c2112b7010ab
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: bf7ab3ac-f0c6-40f8-8799-8cacf5d19e87
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: d0ba0836-aecd-4769-9f05-f74cea0b4b12
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: d2eec823-5f46-4618-83fd-eff5dbd322c2
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: ceccf3f9-d4ab-4ae4-9146-629b71e650b8
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: c91595ad-5da2-475d-8398-00c9ca74e085
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: ccba0a6f-cb68-44a8-9cf0-4cd8210efcfe
May 27 17:33:47 katello.ctsi.mcw.edu pulpcore-worker-8[50551]: pulp: rq.worker:INFO: Cleaning registries for queue: 50551@katello.ctsi.mcw.edu
May 27 17:33:48 katello.ctsi.mcw.edu pulpcore-worker-4[51201]: pulp: rq.worker:INFO: Cleaning registries for queue: 51201@katello.ctsi.mcw.edu
May 27 17:33:48 katello.ctsi.mcw.edu pulpcore-worker-6[51202]: pulp: rq.worker:INFO: Cleaning registries for queue: 51202@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-7[50814]: pulp: rq.worker:INFO: Cleaning registries for queue: 50814@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-3[51204]: pulp: rq.worker:INFO: Cleaning registries for queue: 51204@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-5[51200]: pulp: rq.worker:INFO: Cleaning registries for queue: 51200@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-1[51203]: pulp: rq.worker:INFO: Cleaning registries for queue: 51203@katello.ctsi.mcw.edu
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:ERROR: Traceback (most recent call last):
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: rv = job.perform()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self._result = self._execute()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return self.func(*self.args, **self.kwargs)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 81, in migrate_from_pulp2
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: migrate_content(plan, skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 55, in migrate_content
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 150, in migrate_content_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: loop.run_until_complete(dm.create())
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return future.result()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 89, in create
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await asyncio.gather(*futures)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await self.run()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 181, in run
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self.migrate_to_pulp3(cmodel, ctype)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 375, in migrate_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: url=os.path.join(remote.url, relative_path),
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: AttributeError: 'NoneType' object has no attribute 'url'
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: Traceback (most recent call last):
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: rv = job.perform()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self._result = self._execute()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return self.func(*self.args, **self.kwargs)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 81, in migrate_from_pulp2
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: migrate_content(plan, skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 55, in migrate_content
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 150, in migrate_content_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: loop.run_until_complete(dm.create())
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return future.result()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 89, in create
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await asyncio.gather(*futures)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await self.run()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 181, in run
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self.migrate_to_pulp3(cmodel, ctype)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 375, in migrate_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: url=os.path.join(remote.url, relative_path),
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: AttributeError: 'NoneType' object has no attribute 'url'
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:INFO: Cleaning registries for queue: 51205@katello.ctsi.mcw.edu
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:INFO: 51205@katello.ctsi.mcw.edu: 9f786f0e-c6be-4d40-bd02-30625d04bc0d
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:INFO: 51205@katello.ctsi.mcw.edu: Job OK (9f786f0e-c6be-4d40-bd02-30625d04bc0d)
[root@katello ~]#

History

#1 Updated by dalley 10 days ago

  • Description updated (diff)

#2 Updated by dalley 10 days ago

If not remote and downloaded==True then None will be added to the set of remotes.

https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/plugin/content.py#L326-L329

And then later on, we go through the set of remotes and try to make a DeclarativeArtifact for each, and that's when we try to access remote.url, which blows up

https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/plugin/content.py#L371-L380

#3 Updated by dalley 10 days ago

  • Tags Katello added

#4 Updated by ipanova@redhat.com 9 days ago

I have read the code a bit more and here is my understanding.

Pulp2Importer has not been pre-migrated or pulp3 relation to pulp2importer has not been created, as result None is returned https://github.com/pulp/pulp-2to3-migration/blob/6698454678b8f984b72913f8219334d521d2b0a2/pulp_2to3_migration/app/plugin/content.py#L207

Later on this condition does not result to True because storage_path for the downloaded path exists in pulp2 https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/plugin/content.py#L326

Need to investigate why pulp2importer is not available or why the pulp3 relation is missing

Please register to edit this issue

Also available in: Atom PDF