Issue #8862
closed‘NoneType’ object has no attribute ‘url’ when migrating distribution tree content
Description
https://community.theforeman.org/t/katello-nonetype-object-has-no-attribute-url/23761/3
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: c5f35d84-3b21-41de-a63f-c2112b7010ab
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: bf7ab3ac-f0c6-40f8-8799-8cacf5d19e87
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: d0ba0836-aecd-4769-9f05-f74cea0b4b12
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: d2eec823-5f46-4618-83fd-eff5dbd322c2
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: ceccf3f9-d4ab-4ae4-9146-629b71e650b8
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: c91595ad-5da2-475d-8398-00c9ca74e085
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: ccba0a6f-cb68-44a8-9cf0-4cd8210efcfe
May 27 17:33:47 katello.ctsi.mcw.edu pulpcore-worker-8[50551]: pulp: rq.worker:INFO: Cleaning registries for queue: 50551@katello.ctsi.mcw.edu
May 27 17:33:48 katello.ctsi.mcw.edu pulpcore-worker-4[51201]: pulp: rq.worker:INFO: Cleaning registries for queue: 51201@katello.ctsi.mcw.edu
May 27 17:33:48 katello.ctsi.mcw.edu pulpcore-worker-6[51202]: pulp: rq.worker:INFO: Cleaning registries for queue: 51202@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-7[50814]: pulp: rq.worker:INFO: Cleaning registries for queue: 50814@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-3[51204]: pulp: rq.worker:INFO: Cleaning registries for queue: 51204@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-5[51200]: pulp: rq.worker:INFO: Cleaning registries for queue: 51200@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-1[51203]: pulp: rq.worker:INFO: Cleaning registries for queue: 51203@katello.ctsi.mcw.edu
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:ERROR: Traceback (most recent call last):
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: rv = job.perform()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self._result = self._execute()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return self.func(*self.args, **self.kwargs)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 81, in migrate_from_pulp2
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: migrate_content(plan, skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 55, in migrate_content
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 150, in migrate_content_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: loop.run_until_complete(dm.create())
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return future.result()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 89, in create
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await asyncio.gather(*futures)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await self.run()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 181, in run
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self.migrate_to_pulp3(cmodel, ctype)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 375, in migrate_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: url=os.path.join(remote.url, relative_path),
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: AttributeError: 'NoneType' object has no attribute 'url'
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: Traceback (most recent call last):
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: rv = job.perform()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self._result = self._execute()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return self.func(*self.args, **self.kwargs)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 81, in migrate_from_pulp2
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: migrate_content(plan, skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 55, in migrate_content
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 150, in migrate_content_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: loop.run_until_complete(dm.create())
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return future.result()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 89, in create
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await asyncio.gather(*futures)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await self.run()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 181, in run
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self.migrate_to_pulp3(cmodel, ctype)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 375, in migrate_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: url=os.path.join(remote.url, relative_path),
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: AttributeError: 'NoneType' object has no attribute 'url'
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:INFO: Cleaning registries for queue: 51205@katello.ctsi.mcw.edu
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:INFO: 51205@katello.ctsi.mcw.edu: 9f786f0e-c6be-4d40-bd02-30625d04bc0d
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:INFO: 51205@katello.ctsi.mcw.edu: Job OK (9f786f0e-c6be-4d40-bd02-30625d04bc0d)
[root@katello ~]#
Related issues
Updated by dalley about 3 years ago
If not remote
and downloaded==True
then None
will be added to the set of remotes.
And then later on, we go through the set of remotes and try to make a DeclarativeArtifact for each, and that's when we try to access remote.url
, which blows up
Updated by ipanova@redhat.com about 3 years ago
I have read the code a bit more and here is my understanding.
Pulp2Importer has not been pre-migrated or pulp3 relation to pulp2importer has not been created, as result None is returned https://github.com/pulp/pulp-2to3-migration/blob/6698454678b8f984b72913f8219334d521d2b0a2/pulp_2to3_migration/app/plugin/content.py#L207
Later on this condition does not result to True because storage_path for the downloaded path exists in pulp2 https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/plugin/content.py#L326
Need to investigate why pulp2importer is not available or why the pulp3 relation is missing
Updated by ttereshc almost 3 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to ttereshc
- Sprint set to Sprint 103
Updated by rchan almost 3 years ago
- Sprint changed from Sprint 103 to Sprint 104
Updated by ttereshc almost 3 years ago
- Status changed from ASSIGNED to POST
Updated by ttereshc almost 3 years ago
To reproduce:
- sync a kickstart repo which has images with the immediate policy into pulp2
- create a copy of that repo
- migrate only a copied repo
Updated by rchan almost 3 years ago
- Sprint changed from Sprint 104 to Sprint 105
Updated by ttereshc almost 3 years ago
- Copied to Backport #9372: Backport "‘NoneType’ object has no attribute ‘url’ when migrating distribution tree content" to 0.11.z added
Added by ttereshc almost 3 years ago
Added by ttereshc almost 3 years ago
Revision de9c18a3 | View on GitHub
Fix the case when pulp2 importer is not migrated but its LCE is used.
Added by ttereshc almost 3 years ago
Revision de9c18a3 | View on GitHub
Fix the case when pulp2 importer is not migrated but its LCE is used.
Updated by ttereshc almost 3 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulp:pulp-2to3-migration|de9c18a3ab827f06800c3c02286d6fbc2b9b2c83.
Updated by pulpbot almost 3 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Fix the case when pulp2 importer is not migrated but its LCE is used.
closes #8862 https://pulp.plan.io/issues/8862