Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #8862

closed

‘NoneType’ object has no attribute ‘url’ when migrating distribution tree content

Added by dalley over 3 years ago. Updated about 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
ttereshc
Sprint/Milestone:
0.15.0
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 105
Quarter:

Description

https://community.theforeman.org/t/katello-nonetype-object-has-no-attribute-url/23761/3

May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: c5f35d84-3b21-41de-a63f-c2112b7010ab
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: bf7ab3ac-f0c6-40f8-8799-8cacf5d19e87
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: d0ba0836-aecd-4769-9f05-f74cea0b4b12
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: d2eec823-5f46-4618-83fd-eff5dbd322c2
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: ceccf3f9-d4ab-4ae4-9146-629b71e650b8
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: c91595ad-5da2-475d-8398-00c9ca74e085
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: ccba0a6f-cb68-44a8-9cf0-4cd8210efcfe
May 27 17:33:47 katello.ctsi.mcw.edu pulpcore-worker-8[50551]: pulp: rq.worker:INFO: Cleaning registries for queue: 50551@katello.ctsi.mcw.edu
May 27 17:33:48 katello.ctsi.mcw.edu pulpcore-worker-4[51201]: pulp: rq.worker:INFO: Cleaning registries for queue: 51201@katello.ctsi.mcw.edu
May 27 17:33:48 katello.ctsi.mcw.edu pulpcore-worker-6[51202]: pulp: rq.worker:INFO: Cleaning registries for queue: 51202@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-7[50814]: pulp: rq.worker:INFO: Cleaning registries for queue: 50814@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-3[51204]: pulp: rq.worker:INFO: Cleaning registries for queue: 51204@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-5[51200]: pulp: rq.worker:INFO: Cleaning registries for queue: 51200@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-1[51203]: pulp: rq.worker:INFO: Cleaning registries for queue: 51203@katello.ctsi.mcw.edu
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:ERROR: Traceback (most recent call last):
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: rv = job.perform()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self._result = self._execute()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return self.func(*self.args, **self.kwargs)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 81, in migrate_from_pulp2
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: migrate_content(plan, skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 55, in migrate_content
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 150, in migrate_content_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: loop.run_until_complete(dm.create())
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return future.result()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 89, in create
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await asyncio.gather(*futures)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await self.run()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 181, in run
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self.migrate_to_pulp3(cmodel, ctype)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 375, in migrate_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: url=os.path.join(remote.url, relative_path),
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: AttributeError: 'NoneType' object has no attribute 'url'
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: Traceback (most recent call last):
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: rv = job.perform()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self._result = self._execute()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return self.func(*self.args, **self.kwargs)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 81, in migrate_from_pulp2
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: migrate_content(plan, skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 55, in migrate_content
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 150, in migrate_content_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: loop.run_until_complete(dm.create())
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return future.result()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 89, in create
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await asyncio.gather(*futures)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await self.run()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 181, in run
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self.migrate_to_pulp3(cmodel, ctype)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 375, in migrate_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: url=os.path.join(remote.url, relative_path),
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: AttributeError: 'NoneType' object has no attribute 'url'
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:INFO: Cleaning registries for queue: 51205@katello.ctsi.mcw.edu
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:INFO: 51205@katello.ctsi.mcw.edu: 9f786f0e-c6be-4d40-bd02-30625d04bc0d
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:INFO: 51205@katello.ctsi.mcw.edu: Job OK (9f786f0e-c6be-4d40-bd02-30625d04bc0d)
[root@katello ~]#

Related issues

Copied to Migration Plugin - Backport #9372: Backport "‘NoneType’ object has no attribute ‘url’ when migrating distribution tree content" to 0.11.zCLOSED - CURRENTRELEASEttereshc

Actions
Actions
Copy link
 #1

Updated by dalley over 3 years ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by dalley over 3 years ago

If not remote and downloaded==True then None will be added to the set of remotes.

https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/plugin/content.py#L326-L329

And then later on, we go through the set of remotes and try to make a DeclarativeArtifact for each, and that's when we try to access remote.url, which blows up

https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/plugin/content.py#L371-L380

Actions
Copy link
 #3

Updated by dalley over 3 years ago

  • Tags Katello added
Actions
Copy link
 #4

Updated by ipanova@redhat.com over 3 years ago

I have read the code a bit more and here is my understanding.

Pulp2Importer has not been pre-migrated or pulp3 relation to pulp2importer has not been created, as result None is returned https://github.com/pulp/pulp-2to3-migration/blob/6698454678b8f984b72913f8219334d521d2b0a2/pulp_2to3_migration/app/plugin/content.py#L207

Later on this condition does not result to True because storage_path for the downloaded path exists in pulp2 https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/plugin/content.py#L326

Need to investigate why pulp2importer is not available or why the pulp3 relation is missing

Actions
Copy link
 #5

Updated by ttereshc about 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc
  • Sprint set to Sprint 103
Actions
Copy link
 #6

Updated by rchan about 3 years ago

  • Sprint changed from Sprint 103 to Sprint 104
Actions
Copy link
 #7

Updated by ttereshc about 3 years ago

  • Status changed from ASSIGNED to POST
Actions
Copy link
 #8

Updated by ttereshc about 3 years ago

To reproduce:

  • sync a kickstart repo which has images with the immediate policy into pulp2
  • create a copy of that repo
  • migrate only a copied repo
Actions
Copy link
 #10

Updated by rchan about 3 years ago

  • Sprint changed from Sprint 104 to Sprint 105
Actions
Copy link
 #11

Updated by ttereshc about 3 years ago

  • Copied to Backport #9372: Backport "‘NoneType’ object has no attribute ‘url’ when migrating distribution tree content" to 0.11.z added

Added by ttereshc about 3 years ago

Revision de9c18a3 | View on GitHub

Fix the case when pulp2 importer is not migrated but its LCE is used.

closes #8862 https://pulp.plan.io/issues/8862

Added by ttereshc about 3 years ago

Revision de9c18a3 | View on GitHub

Fix the case when pulp2 importer is not migrated but its LCE is used.

closes #8862 https://pulp.plan.io/issues/8862

Added by ttereshc about 3 years ago

Revision de9c18a3 | View on GitHub

Fix the case when pulp2 importer is not migrated but its LCE is used.

closes #8862 https://pulp.plan.io/issues/8862

Actions
Copy link
 #13

Updated by ttereshc about 3 years ago

  • Status changed from POST to MODIFIED
Actions
Copy link
 #14

Updated by ttereshc about 3 years ago

  • Sprint/Milestone set to 0.15.0
Actions
Copy link
 #15

Updated by pulpbot about 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions
Send by e-mail Copy link

Also available in: Atom PDF