Actions
Task #2380
closed
Create a redmine task for each 2.y celery task to be converted to 3.0
Added by bmbouter over 7 years ago. Updated over 3 years ago.
Start date:
Due date:
% Done:
0%
Estimated time:
Platform Release:
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Sprint:
Sprint 10
Quarter:
Description
We have roughly 20-30 celery tasks in Pulp 2.y. We need to have a redmine task created for each one tracking the conversion of that celery task to Pulp3.
To identify the list of celery tasks in 2.y I recommend inspecting the celery registry of a 2.y dev install. You can inspect the registry using a technique similar to what this[0] unit test does.
Each task needs to be moved out of the pulp.server and into somewhere. We should discuss on the issues themselves where each one should go. Also on Pulp3 all of the tasks are expected to be imported into pulp.tasking.registry which is where celery will discover them. Note that it's probably not a good idea to have the task code itself live in pulp.tasking.registry so where to house that code is something we should discuss.
The details of what the Pulp3 behaviors should be for that task can be discussed on each issue created. There are a few that will be deleted. Let's make issues for all the 2.y tasks and then we can identify ones to close on the individual issues.
Related issues
Updated by mhrivnak over 7 years ago
$ sudo -u apache celery inspect registered -A pulp.server.async.app
-> reserved_resource_worker-0@dev: OK
* pulp.server.async.tasks.PulpTask
* pulp.server.async.tasks.Task
* pulp.server.async.tasks._queue_reserved_task
* pulp.server.async.tasks._release_resource
* pulp.server.controllers.repository.download_deferred
* pulp.server.controllers.repository.download_repo
* pulp.server.controllers.repository.queue_download_deferred
* pulp.server.controllers.repository.queue_publish
* pulp.server.db.reaper.queue_reap_expired_documents
* pulp.server.db.reaper.reap_expired_documents
* pulp.server.maintenance.monthly.monthly_maintenance
* pulp.server.maintenance.monthly.queue_monthly_maintenance
* pulp.server.managers.auth.permission.cud.grant
* pulp.server.managers.auth.permission.cud.revoke
* pulp.server.managers.auth.role.cud.add_permissions_to_role
* pulp.server.managers.auth.role.cud.add_user_to_role
* pulp.server.managers.auth.role.cud.create_role
* pulp.server.managers.auth.role.cud.delete_role
* pulp.server.managers.auth.role.cud.remove_permissions_from_role
* pulp.server.managers.auth.role.cud.remove_user_from_role
* pulp.server.managers.auth.role.cud.update_role
* pulp.server.managers.consumer.agent.delete_queue
* pulp.server.managers.consumer.applicability.batch_regenerate_applicability
* pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers
* pulp.server.managers.consumer.applicability.regenerate_applicability_for_repos
* pulp.server.managers.consumer.bind.bind
* pulp.server.managers.consumer.bind.delete
* pulp.server.managers.consumer.bind.unbind
* pulp.server.managers.consumer.cud.register
* pulp.server.managers.consumer.cud.unregister
* pulp.server.managers.consumer.cud.update
* pulp.server.managers.consumer.group.cud.associate
* pulp.server.managers.consumer.group.cud.bind
* pulp.server.managers.consumer.group.cud.create_consumer_group
* pulp.server.managers.consumer.group.cud.delete_consumer_group
* pulp.server.managers.consumer.group.cud.unassociate
* pulp.server.managers.consumer.group.cud.unbind
* pulp.server.managers.consumer.group.cud.update_consumer_group
* pulp.server.managers.consumer.profile.create
* pulp.server.managers.consumer.profile.delete
* pulp.server.managers.consumer.profile.update
* pulp.server.managers.content.orphan.delete_all_orphans
* pulp.server.managers.content.orphan.delete_orphans_by_id
* pulp.server.managers.content.orphan.delete_orphans_by_type
* pulp.server.managers.content.upload.import_uploaded_unit
* pulp.server.managers.repo.group.cud.associate
* pulp.server.managers.repo.group.cud.create_and_configure_repo_group
* pulp.server.managers.repo.group.cud.delete_repo_group
* pulp.server.managers.repo.group.cud.unassociate
* pulp.server.managers.repo.group.cud.update_repo_group
* pulp.server.managers.repo.group.distributor.add_distributor
* pulp.server.managers.repo.group.distributor.remove_distributor
* pulp.server.managers.repo.group.distributor.update_distributor_config
* pulp.server.managers.repo.group.publish.publish
* pulp.server.managers.repo.importer.remove_importer
* pulp.server.managers.repo.importer.set_importer
* pulp.server.managers.repo.importer.update_importer_config
* pulp.server.managers.repo.publish.publish
* pulp.server.managers.repo.sync.sync
* pulp.server.managers.repo.unit_association.associate_from_repo
* pulp.server.managers.repo.unit_association.unassociate_by_criteria
* pulp.server.tasks.consumer.install_content
* pulp.server.tasks.consumer.uninstall_content
* pulp.server.tasks.consumer.update_content
* pulp.server.tasks.content.refresh_content_source
* pulp.server.tasks.content.refresh_content_sources
* pulp.server.tasks.repository.delete
* pulp.server.tasks.repository.distributor_delete
* pulp.server.tasks.repository.distributor_update
* pulp.server.tasks.repository.sync_with_auto_publish
Updated by mhrivnak over 7 years ago
I think these can be ignored, and not ported to pulp 3:
- auth tasks
- consumer tasks that aren't related to applicability
- repo group tasks
- Any task that was only used by the scheduler to queue other user-facing tasks. Example: queue_publish (also, I'm not sure why "queue_sync_with_auto_publish" is missing from this list)
Updated by bmbouter over 7 years ago
Thanks mhrivnak. Yes all of those things should not be ported.
Also, these 4 are already ported so they can be ignored.
* pulp.server.async.tasks.PulpTask
* pulp.server.async.tasks.Task
* pulp.server.async.tasks._queue_reserved_task
* pulp.server.async.tasks._release_resource
Updated by ttereshc over 7 years ago
mhrivnak wrote:
- Any task that was only used by the scheduler to queue other user-facing tasks. Example: queue_publish (also, I'm not sure why "queue_sync_with_auto_publish" is missing from this list)
It is there, the name is without "queue_" for some reason, just sync_with_auto_publish
Updated by ttereshc over 7 years ago
This comment is just to track what celery tasks have been covered so far. Feel free to edit it accordingly.
- task #2154
pulp.server.async.tasks.PulpTask - task #2154
pulp.server.async.tasks.Task - task #2154
pulp.server.async.tasks._queue_reserved_task - task #2154
pulp.server.async.tasks._release_resource - task #2386
pulp.server.controllers.repository.download_deferred - task #2388
pulp.server.controllers.repository.download_repo - not to be ported
pulp.server.controllers.repository.queue_download_deferred - not to be ported
pulp.server.controllers.repository.queue_publish - not to be ported
pulp.server.db.reaper.queue_reap_expired_documents - task #2392
pulp.server.db.reaper.reap_expired_documents - task #2393
pulp.server.maintenance.monthly.monthly_maintenance - not to be ported
pulp.server.maintenance.monthly.queue_monthly_maintenance - not to be ported
pulp.server.managers.auth.permission.cud.grant - not to be ported
pulp.server.managers.auth.permission.cud.revoke - not to be ported
pulp.server.managers.auth.role.cud.add_permissions_to_role - not to be ported
pulp.server.managers.auth.role.cud.add_user_to_role - not to be ported
pulp.server.managers.auth.role.cud.create_role - not to be ported
pulp.server.managers.auth.role.cud.delete_role - not to be ported
pulp.server.managers.auth.role.cud.remove_permissions_from_role - not to be ported
pulp.server.managers.auth.role.cud.remove_user_from_role - not to be ported
pulp.server.managers.auth.role.cud.update_role - not to be ported
pulp.server.managers.consumer.agent.delete_queue - task #2389
pulp.server.managers.consumer.applicability.batch_regenerate_applicability - task #2390
pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers - task #2391
pulp.server.managers.consumer.applicability.regenerate_applicability_for_repos - not to be ported
pulp.server.managers.consumer.bind.bind - not to be ported
pulp.server.managers.consumer.bind.delete - not to be ported
pulp.server.managers.consumer.bind.unbind - not to be ported
pulp.server.managers.consumer.cud.register - not to be ported
pulp.server.managers.consumer.cud.unregister - not to be ported
pulp.server.managers.consumer.cud.update - not to be ported
pulp.server.managers.consumer.group.cud.associate - not to be ported
pulp.server.managers.consumer.group.cud.bind - not to be ported
pulp.server.managers.consumer.group.cud.create_consumer_group - not to be ported
pulp.server.managers.consumer.group.cud.delete_consumer_group - not to be ported
pulp.server.managers.consumer.group.cud.unassociate - not to be ported
pulp.server.managers.consumer.group.cud.unbind - not to be ported
pulp.server.managers.consumer.group.cud.update_consumer_group - not to be ported
pulp.server.managers.consumer.profile.create - not to be ported
pulp.server.managers.consumer.profile.delete - not to be ported
pulp.server.managers.consumer.profile.update - task #2394
pulp.server.managers.content.orphan.delete_all_orphans - task #2395
pulp.server.managers.content.orphan.delete_orphans_by_id - task #2396
pulp.server.managers.content.orphan.delete_orphans_by_type - task #2397
pulp.server.managers.content.upload.import_uploaded_unit - not to be ported
pulp.server.managers.repo.group.cud.associate - not to be ported
pulp.server.managers.repo.group.cud.create_and_configure_repo_group - not to be ported
pulp.server.managers.repo.group.cud.delete_repo_group - not to be ported
pulp.server.managers.repo.group.cud.unassociate - not to be ported
pulp.server.managers.repo.group.cud.update_repo_group - not to be ported
pulp.server.managers.repo.group.distributor.add_distributor - not to be ported
pulp.server.managers.repo.group.distributor.remove_distributor - not to be ported
pulp.server.managers.repo.group.distributor.update_distributor_config - not to be ported
pulp.server.managers.repo.group.publish.publish - task #2400
pulp.server.managers.repo.importer.remove_importer - task #2401
pulp.server.managers.repo.importer.set_importer - task #2402
pulp.server.managers.repo.importer.update_importer_config - task #2398
pulp.server.managers.repo.publish.publish - task #2399
pulp.server.managers.repo.sync.sync - task #2403
pulp.server.managers.repo.unit_association.associate_from_repo - task #2404
pulp.server.managers.repo.unit_association.unassociate_by_criteria - not to be ported
pulp.server.tasks.consumer.install_content - not to be ported
pulp.server.tasks.consumer.uninstall_content - not to be ported
pulp.server.tasks.consumer.update_content - task #2405
pulp.server.tasks.content.refresh_content_source - task #2406
pulp.server.tasks.content.refresh_content_sources - task #2407
pulp.server.tasks.repository.delete - task #2408
pulp.server.tasks.repository.distributor_delete - task #2409
pulp.server.tasks.repository.distributor_update - not to be ported
pulp.server.tasks.repository.sync_with_auto_publish
Updated by ttereshc over 7 years ago
- Related to Refactor #2154: Convert task system to use django models added
Updated by ttereshc over 7 years ago
- Related to Task #2386: Convert celery task repository.download_deferred to Pulp 3 added
Updated by mhrivnak over 7 years ago
All non-applicability consumer actions do not need porting. They can be done just through the REST API. They are tasks in pulp 2 because they often include performing actions on a consumer via gofer, which can take a while. Perhaps we were also doing resource locking for some, but that is definitely not needed with pulp 3.
Updated by ttereshc over 7 years ago
- Related to Task #2388: Convert celery task repository.download_repo to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2389: Convert celery task applicability.batch_regenerate_applicability to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2390: Convert celery task applicability.regenerate_applicability_for_consumers to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2391: Convert celery task applicability.regenerate_applicability_for_repos to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2392: Convert celery task db.reaper.reap_expired_documents to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2393: Convert celery task monthly_maintenance to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2394: Convert celery task content.orphan.delete_all_orphans to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2395: Convert celery task content.orphan.delete_orphans_by_id to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Story #2396: As a user, I can remove orphaned content and related artifacts by content type added
Updated by ttereshc over 7 years ago
- Related to Task #2397: Convert celery task content.upload.import_uploaded_unit to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2398: Convert celery task repo.publish to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2399: Convert celery task repo.sync to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2400: Convert celery task repo.importer.remove_importer to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2401: Convert celery task repo.importer.set_importer to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Story #2402: As an API user, a call to update an Importer generates a Task added
Updated by ttereshc over 7 years ago
- Related to Task #2403: Convert celery task repo.unit_association.associate_from_repo to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2404: Convert celery task repo.unit_association.unassociate_by_criteria to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2405: Convert celery task content.refresh_content_source to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2406: Convert celery task content.refresh_content_sourceS to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2407: Convert celery task repository.delete to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Task #2408: Convert celery task repository.distributor_delete to Pulp 3 added
Updated by ttereshc over 7 years ago
- Related to Story #2409: As an API user, a call to update an Publisher generates a Task added
Updated by ttereshc over 7 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to ttereshc
- Sprint/Milestone set to 28
Someone, please, take a look at the comment 5 and check that I have not ruled out any celery task that we potentially need in Pulp 3.
If you think that I created a task which is not needed, just comment on it and close it.
Updated by bmbouter over 7 years ago
Can all of the related issues have the 'Pulp 3' tag added. Other then that I think this issue can be closed and we can start commenting on the others.
Updated by ttereshc over 7 years ago
- Status changed from ASSIGNED to CLOSED - CURRENTRELEASE
Thanks, I've added the tag.
Actions
.