Project

Profile

Help

Issue #2768

Worker foreign key does not get associated with task in Pulp3

Added by bizhang over 2 years ago. Updated 6 months ago.

Status:
MODIFIED
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Severity:
2. Medium
Version:
Platform Release:
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 21

Description

When a task is kicked off I would expect it to populate the worker foreign key [0] when the task is ran

I would expect this to be updated in the apply_async function when the Task status object is created [1].
Tasks kicked off with apply_async should have the worker field populated upon creation.
Tasks kicked off with apply_async_with_reservations would have the worker field updated when it gets routed to the worker.

For reference pulp2 updates the field here [2]

[0] https://github.com/pulp/pulp/blob/b3ff9a93bba4af1405f57e907944270b3052668a/platform/pulp/app/models/task.py#L158
[1] https://github.com/pulp/pulp/blob/b3ff9a93bba4af1405f57e907944270b3052668a/platform/pulp/tasking/tasks.py#L243
[2] https://github.com/pulp/pulp/blob/master/server/pulp/server/async/tasks.py#L484


Checklist

Associated revisions

Revision b00649ec View on GitHub
Added by dalley over 2 years ago

Associate worker foreign key with task

Ensure the worker field on the Task model gets populated.

closes #2768
https://pulp.plan.io/issues/2768

Revision b00649ec View on GitHub
Added by dalley over 2 years ago

Associate worker foreign key with task

Ensure the worker field on the Task model gets populated.

closes #2768
https://pulp.plan.io/issues/2768

Revision b00649ec View on GitHub
Added by dalley over 2 years ago

Associate worker foreign key with task

Ensure the worker field on the Task model gets populated.

closes #2768
https://pulp.plan.io/issues/2768

History

#1 Updated by bizhang over 2 years ago

  • Subject changed from Worker foreign key does not get associated with task to Worker foreign key does not get associated with task in Pulp3

#2 Updated by bizhang over 2 years ago

  • Description updated (diff)

#3 Updated by ttereshc over 2 years ago

  • Sprint/Milestone set to 39
  • Groomed changed from No to Yes

#4 Updated by bizhang over 2 years ago

  • Description updated (diff)

#5 Updated by ttereshc over 2 years ago

  • Triaged changed from No to Yes
  • Groomed changed from Yes to No

#6 Updated by bmbouter over 2 years ago

  • Sprint/Milestone deleted (39)

I thought about this more, and I think this is the patch we need. We should only set the worker in the _queue_reserve_resource task because that is the earliest moment the worker is known and that is the only time in all of Pulp that a task is dispatched to a dedicated worker.

diff --git a/platform/pulp/tasking/tasks.py b/platform/pulp/tasking/tasks.py
index cd0e21b..41c6890 100644
--- a/platform/pulp/tasking/tasks.py
+++ b/platform/pulp/tasking/tasks.py
@@ -87,6 +87,9 @@ def _queue_reserved_task(name, inner_task_id, resource_id, inner_args, inner_kwa
         time.sleep(0.25)

     task_status = TaskStatus.objects.get(pk=inner_task_id)
+    task_status.worker = worker
+    task_status.save()
+
     ReservedResource.objects.create(task=task_status, worker=worker, resource=resource_id)

     inner_kwargs['routing_key'] = worker.name

#7 Updated by bmbouter over 2 years ago

  • Sprint/Milestone set to 39

#8 Updated by bizhang over 2 years ago

https://pulp.plan.io/issues/2768#note-6 would only associate the worker when the task is called with apply_async_with_reservation. This is an issues that also exists in pulp2

As discussed with @bmbouter the change should be made in https://github.com/pulp/pulp/blob/b3ff9a93bba4af1405f57e907944270b3052668a/platform/pulp/tasking/tasks.py#L255
Since __call__() is called on the worker itself, worker name should be one of the *args or **kwargs

#9 Updated by dalley over 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley

#10 Updated by bizhang over 2 years ago

  • Description updated (diff)

#11 Updated by bizhang over 2 years ago

  • Description updated (diff)

#12 Updated by bizhang over 2 years ago

  • Description updated (diff)

#13 Updated by mhrivnak over 2 years ago

  • Sprint/Milestone changed from 39 to 40

#14 Updated by dalley over 2 years ago

  • Status changed from ASSIGNED to POST

PR is up
https://github.com/pulp/pulp/pull/3054

Commands to test:

from pulpcore.app.tasks import repository
from pulpcore.app.models import Repository
import uuid
repo_uuid=str(uuid.uuid4())
repo=Repository(name=repo_uuid)
repo.save()

repository.delete.apply_async(kwargs={'repo_name':repo_uuid})
repository.delete.apply_async_with_reservation("foo","bar",kwargs={'repo_name':repo_uuid})

#15 Updated by dalley over 2 years ago

  • Status changed from POST to MODIFIED

#16 Updated by bmbouter over 1 year ago

  • Sprint set to Sprint 21

#17 Updated by bmbouter over 1 year ago

  • Sprint/Milestone deleted (40)

#18 Updated by daviddavis 6 months ago

  • Sprint/Milestone set to 3.0

#19 Updated by bmbouter 6 months ago

  • Tags deleted (Pulp 3)

Please register to edit this issue

Also available in: Atom PDF