Using special name to determine RQ worker role can be problematic.
Parsing a special name to determine the role of the PulpWorker can be error prone and lead to failure scenarios that can be difficult to troubleshoot. I ran into this nearly silent failure with the celery code as well and burned a bit of time figuring it out.
rq worker -n 'reserved_resource_worker_1@%h' -w 'pulpcore.tasking.worker.PulpWorker'
Suggest a cleaner, more explicit approach is to create have 2 concrete classes of workers that can determine the queues which are passed to super(). And, perhaps determine its name?
rq worker -w 'pulpcore.tasking.worker.ReservedWorker' rq worker -w 'pulpcore.tasking.worker.ResourceManager'
Would -n <name> still be needed?
Simpler and if the user get's the classpath to the worker wrong, the logged exception will be self explanatory.
#2 Updated by bmbouter over 1 year ago
How will the class know to name the worker with a _1 versus a _2 when you start multiple worker instances? It's not required that they have numbers in them necessarily, but they do need to be unique.
One option to adopt this change would be for us to start randomizing the names something like firstname.lastname@example.org or similar. That would make running on things like kubernetes probably easier since you can use the same command to start a worker and it will receive a unique name.
Please register to edit this issue