Project

Profile

Help

Story #235

closed

[RFE] update pulp_worker-*.service configuration so that starting ensures all resources are initialized

Added by mkovacik@redhat.com about 9 years ago. Updated almost 5 years ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

++ This bug was initially created as a clone of Bugzilla Bug #1104320 ++

Description of problem:

Created attachment 901871
Screen log of immediate repo sync call

Description of problem:
Currently, pulp allows requests even though the pulp_worker-*.services are not
fully operational.
A related scenario is a repo sync request right after a pulp_worker restart, where pulp_database.available queues entries get lost from mongo and stale reservation appears in the pulp_database.reserved_resources collection.

Version-Release number of selected component (if applicable):
- Fedora 20 (systemd)
- pulp-server-2.4.0-0.19.beta.fc20.noarch

How reproducible:
~100% (on my system)

Steps to Reproduce:
1. create a rpm repo (via pulp-admin)
pulp-admin -u admin -p admin rpm repo create --repo-id zoo --feed \
http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
2. systemctl restart pulp_worker-{0..8}.service ; \
pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo

Actual results:
- a task queue lost even though the both the pulp_worker processes are up and running
- stale resource reservation item

[root@ec2-79-125-97-56 ~]# fg
mongo
> db.available_queues.find()
{ "_id" : "reserved_resource_worker-0@ec2-79-125-97-56.eu-west-1.compute.amazonaws.com", "last_heartbeat" : ISODate("2014-06-03T17:07:26.060Z"), "num_reservations" : 0 }
> db.reserved_resources.find()
{ "_id" : "repository:zoo", "assigned_queue" : null, "num_reservations" : 1 }
> db.task_status.find()
> db.available_queues.find()
{ "_id" : "reserved_resource_worker-0@ec2-79-125-97-56.eu-west-1.compute.amazonaws.com", "last_heartbeat" : ISODate("2014-06-03T17:08:08.281Z"), "num_reservations" : 0 }
>

Expected results:
503 Service Unavailable Response until the pulp_worker-*.service prepare all necessary resources
i.e. Pulp server (wsgi) doesn't allow requests while pulp_worker-*.service aren't fully operational

Additional info:

--- Additional comment from at 06/03/2014 18:31:42 ---

Created attachment 901873
journalctl screen log of the scenario

--- Additional comment from at 06/04/2014 12:02:04 ---

Created attachment 902159
reproducer bash script


Files

1019697f889c882c305295c87f8648e2 (171 KB) 1019697f889c882c305295c87f8648e2 journalctl screen log of the scenario mkovacik@redhat.com, 02/19/2015 02:13 AM
d771922ffcf818506c71edb9affb9983 (1.46 KB) d771922ffcf818506c71edb9affb9983 reproducer bash script mkovacik@redhat.com, 02/19/2015 02:13 AM

Also available in: Atom PDF