Issue #922
closed
If a sync, publish, or consumer schedule is updated to change the time it runs, celerybeat does not run it at the new start time
Description
It seems I cannot update an existing sync, publish, or consumer action schedule. For example, I had an existing schedule, and I wanted to change it to run next in about four minutes (so that it's more than 90 seconds out, giving the scheduler enough time to notice the change). Here is me listing the schedule, and updating it to a new value:
[vagrant@cloud-basebox pulp]$ pulp-admin rpm repo sync schedules list --repo-id zoo
+----------------------------------------------------------------------+
Schedules
+----------------------------------------------------------------------+
Schedule: 2015-04-27T21:24:00Z/P1D
Id: 553be0de7517d40591882021
Enabled: True
Next Run: 2015-04-28T18:48:00Z
[vagrant@cloud-basebox pulp]$ pulp-admin rpm repo sync schedules update --repo-id zoo --schedule-id 553be0de7517d40591882021 --schedule 2015-04-27T21:32:00Z/P1D
Successfully updated schedule
[vagrant@cloud-basebox pulp]$ date -u
Mon Apr 27 21:28:44 UTC 2015
The Next Run continues to show 2015-04-28T18:48:00Z well past the new schedule start time, and according to the logs the new schedule never fired (I didn't see a sync happen in the log.)
A workaround is to delete the existing schedule and create a new schedule with the desired start time.
Updated by rbarlow almost 9 years ago
- Description updated (diff)
Here is some more information I gathered about this. I updated the schedule again to be a few minutes in the future:
$ pulp-admin rpm repo sync schedules update --repo-id zoo --schedule-id 553be0de7517d40591882021 --schedule 2015-04-27T22:20:00Z/P1D
Successfully updated schedule
Then I configured celerybeat to run in Debug logging, and saw this:
Apr 27 22:17:28 cloud-basebox.localdomain pulp[6978]: root:DEBUG: one or more enabled schedules has been updated
Apr 27 22:17:28 cloud-basebox.localdomain pulp[6978]: pulp.server.async.scheduler:DEBUG: loading schedules from app
Apr 27 22:17:28 cloud-basebox.localdomain pulp[6978]: pulp.server.async.scheduler:DEBUG: loading schedules from DB
Apr 27 22:17:28 cloud-basebox.localdomain pulp[6978]: pulp.server.async.scheduler:DEBUG: loaded 2 schedules
Apr 27 22:17:28 cloud-basebox.localdomain pulp[6978]: pulp.server.db.model.dispatch:DEBUG: not running task 553eaee47517d419c494b574: 84931 seconds remaining
Apr 27 22:17:28 cloud-basebox.localdomain pulp[6978]: pulp.server.db.model.dispatch:DEBUG: not running task 553be0de7517d40591882021: 73831 seconds remaining
Apr 27 22:17:28 cloud-basebox.localdomain pulp[6978]: pulp.server.async.worker_watcher:DEBUG: received 'scheduler-event' from scheduler@cloud-basebox.localdomain at time: 2015-04-27 22:17:28.973679
Apr 27 22:17:28 cloud-basebox.localdomain pulp[6978]: celery.beat:DEBUG: beat: Waking up in 1.50 minutes.
Note that the scheduler saw the change at 22:17, which was before the next run at 22:20. However, note that it still thinks the next run for that schedule is 73,831 seconds away rather than ~2.5 minutes.
Updated by rbarlow almost 9 years ago
On 04/28/2015 10:48 AM, bmbouter wrote:
Is it possible this is related to #516 <https://pulp.plan.io/issues/516>?
It is possible, though I'm not certain at this point. I didn't try to
adjust the frequency, but instead I tried to update the start time. That
bug seems to be about adjusting the frequency, so it may or may not be
the same. It's worth looking into, thanks for the pointer!
--
Randy Barlow
Updated by mhrivnak almost 9 years ago
- Priority changed from Normal to High
- Severity changed from 2. Medium to 3. High
- Version set to Master
- Triaged changed from No to Yes
Investigate whether this affects 2.6.
Updated by bmbouter almost 9 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to bmbouter
Added by bmbouter almost 9 years ago
Added by bmbouter almost 9 years ago
Revision 472b84cb | View on GitHub
Updates to schedules are now respected by celerybeat
The schedule field was being updated, but next_run and first_run were not being updated. These values are used to determine if a schedued task should be run so updating them is necessary.
Updated by bmbouter almost 9 years ago
- Subject changed from If a sync schedule is updated to change the time it runs, the scheduler does not run it at the new start time to If a sync, publish, or consumer schedules are updated to change the time it runs, the celerybeat does not run it at the new start time
- Description updated (diff)
QE to verify this, please do the following.
0) Install a fully working version of Pulp and start it up
1) Create a zoo repo
2) Create a sync schedule for zoo that will sync very far in the future (ie: a date in 2016)
3) List that sync and verify it got created
4) Update the sync with pulp-admin to have it run a few minutes in the future (at least 120 seconds from the time you issue the command)
5) Verify that the sync runs at the new time you specified
6) Do the same steps 2-5 for a publish schedule and verify that
7) Install a consumer
8) Do steps 2-5 only with a consumer action and verify that type of schedule also updates.
Updated by bmbouter almost 9 years ago
- Status changed from ASSIGNED to POST
PR available at: https://github.com/pulp/pulp/pull/1861
Updated by bmbouter almost 9 years ago
- Subject changed from If a sync, publish, or consumer schedules are updated to change the time it runs, the celerybeat does not run it at the new start time to If a sync, publish, or consumer schedule is updated to change the time it runs, celerybeat does not run it at the new start time
Updated by bmbouter almost 9 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulp|472b84cb765490752a0aef21147abecee7830b23.
Updated by dkliban@redhat.com almost 9 years ago
- Status changed from MODIFIED to 5
Updated by pthomas@redhat.com almost 9 years ago
- Status changed from 5 to 6
verified
[root@mgmt12 ~]# rpm -qa pulp-server
pulp-server-2.6.2-0.2.beta.el6.noarch
[root@mgmt12 ~]#
1. verified sync update works as expected
2. also verified consumer install schedule work
[root@mgmt12 ~]# pulp-admin rpm consumer package install schedules create --consumer-id patch --schedule 2016-04-27T22:20:00Z/P1D -n dog
Schedule successfully created
[root@mgmt12 ~]#
[root@mgmt12 ~]#
[root@mgmt12 ~]# pulp-admin rpm consumer package install schedules list --consumer-id patch
+----------------------------------------------------------------------+
Schedules
+----------------------------------------------------------------------+
Schedule: 2015-05-20T15:05:00Z/P1D
Id: 555c9d62e8ec9860dcbdcadb
Enabled: True
Next Run: 2015-05-21T15:05:00Z
Schedule: 2016-04-27T22:20:00Z/P1D
Id: 555ca3b3e8ec9860dbfe47fa
Enabled: True
Next Run: 2016-04-27T22:20:00Z
[root@mgmt12 ~]# pulp-admin rpm consumer package install schedules update --consumer-id patch --schedule-id 555ca3b3e8ec9860dbfe47fa -s 2015-05-20T15:15:00Z/P1D
Successfully updated schedule
[root@mgmt12 ~]# pulp-admin rpm consumer package install schedules list --consumer-id patch
+----------------------------------------------------------------------+
Schedules
+----------------------------------------------------------------------+
Schedule: 2015-05-20T15:05:00Z/P1D
Id: 555c9d62e8ec9860dcbdcadb
Enabled: True
Next Run: 2015-05-21T15:05:00Z
Schedule: 2015-05-20T15:15:00Z/P1D
Id: 555ca3b3e8ec9860dbfe47fa
Enabled: True
Next Run: 2015-05-20T15:15:00Z
<\pre>
Updated by dkliban@redhat.com over 8 years ago
- Status changed from 6 to CLOSED - CURRENTRELEASE
Added by ipanova@redhat.com over 8 years ago
Revision 7fd27725 | View on GitHub
Adding saslwrapper to the external deps list.
Added by ipanova@redhat.com over 8 years ago
Revision 7fd27725 | View on GitHub
Adding saslwrapper to the external deps list.
.
Updates to schedules are now respected by celerybeat
The schedule field was being updated, but next_run and first_run were not being updated. These values are used to determine if a schedued task should be run so updating them is necessary.
fixes #922 https://pulp.plan.io/issues/922