Pulp

1. Run "pstart"
2. From the same machine, run "celery beat --app=pulp.server.async.celery_instance.celery --scheduler=pulp.server.async.scheduler.Scheduler"
3. From a new terminal window, create a scheduled sync using the command "pulp-admin rpm repo sync schedules create --repo-id zoo -s 2016-06-02T14:49:30+00:00", BUT modify the date and time to be in the near future of your present time. You may also use a different repo if desired.
4. Wait until some time after the time used in step 3
5. Execute the command "pulp-admin tasks list --all"
6. Observe that the sync and publish tasks were executed twice, instead of once

Background¶

The scheduler uses the following logic to prevent multiple instances from actively dispatching tasks
https://github.com/pulp/pulp/blob/a63536ac2384626f58939e3a41d7f2f31bd50750/server/pulp/server/async/scheduler.py#L245

celerybeat_name = constants.SCHEDULER_WORKER_NAME + "@" + platform.node()

....

# Updating the current lock if lock is on this instance of celerybeat
result = CeleryBeatLock.objects(celerybeat_name=celerybeat_name).\
    update(set__timestamp=datetime.utcnow())

# If current instance has lock and updated lock_timestamp, call super
if result == 1:
    _logger.debug(_('Lock updated by %(celerybeat_name)s')
                  % {'celerybeat_name': celerybeat_name})
    ret = self.call_tick(self, celerybeat_name)

If two instances of pulp_celerybeat share the same name (ie. when they exist on the same platform), each will be able to successfully update the same celerybeat_lock object without violating the uniqueness constraint. Both pulp_celerybeat instances expect that they have acquired the lock, and commence dispatching tasks.

The correct behavior here would be to ensure that regardless of how many are running, only one scheduler is able to dispatch tasks.

dkliban pointed out that users shouldn't be running 2 schedulers on the same machine anyway, but that it's very likely that some users will do so regardless, hence this bug should be filed.