Project

Profile

Help

Story #3186

Updated by bmbouter over 2 years ago

h2. Motivation

Versioned repositories will have significant impact on the locking system. Due to the bookkeeping approach, anytime you modify content or a RepositoryVersion, you need to serialize those operations with other content manipulations on that repo.

Also, the decision to not associate an importer directly with a repo means that we need to lock differently when updating importer settings.

Finally, publication manipulation also needs to lock on repository versions.

h2. Design

The existing tasking system prevents two tasks from running at the same time that have the same lock label. For correctness, I think we will need to introduce multi-resource locking, which is the ability to have one task require lock N resources. This is easily added to the tasking system. The resource locked and in what situation are listed here:

* an update/delete importer task locks on the importer instance
* an update/delete publisher task locks on the publisher instance

* a sync task locks on the importer instance
* a sync task locks on the repository

* a publish task locks on the publisher instance
* a publish task locks on the repository versioned repo

* a delete versioned repo task locks on the versioned repo
* a delete versioned repo task locks on the repository

h2. A three task example

Say a user updates an importer with a PUT to /api/v3/importers/file-importer/55. That would cause the update task to lock on just that importer ["/api/v3/importers/file-importer/55"]

Then a user will POST to /api/v3/repository/foo/repo_version/ with {"importer": "/api/v3/importers/file-importer/55"}

That task would multi-lock on the importer, the repo version, and the repository : ["/api/v3/importers/file-importer/55", "/api/v3/repository/foo/repo_version/72", "/api/v3/repository/foo/"] <---- Note that the viewset has to create the RepoVersion so it can know the detail view to lock on

Then a user will DELETE /api/v3/repository/foo/repo_version/72 which will cause a lock of the repo version and the repository itself ["/api/v3/repository/foo/repo_version/72", "/api/v3/repository/foo/"] to be used.

h4. Example Run Order

Even on a Pulp system with 3+ workers, the above 3 tasks will be executed fully serially because each one requires a resource locked by the one previous.

Back