As a plugin author, I have documentation on how to implement a sync operation for my importer
The syncing with an importer portion of the plugin writer's guide needs to be updated. The guide needs to explain that it's the author's responsibility to
- define a UserFacingTask that creates a new RepositoryVersion of a repository by syncing with a specific importer.
- the task should create a repository version and add it to `task.created_resources` in a single transaction
- the task should add / remove content using the Importer to drive the download API
- the task should set the repository version's to complete field to true after all content manipulation is done
- define a detail route on the Importer ViewSet for 'sync' that accepts a 'repository' POST body parameter
- the route should perform validation and then dispatch the the sync task with reservations for the repository and the importer.
#7 Updated by daviddavis about 3 years ago
I talked a bit with @dkliban and it seems like the order should be sync, add, then remove. For instance, suppose I have a version that was synced that contains units A and B. If I want to sync again to get unit C but also remove unit B, then I can't actually remove B (because sync will just re-add it). I thinks users would be surprised if they called sync and remove B, only to find that B was still there because it was removed before sync and then sync re-added it.
#10 Updated by daviddavis about 3 years ago
- Subject changed from As an authenticated user, I can create a new repository version. to As an authenticated user, I can create a new repository version by POSTing a importer href.
- Description updated (diff)
- Status changed from ASSIGNED to NEW
- Assignee deleted (
#16 Updated by ttereshc about 3 years ago
I would move
- define a detail route on the Importer ViewSet for 'sync' that accepts a 'repository' POST body parameter one level up or it seems like it's a part of requirements for the sync task.
Resource reservation should probably also be on a repo version, what do you think?
s/task's created list/
#17 Updated by firstname.lastname@example.org about 3 years ago
If we are going to lock on the repository version being created, then the repository version needs to be created in the viewset. A lock on the repository should prevent a repository version from being deleted from under it. Please correct me if my understanding of the locking is wrong.
#18 Updated by bmbouter about 3 years ago
To align w/ the locking that core would do as part of #3186. We need to tell them to lock on:
- The repo version
- The repository itself
- The importer being used
The lock on the repository version is needed in addition to serialize any concurrent publishes that may also try to publish on that version. If that case is prevented due to the a publication not being allowed while complete=False then the lock would not be required.
#19 Updated by email@example.com about 3 years ago
We should definitely prevent publishing a repository version with complete=False. We would also need to lock on repository when deleting a repository version. Based on info in https://pulp.plan.io/issues/3219 we are planning to do that. With this in mind, I don't think we need to lock on a repository version during a sync.
#22 Updated by ttereshc about 3 years ago
So how publish task will behave if it sees incomplete repo version?
Without lock on a repo version the publish task will run and fail? Or it won't be possible to create a publish task for incomplete repo version?
If sync task will lock on repo version then a publish task can be created and wait in a queue.
Please register to edit this issue