Story #3173
closedAs a plugin author, I have documentation on how to implement a sync operation for my importer
100%
Description
The syncing with an importer portion of the plugin writer's guide needs to be updated. The guide needs to explain that it's the author's responsibility to
- define a UserFacingTask that creates a new RepositoryVersion of a repository by syncing with a specific importer.
- the task should create a repository version and add it to `task.created_resources` in a single transaction
- the task should add / remove content using the Importer to drive the download API
- the task should set the repository version's to complete field to true after all content manipulation is done
- define a detail route on the Importer ViewSet for 'sync' that accepts a 'repository' POST body parameter
- the route should perform validation and then dispatch the the sync task with reservations for the repository and the importer.
Related issues
Updated by dkliban@redhat.com about 7 years ago
- Has duplicate Issue #3228: As an authenticated user, I can create a new repository version added
Updated by daviddavis about 7 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to daviddavis
Updated by daviddavis about 7 years ago
A couple items:
- add_many and remove_many seem vague. I was thinking about something like add_content_units and remove_content_units.
- Is the order of the actions (add, remove, then sync) correct? I thought we discussed sync, add, remove in the past.
Updated by daviddavis about 7 years ago
- Blocks Story #3234: As an authenticated user, I can create a new version by adding or removing content to the latest version. added
Updated by daviddavis about 7 years ago
I talked a bit with @dkliban and it seems like the order should be sync, add, then remove. For instance, suppose I have a version that was synced that contains units A and B. If I want to sync again to get unit C but also remove unit B, then I can't actually remove B (because sync will just re-add it). I thinks users would be surprised if they called sync and remove B, only to find that B was still there because it was removed before sync and then sync re-added it.
Updated by bmbouter almost 7 years ago
daviddavis@redhat.com wrote:
I talked a bit with @dkliban and it seems like the order should be sync, add, then remove.
This makes sense to me. +1 to doing it in this way.
Updated by daviddavis almost 7 years ago
- Subject changed from As an authenticated user, I can create a new repository version. to As an authenticated user, I can create a new repository version by POSTing a importer href.
- Description updated (diff)
- Status changed from ASSIGNED to NEW
- Assignee deleted (
daviddavis)
Updated by dkliban@redhat.com almost 7 years ago
- Groomed changed from No to Yes
- Sprint Candidate changed from No to Yes
Updated by dkliban@redhat.com almost 7 years ago
- Blocks Task #3220: Expand plugin writer's guide to include RepositoryVersion creating actions (including sync) added
Updated by amacdona@redhat.com almost 7 years ago
- Blocks deleted (Story #3234: As an authenticated user, I can create a new version by adding or removing content to the latest version.)
Updated by dkliban@redhat.com almost 7 years ago
- Description updated (diff)
- Tags Documentation added
Updated by dkliban@redhat.com almost 7 years ago
- Subject changed from As an authenticated user, I can create a new repository version by POSTing a importer href. to As a plugin author, I have documentation on how to implement a sync operation for my importer
Updated by ttereshc almost 7 years ago
I would move - define a detail route on the Importer ViewSet for 'sync' that accepts a 'repository' POST body parameter
one level up or it seems like it's a part of requirements for the sync task.
Resource reservation should probably also be on a repo version, what do you think?
s/task's created list/task.created_resources
Updated by dkliban@redhat.com almost 7 years ago
If we are going to lock on the repository version being created, then the repository version needs to be created in the viewset. A lock on the repository should prevent a repository version from being deleted from under it. Please correct me if my understanding of the locking is wrong.
Updated by bmbouter almost 7 years ago
To align w/ the locking that core would do as part of #3186. We need to tell them to lock on:
- The repo version
- The repository itself
- The importer being used
The lock on the repository version is needed in addition to serialize any concurrent publishes that may also try to publish on that version. If that case is prevented due to the a publication not being allowed while complete=False then the lock would not be required.
Updated by dkliban@redhat.com almost 7 years ago
We should definitely prevent publishing a repository version with complete=False. We would also need to lock on repository when deleting a repository version. Based on info in https://pulp.plan.io/issues/3219 we are planning to do that. With this in mind, I don't think we need to lock on a repository version during a sync.
Updated by amacdona@redhat.com almost 7 years ago
We do still need to lock on repo version. Otherwise, the version can be deleted, or the version before it could be deleted, which would modify the version in question.
Updated by ttereshc almost 7 years ago
So how publish task will behave if it sees incomplete repo version?
Without lock on a repo version the publish task will run and fail? Or it won't be possible to create a publish task for incomplete repo version?
If sync task will lock on repo version then a publish task can be created and wait in a queue.
Updated by dkliban@redhat.com almost 7 years ago
- Blocks deleted (Task #3220: Expand plugin writer's guide to include RepositoryVersion creating actions (including sync))
Updated by amacdona@redhat.com almost 7 years ago
- Is duplicate of Task #3220: Expand plugin writer's guide to include RepositoryVersion creating actions (including sync) added
Updated by amacdona@redhat.com almost 7 years ago
- Groomed changed from Yes to No
Updated by amacdona@redhat.com almost 7 years ago
- Status changed from NEW to CLOSED - DUPLICATE
Updated by amacdona@redhat.com almost 7 years ago
- % Done changed from 0 to 100
- Parent issue deleted (
#3209)