Task #3308
closed
Skip creating new repo versions when content has not changed
Status:
CLOSED - CURRENTRELEASE
Description
Per our discussion on pulp-dev[0], we should stop creating new repo versions if content has not changed. To recap the reasons:
- They were meant to serve as a historical record but this information is available by looking at the tasks api
- It creates additional, unnecessary versions and bumps the latest version number of the repo
- If we ever have a feature to retain only the latest X repo versions, it'll be less useful since some repo versions may not have any changes
I believe sync is the only place that needs to be updated but we may also need to consider any other places in the code.
Also, we should document that this is the intended behavior.
[0] https://www.redhat.com/archives/pulp-dev/2019-November/msg00003.html
- Description updated (diff)
- Status changed from NEW to CLOSED - WONTFIX
This is correct behavior. A new RepositoryVersion needs to be created, even if the sync does not add or remove any content units.
The reasoning for this is made clear by a worker crash scenario. When a sync is started, a new RepositoryVersion is created, (number= 2) and Repository.last_version = 2. Before the sync is completed, the worker crashes. While the crash is waiting to be cleaned up, a second sync task is run, which creates another new RepositoryVersion. The second sync task creates a new RepositoryVersion(number=3). If the crash/failure cleanup decrements Repository.last_version, then Repository.last_version will be incorrect, leading to duplicate key errors.
All of this complexity goes away if we have "empty" RepositoryVersions, which can be deleted by the user.
- Sprint/Milestone set to 3.0.0
- Status changed from CLOSED - WONTFIX to NEW
In pulp_ansible, we couldn't use the Stages code and we made the decision to NOT create a repo version when content doesn't change. This of course is not consistent with other plugins. I am reopening this issue to rediscuss this and also to make sure we have proper docs around the correct behavior for plugin writers.
- Related to Issue #4920: Collection - Repository versions not being update after successive syncs added
- Subject changed from Sync of unchanged importer update repo version to Document that a repo version is expected to be created for all sync's regardles of if there is new content remotely or now
- Description updated (diff)
- Sprint set to Sprint 54
- Tags Documentation added
After discussion during today's open floor we decided:
1) the current behavior is a good behavior. It's less surprising to users when they ask Pulp to sync and receive a repo version N+1 that is that same as repo version N, and more confusing if they get some repo verisons for some sync's and not for others depending on the remote state.
2) We should document this expectation in the plugin write API, so now this is a docs story
3) it should go on the sprint
- Tracker changed from Issue to Story
- % Done set to 0
- Sprint changed from Sprint 54 to Sprint 55
- Sprint changed from Sprint 55 to Sprint 56
- Sprint changed from Sprint 56 to Sprint 57
- Sprint changed from Sprint 57 to Sprint 58
- Sprint/Milestone deleted (
3.0.0)
- Sprint deleted (
Sprint 58)
- Sprint/Milestone set to 3.0.0
Making this a GA blocker. It keeps coming up and this should only take 5 min to fix.
- Tracker changed from Story to Task
- Subject changed from Document that a repo version is expected to be created for all sync's regardles of if there is new content remotely or now to Skip creating new repo versions when content has not changed
- Description updated (diff)
- Tags deleted (
Documentation)
- Status changed from NEW to ASSIGNED
- Assignee set to dkliban@redhat.com
- Status changed from ASSIGNED to POST
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
- Status changed from MODIFIED to POST
- Status changed from POST to MODIFIED
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Also available in: Atom
PDF
.
Update tests to not rely on version always incrementing.
Required PR: https://github.com/pulp/pulpcore/pull/380
re: #3308 https://pulp.plan.io/issues/3308