Project

Profile

Help

Task #3308

Skip creating new repo versions when content has not changed

Added by kersom about 2 years ago. Updated about 2 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Platform Release:
Blocks Release:
Backwards Incompatible:
No
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

Per our discussion on pulp-dev0, we should stop creating new repo versions if content has not changed. To recap the reasons:

- They were meant to serve as a historical record but this information is available by looking at the tasks api
- It creates additional, unnecessary versions and bumps the latest version number of the repo
- If we ever have a feature to retain only the latest X repo versions, it'll be less useful since some repo versions may not have any changes

I believe sync is the only place that needs to be updated but we may also need to consider any other places in the code.

Also, we should document that this is the intended behavior.

[0] https://www.redhat.com/archives/pulp-dev/2019-November/msg00003.html


Related issues

Related to Ansible Plugin - Issue #4920: Collection - Repository versions not being update after successive syncs MODIFIED Actions

Associated revisions

Revision 926079bd View on GitHub
Added by dkliban@redhat.com 3 months ago

Update tests to not rely on version always incrementing.

Required PR: https://github.com/pulp/pulpcore/pull/380

re: #3308
https://pulp.plan.io/issues/3308

Revision 25d0d206 View on GitHub
Added by dkliban@redhat.com 3 months ago

Update tests to not rely on version always incrementing

Required PR: https://github.com/pulp/pulpcore/pull/380

re: #3308
https://pulp.plan.io/issues/3308

Revision 3554b310 View on GitHub
Added by dkliban@redhat.com 3 months ago

Don't create a new repository version if no content was added or removed.

closes: #3308
https://pulp.plan.io/issues/3308

Revision a1c3249b View on GitHub
Added by dkliban@redhat.com 3 months ago

Update tests to not rely on repo version number always incrementing

Required PR: https://github.com/pulp/pulp_file/pull/311

re: #3308
https://pulp.plan.io/issues/3308

Revision 322962da View on GitHub
Added by ipanova@redhat.com 3 months ago

Update tests to not rely on version always incrementing.

closes #3308

Revision 322962da View on GitHub
Added by ipanova@redhat.com 3 months ago

Update tests to not rely on version always incrementing.

closes #3308

History

#1 Updated by kersom about 2 years ago

  • Description updated (diff)

#2 Updated by amacdona@redhat.com about 2 years ago

  • Status changed from NEW to CLOSED - WONTFIX

This is correct behavior. A new RepositoryVersion needs to be created, even if the sync does not add or remove any content units.

The reasoning for this is made clear by a worker crash scenario. When a sync is started, a new RepositoryVersion is created, (number= 2) and Repository.last_version = 2. Before the sync is completed, the worker crashes. While the crash is waiting to be cleaned up, a second sync task is run, which creates another new RepositoryVersion. The second sync task creates a new RepositoryVersion(number=3). If the crash/failure cleanup decrements Repository.last_version, then Repository.last_version will be incorrect, leading to duplicate key errors.

All of this complexity goes away if we have "empty" RepositoryVersions, which can be deleted by the user.

#3 Updated by daviddavis 9 months ago

  • Sprint/Milestone set to 3.0.0

#4 Updated by bmbouter 9 months ago

  • Tags deleted (Pulp 3)

#5 Updated by daviddavis 8 months ago

  • Status changed from CLOSED - WONTFIX to NEW

In pulp_ansible, we couldn't use the Stages code and we made the decision to NOT create a repo version when content doesn't change. This of course is not consistent with other plugins. I am reopening this issue to rediscuss this and also to make sure we have proper docs around the correct behavior for plugin writers.

#6 Updated by kersom 8 months ago

  • Related to Issue #4920: Collection - Repository versions not being update after successive syncs added

#7 Updated by bmbouter 8 months ago

  • Subject changed from Sync of unchanged importer update repo version to Document that a repo version is expected to be created for all sync's regardles of if there is new content remotely or now
  • Description updated (diff)
  • Sprint set to Sprint 54
  • Tags Documentation added

After discussion during today's open floor we decided:

1) the current behavior is a good behavior. It's less surprising to users when they ask Pulp to sync and receive a repo version N+1 that is that same as repo version N, and more confusing if they get some repo verisons for some sync's and not for others depending on the remote state.

2) We should document this expectation in the plugin write API, so now this is a docs story

3) it should go on the sprint

#8 Updated by amacdona@redhat.com 8 months ago

  • Tracker changed from Issue to Story
  • % Done set to 0

#9 Updated by ttereshc 7 months ago

  • Sprint changed from Sprint 54 to Sprint 55

#10 Updated by dkliban@redhat.com 7 months ago

  • Sprint changed from Sprint 55 to Sprint 56

#11 Updated by rchan 6 months ago

  • Sprint changed from Sprint 56 to Sprint 57

#12 Updated by rchan 5 months ago

  • Sprint changed from Sprint 57 to Sprint 58

#13 Updated by bmbouter 5 months ago

  • Sprint/Milestone deleted (3.0.0)

#14 Updated by rchan 4 months ago

  • Sprint deleted (Sprint 58)

#15 Updated by daviddavis 3 months ago

  • Sprint/Milestone set to 3.0.0

Making this a GA blocker. It keeps coming up and this should only take 5 min to fix.

#16 Updated by daviddavis 3 months ago

  • Tracker changed from Story to Task
  • Subject changed from Document that a repo version is expected to be created for all sync's regardles of if there is new content remotely or now to Skip creating new repo versions when content has not changed
  • Description updated (diff)
  • Tags deleted (Documentation)

#17 Updated by dkliban@redhat.com 3 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dkliban@redhat.com

#18 Updated by dalley 3 months ago

  • Status changed from ASSIGNED to POST

#19 Updated by dkliban@redhat.com 3 months ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#20 Updated by ipanova@redhat.com 3 months ago

  • Status changed from MODIFIED to POST

#21 Updated by ipanova@redhat.com 3 months ago

  • Status changed from POST to MODIFIED

#22 Updated by bmbouter about 2 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF