Project

Profile

Help

Task #3308

Skip creating new repo versions when content has not changed

Added by kersom about 3 years ago. Updated about 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Per our discussion on pulp-dev[0], we should stop creating new repo versions if content has not changed. To recap the reasons:

- They were meant to serve as a historical record but this information is available by looking at the tasks api
- It creates additional, unnecessary versions and bumps the latest version number of the repo
- If we ever have a feature to retain only the latest X repo versions, it'll be less useful since some repo versions may not have any changes

I believe sync is the only place that needs to be updated but we may also need to consider any other places in the code.

Also, we should document that this is the intended behavior.

[0] https://www.redhat.com/archives/pulp-dev/2019-November/msg00003.html


Related issues

Related to Ansible Plugin - Issue #4920: Collection - Repository versions not being update after successive syncsCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 926079bd View on GitHub
Added by dkliban@redhat.com about 1 year ago

Update tests to not rely on version always incrementing.

Required PR: https://github.com/pulp/pulpcore/pull/380

re: #3308 https://pulp.plan.io/issues/3308

Revision 25d0d206 View on GitHub
Added by dkliban@redhat.com about 1 year ago

Update tests to not rely on version always incrementing

Required PR: https://github.com/pulp/pulpcore/pull/380

re: #3308 https://pulp.plan.io/issues/3308

Revision 3554b310 View on GitHub
Added by dkliban@redhat.com about 1 year ago

Don't create a new repository version if no content was added or removed.

closes: #3308 https://pulp.plan.io/issues/3308

Revision a1c3249b View on GitHub
Added by dkliban@redhat.com about 1 year ago

Update tests to not rely on repo version number always incrementing

Required PR: https://github.com/pulp/pulp_file/pull/311

re: #3308 https://pulp.plan.io/issues/3308

Revision 322962da View on GitHub
Added by ipanova@redhat.com about 1 year ago

Update tests to not rely on version always incrementing.

closes #3308

Revision 322962da View on GitHub
Added by ipanova@redhat.com about 1 year ago

Update tests to not rely on version always incrementing.

closes #3308

History

#1 Updated by kersom about 3 years ago

  • Description updated (diff)

#2 Updated by amacdona@redhat.com about 3 years ago

  • Status changed from NEW to CLOSED - WONTFIX

This is correct behavior. A new RepositoryVersion needs to be created, even if the sync does not add or remove any content units.

The reasoning for this is made clear by a worker crash scenario. When a sync is started, a new RepositoryVersion is created, (number= 2) and Repository.last_version = 2. Before the sync is completed, the worker crashes. While the crash is waiting to be cleaned up, a second sync task is run, which creates another new RepositoryVersion. The second sync task creates a new RepositoryVersion(number=3). If the crash/failure cleanup decrements Repository.last_version, then Repository.last_version will be incorrect, leading to duplicate key errors.

All of this complexity goes away if we have "empty" RepositoryVersions, which can be deleted by the user.

#3 Updated by daviddavis almost 2 years ago

  • Sprint/Milestone set to 3.0.0

#4 Updated by bmbouter almost 2 years ago

  • Tags deleted (Pulp 3)

#5 Updated by daviddavis over 1 year ago

  • Status changed from CLOSED - WONTFIX to NEW

In pulp_ansible, we couldn't use the Stages code and we made the decision to NOT create a repo version when content doesn't change. This of course is not consistent with other plugins. I am reopening this issue to rediscuss this and also to make sure we have proper docs around the correct behavior for plugin writers.

#6 Updated by kersom over 1 year ago

  • Related to Issue #4920: Collection - Repository versions not being update after successive syncs added

#7 Updated by bmbouter over 1 year ago

  • Subject changed from Sync of unchanged importer update repo version to Document that a repo version is expected to be created for all sync's regardles of if there is new content remotely or now
  • Description updated (diff)
  • Sprint set to Sprint 54
  • Tags Documentation added

After discussion during today's open floor we decided:

1) the current behavior is a good behavior. It's less surprising to users when they ask Pulp to sync and receive a repo version N+1 that is that same as repo version N, and more confusing if they get some repo verisons for some sync's and not for others depending on the remote state.

2) We should document this expectation in the plugin write API, so now this is a docs story

3) it should go on the sprint

#8 Updated by amacdona@redhat.com over 1 year ago

  • Tracker changed from Issue to Story
  • % Done set to 0

#9 Updated by ttereshc over 1 year ago

  • Sprint changed from Sprint 54 to Sprint 55

#10 Updated by dkliban@redhat.com over 1 year ago

  • Sprint changed from Sprint 55 to Sprint 56

#11 Updated by rchan over 1 year ago

  • Sprint changed from Sprint 56 to Sprint 57

#12 Updated by rchan over 1 year ago

  • Sprint changed from Sprint 57 to Sprint 58

#13 Updated by bmbouter over 1 year ago

  • Sprint/Milestone deleted (3.0.0)

#14 Updated by rchan over 1 year ago

  • Sprint deleted (Sprint 58)

#15 Updated by daviddavis over 1 year ago

  • Sprint/Milestone set to 3.0.0

Making this a GA blocker. It keeps coming up and this should only take 5 min to fix.

#16 Updated by daviddavis about 1 year ago

  • Tracker changed from Story to Task
  • Subject changed from Document that a repo version is expected to be created for all sync's regardles of if there is new content remotely or now to Skip creating new repo versions when content has not changed
  • Description updated (diff)
  • Tags deleted (Documentation)

#17 Updated by dkliban@redhat.com about 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dkliban@redhat.com

#18 Updated by dalley about 1 year ago

  • Status changed from ASSIGNED to POST

#19 Updated by dkliban@redhat.com about 1 year ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#20 Updated by ipanova@redhat.com about 1 year ago

  • Status changed from MODIFIED to POST

#21 Updated by ipanova@redhat.com about 1 year ago

  • Status changed from POST to MODIFIED

#22 Updated by bmbouter about 1 year ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF