Project

Profile

Help

Story #1983

closed

As a user, an importer config change or content removal will cause the next sync to be full

Added by mhrivnak over 8 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.11.0
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Sprint 6
Quarter:

Description

__A sync optimization exists that checks the remote metadata's revision number, and if it hasn't increased since the last successful sync, skips most of the current sync. It tries to accommodate several edge cases where the full sync should happen, but it's difficult and cumbersome to try catching them all. Here's what the decision looks like today:

        if 0 < metadata_files.revision <= previous_revision \
                and not self.config.override_config.get(importer_constants.KEY_FEED) \
                and previous_skip_set - current_skip_set == set() \
                and (self.download_deferred or not missing_units) \
                and not sync_due_to_unit_removal:

It would be valuable in this importer, and in others, to simplify some of the conditions down to these questions:

The sync should be full in case one of these answers are true:

1. Did the config change in any way since the last sync?
- A new field should be added to the Importer model 'last_updated'. As a consequence a migration should be written.
2. Has content been removed since the last sync?
3. If there was something specified in the override_config( because it changes current sync).
- A new field should be added to the Importer model 'last_override_config'. As a consequence a migration should be written.
4. Force-full option was specified.

Ideally, the platform should make it easy to answer those two questions. Then individual imports like this one can use that knowledge, combined with plugin-specific logic (like the repomd revision number) to decide if some or all of the sync can be skipped.

There is already code in the yum importer to answer question 2. It's only about 6 lines of code, but it would be easy and convenient to put that in the platform as a function, perhaps as a method on the Importer model.

Similar work was done here for distributors: https://pulp.plan.io/issues/1724


Related issues

Related to RPM Support - Story #1982: As a user, I can force a full syncCLOSED - CURRENTRELEASEfdobrovo

Actions
Related to RPM Support - Issue #1971: '--retain-old-count' option not retroactiveCLOSED - WONTFIXActions
Related to RPM Support - Issue #2027: remove_missing option does not work unless repository is updatedCLOSED - WONTFIXActions
Actions #1

Updated by mhrivnak over 8 years ago

  • Related to Story #1982: As a user, I can force a full sync added
Actions #2

Updated by mhrivnak over 8 years ago

  • Related to Issue #1971: '--retain-old-count' option not retroactive added
Actions #3

Updated by ttereshc over 8 years ago

  • Groomed changed from No to Yes
Actions #4

Updated by ipanova@redhat.com over 8 years ago

  • Description updated (diff)
Actions #5

Updated by mhrivnak over 8 years ago

  • Sprint/Milestone set to 22
Actions #6

Updated by ipanova@redhat.com over 8 years ago

  • Related to Issue #2027: remove_missing option does not work unless repository is updated added
Actions #9

Updated by mhrivnak over 8 years ago

  • Sprint/Milestone changed from 22 to 23
Actions #10

Updated by dalley over 8 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley
Actions #12

Updated by mhrivnak over 8 years ago

  • Sprint/Milestone changed from 23 to 24
Actions #13

Updated by dalley over 8 years ago

  • Status changed from ASSIGNED to POST
  • Platform Release set to 2.11.0

Added by dalley over 8 years ago

Revision 6868481c | View on GitHub

Add generic checks for whether a full sync should be forced to the repo controller. closes #1983

Added by dalley over 8 years ago

Revision 6868481c | View on GitHub

Add generic checks for whether a full sync should be forced to the repo controller. closes #1983

Actions #15

Updated by dalley over 8 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #16

Updated by semyers about 8 years ago

  • Status changed from MODIFIED to 5
Actions #17

Updated by semyers about 8 years ago

  • Project changed from RPM Support to Pulp
Actions #18

Updated by pcreech almost 8 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #20

Updated by bmbouter over 6 years ago

  • Sprint set to Sprint 6
Actions #21

Updated by bmbouter over 6 years ago

  • Sprint/Milestone deleted (24)
Actions #22

Updated by bmbouter over 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF