Story #1983
closedAs a user, an importer config change or content removal will cause the next sync to be full
100%
Description
__A sync optimization exists that checks the remote metadata's revision number, and if it hasn't increased since the last successful sync, skips most of the current sync. It tries to accommodate several edge cases where the full sync should happen, but it's difficult and cumbersome to try catching them all. Here's what the decision looks like today:
if 0 < metadata_files.revision <= previous_revision \
and not self.config.override_config.get(importer_constants.KEY_FEED) \
and previous_skip_set - current_skip_set == set() \
and (self.download_deferred or not missing_units) \
and not sync_due_to_unit_removal:
It would be valuable in this importer, and in others, to simplify some of the conditions down to these questions:
The sync should be full in case one of these answers are true:
1. Did the config change in any way since the last sync?
- A new field should be added to the Importer model 'last_updated'. As a consequence a migration should be written.
2. Has content been removed since the last sync?
3. If there was something specified in the override_config( because it changes current sync).
- A new field should be added to the Importer model 'last_override_config'. As a consequence a migration should be written.
4. Force-full option was specified.
Ideally, the platform should make it easy to answer those two questions. Then individual imports like this one can use that knowledge, combined with plugin-specific logic (like the repomd revision number) to decide if some or all of the sync can be skipped.
There is already code in the yum importer to answer question 2. It's only about 6 lines of code, but it would be easy and convenient to put that in the platform as a function, perhaps as a method on the Importer model.
Similar work was done here for distributors: https://pulp.plan.io/issues/1724
Related issues
Updated by mhrivnak over 8 years ago
- Related to Story #1982: As a user, I can force a full sync added
Updated by mhrivnak over 8 years ago
- Related to Issue #1971: '--retain-old-count' option not retroactive added
Updated by ipanova@redhat.com over 8 years ago
- Related to Issue #2027: remove_missing option does not work unless repository is updated added
Updated by dalley over 8 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to dalley
Updated by dalley over 8 years ago
- Status changed from ASSIGNED to POST
- Platform Release set to 2.11.0
Updated by dalley over 8 years ago
Added by dalley over 8 years ago
Added by dalley over 8 years ago
Revision 6868481c | View on GitHub
Add generic checks for whether a full sync should be forced to the repo controller. closes #1983
Updated by dalley over 8 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulp:pulp|6868481c1a0eeab06d1f18330e7fd7dc533d8007.
Updated by pcreech almost 8 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
Add generic checks for whether a full sync should be forced to the repo controller. closes #1983