Project

Profile

Help

Story #1983

closed

As a user, an importer config change or content removal will cause the next sync to be full

Added by mhrivnak over 8 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.11.0
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Sprint 6
Quarter:

Description

__A sync optimization exists that checks the remote metadata's revision number, and if it hasn't increased since the last successful sync, skips most of the current sync. It tries to accommodate several edge cases where the full sync should happen, but it's difficult and cumbersome to try catching them all. Here's what the decision looks like today:

        if 0 < metadata_files.revision <= previous_revision \
                and not self.config.override_config.get(importer_constants.KEY_FEED) \
                and previous_skip_set - current_skip_set == set() \
                and (self.download_deferred or not missing_units) \
                and not sync_due_to_unit_removal:

It would be valuable in this importer, and in others, to simplify some of the conditions down to these questions:

The sync should be full in case one of these answers are true:

1. Did the config change in any way since the last sync?
- A new field should be added to the Importer model 'last_updated'. As a consequence a migration should be written.
2. Has content been removed since the last sync?
3. If there was something specified in the override_config( because it changes current sync).
- A new field should be added to the Importer model 'last_override_config'. As a consequence a migration should be written.
4. Force-full option was specified.

Ideally, the platform should make it easy to answer those two questions. Then individual imports like this one can use that knowledge, combined with plugin-specific logic (like the repomd revision number) to decide if some or all of the sync can be skipped.

There is already code in the yum importer to answer question 2. It's only about 6 lines of code, but it would be easy and convenient to put that in the platform as a function, perhaps as a method on the Importer model.

Similar work was done here for distributors: https://pulp.plan.io/issues/1724


Related issues

Related to RPM Support - Story #1982: As a user, I can force a full syncCLOSED - CURRENTRELEASEfdobrovo

Actions
Related to RPM Support - Issue #1971: '--retain-old-count' option not retroactiveCLOSED - WONTFIXActions
Related to RPM Support - Issue #2027: remove_missing option does not work unless repository is updatedCLOSED - WONTFIXActions

Also available in: Atom PDF