Project

Profile

Help

Story #1982

closed

As a user, I can force a full sync

Added by mhrivnak over 6 years ago. Updated almost 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.10.0
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Sprint 5
Quarter:

Description

A sync optimization exists that checks the remote metadata's revision number, and if it hasn't increased since the last successful sync, skips most of the current sync. It tries to accommodate several edge cases where the full sync should happen, but it's difficult and cumbersome to try catching them all. Here's what the decision looks like today:

        if 0 < metadata_files.revision <= previous_revision \
                and not self.config.override_config.get(importer_constants.KEY_FEED) \
                and previous_skip_set - current_skip_set == set() \
                and (self.download_deferred or not missing_units) \
                and not sync_due_to_unit_removal:

Users have encountered many cases where they want to force a full re-sync. Rather than try to predict and handle each condition individually as above, pulp should also offer an option to force it just because the user said so. :)

Similar work was done recently for the distributor: https://pulp.plan.io/issues/1158


Related issues

Related to RPM Support - Issue #1971: '--retain-old-count' option not retroactiveCLOSED - WONTFIXActions
Related to Pulp - Story #1983: As a user, an importer config change or content removal will cause the next sync to be fullCLOSED - CURRENTRELEASEdalley

Actions
Actions #1

Updated by mhrivnak over 6 years ago

  • Related to Issue #1971: '--retain-old-count' option not retroactive added
Actions #2

Updated by mhrivnak over 6 years ago

  • Related to Story #1983: As a user, an importer config change or content removal will cause the next sync to be full added
Actions #3

Updated by ttereshc over 6 years ago

What do you think about adding the force_full option to the platform?
No plugin specific decision is required in this case (just do re-sync no matter what) and potentially it can be useful not only for RPM plugin.

Actions #4

Updated by ipanova@redhat.com over 6 years ago

I agree with Tanya.
In case we decide to that in platform a note should be written in platform that there is new optional importer config parameter usable by on all importer plugin types.

Actions #5

Updated by ipanova@redhat.com over 6 years ago

  • Groomed changed from No to Yes
Actions #6

Updated by mhrivnak over 6 years ago

That sounds great.

Actions #7

Updated by mhrivnak over 6 years ago

  • Sprint/Milestone set to 22
Actions #8

Updated by mhrivnak over 6 years ago

  • Sprint/Milestone changed from 22 to 23
Actions #9

Updated by ipanova@redhat.com over 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fdobrovo
Actions #10

Updated by fdobrovo over 6 years ago

Added by fdobrovo over 6 years ago

Revision 51434764

1982 - Added --force-full option to importer pulp-admin

closes #1982 https://pulp.plan.io/issues/1982

pulp.client.commands.repo.sync_publish.SyncPublishCommand

  • Moved generate_override_config from RunPublishRepositoryCommand
    • SyncPublishCommand now accepts override_config_options
    • SyncPublishCommand now have new method generate_override_config

pulp.client.commands.repo.sync_publish.RunSyncRepositoryCommand

  • Now accepts --force-full flag

Added by fdobrovo over 6 years ago

Revision 51434764

1982 - Added --force-full option to importer pulp-admin

closes #1982 https://pulp.plan.io/issues/1982

pulp.client.commands.repo.sync_publish.SyncPublishCommand

  • Moved generate_override_config from RunPublishRepositoryCommand
    • SyncPublishCommand now accepts override_config_options
    • SyncPublishCommand now have new method generate_override_config

pulp.client.commands.repo.sync_publish.RunSyncRepositoryCommand

  • Now accepts --force-full flag

Added by fdobrovo over 6 years ago

Revision df42dea8

1982 - --force-full option for yum importer

re #1982 https://pulp.plan.io/issues/1982

Actions #12

Updated by fdobrovo over 6 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #13

Updated by fdobrovo over 6 years ago

  • Platform Release set to 2.10.0
Actions #14

Updated by semyers over 6 years ago

  • Status changed from MODIFIED to 5
Actions #15

Updated by Ichimonji10 over 6 years ago

  • Status changed from 5 to 6

Verified against a Pulp 2.10.0 system provisioned this morning.

(pulp-smash2) [ichimonji10@beech:pulp-smash]$ python -m unittest2 pulp_smash.tests.rpm.cli.test_sync
...
----------------------------------------------------------------------
Ran 3 tests in 135.682s

OK
(pulp-smash2) [ichimonji10@beech:pulp-smash]$ git grep 1982
pulp_smash/tests/rpm/cli/test_sync.py:    This test case targets `Pulp #1982`_ and `Pulp Smash #353`_. The test
pulp_smash/tests/rpm/cli/test_sync.py:    .. _Pulp #1982: https://pulp.plan.io/issues/1982
pulp_smash/tests/rpm/cli/test_sync.py:        if selectors.bug_is_untestable(1982, cfg.version):
pulp_smash/tests/rpm/cli/test_sync.py:            self.skipTest('https://pulp.plan.io/issues/1982')
(pulp-smash2) [ichimonji10@beech:pulp-smash]$ ssh $hostname rpm -qa | sort | grep -i pulp
Warning: Permanently added 'ibm-x3550m3-09.lab.eng.brq.redhat.com,10.34.36.133' (ECDSA) to the list of known hosts.
pulp-admin-client-2.10.0-0.4.beta.fc24.noarch
pulp-docker-admin-extensions-2.1.0-0.2.beta.fc24.noarch
pulp-docker-plugins-2.1.0-0.2.beta.fc24.noarch
pulp-ostree-admin-extensions-1.1.3-1.fc24.noarch
pulp-ostree-plugins-1.1.3-1.fc24.noarch
pulp-puppet-admin-extensions-2.10.0-0.1.beta.fc24.noarch
pulp-puppet-plugins-2.10.0-0.1.beta.fc24.noarch
pulp-python-admin-extensions-1.1.3-1.fc24.noarch
pulp-python-plugins-1.1.3-1.fc24.noarch
pulp-rpm-admin-extensions-2.10.0-0.4.beta.fc24.noarch
pulp-rpm-plugins-2.10.0-0.4.beta.fc24.noarch
pulp-selinux-2.10.0-0.4.beta.fc24.noarch
pulp-server-2.10.0-0.4.beta.fc24.noarch
python-kombu-3.0.33-6.pulp.fc24.noarch
python-pulp-bindings-2.10.0-0.4.beta.fc24.noarch
python-pulp-client-lib-2.10.0-0.4.beta.fc24.noarch
python-pulp-common-2.10.0-0.4.beta.fc24.noarch
python-pulp-docker-common-2.1.0-0.2.beta.fc24.noarch
python-pulp-oid_validation-2.10.0-0.4.beta.fc24.noarch
python-pulp-ostree-common-1.1.3-1.fc24.noarch
python-pulp-puppet-common-2.10.0-0.1.beta.fc24.noarch
python-pulp-python-common-1.1.3-1.fc24.noarch
python-pulp-repoauth-2.10.0-0.4.beta.fc24.noarch
python-pulp-rpm-common-2.10.0-0.4.beta.fc24.noarch
python-pulp-streamer-2.10.0-0.4.beta.fc24.noarch

Also see: https://github.com/PulpQE/pulp-smash/pull/380

Actions #16

Updated by semyers over 6 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE
Actions #18

Updated by bmbouter almost 5 years ago

  • Sprint set to Sprint 5
Actions #19

Updated by bmbouter almost 5 years ago

  • Sprint/Milestone deleted (23)
Actions #20

Updated by bmbouter almost 4 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF