Project

Profile

Help

Story #1982

As a user, I can force a full sync

Added by mhrivnak about 5 years ago. Updated about 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.10.0
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Sprint 5
Quarter:

Description

A sync optimization exists that checks the remote metadata's revision number, and if it hasn't increased since the last successful sync, skips most of the current sync. It tries to accommodate several edge cases where the full sync should happen, but it's difficult and cumbersome to try catching them all. Here's what the decision looks like today:

        if 0 < metadata_files.revision <= previous_revision \
                and not self.config.override_config.get(importer_constants.KEY_FEED) \
                and previous_skip_set - current_skip_set == set() \
                and (self.download_deferred or not missing_units) \
                and not sync_due_to_unit_removal:

Users have encountered many cases where they want to force a full re-sync. Rather than try to predict and handle each condition individually as above, pulp should also offer an option to force it just because the user said so. :)

Similar work was done recently for the distributor: https://pulp.plan.io/issues/1158


Checklist


Related issues

Related to RPM Support - Issue #1971: '--retain-old-count' option not retroactiveCLOSED - WONTFIX<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to Pulp - Story #1983: As a user, an importer config change or content removal will cause the next sync to be fullCLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 51434764 View on GitHub
Added by fdobrovo almost 5 years ago

1982 - Added --force-full option to importer pulp-admin

closes #1982 https://pulp.plan.io/issues/1982

pulp.client.commands.repo.sync_publish.SyncPublishCommand

  • Moved generate_override_config from RunPublishRepositoryCommand
    • SyncPublishCommand now accepts override_config_options
    • SyncPublishCommand now have new method generate_override_config

pulp.client.commands.repo.sync_publish.RunSyncRepositoryCommand

  • Now accepts --force-full flag

Revision 51434764 View on GitHub
Added by fdobrovo almost 5 years ago

1982 - Added --force-full option to importer pulp-admin

closes #1982 https://pulp.plan.io/issues/1982

pulp.client.commands.repo.sync_publish.SyncPublishCommand

  • Moved generate_override_config from RunPublishRepositoryCommand
    • SyncPublishCommand now accepts override_config_options
    • SyncPublishCommand now have new method generate_override_config

pulp.client.commands.repo.sync_publish.RunSyncRepositoryCommand

  • Now accepts --force-full flag

Revision df42dea8 View on GitHub
Added by fdobrovo almost 5 years ago

1982 - --force-full option for yum importer

re #1982 https://pulp.plan.io/issues/1982

History

#1 Updated by mhrivnak about 5 years ago

  • Related to Issue #1971: '--retain-old-count' option not retroactive added

#2 Updated by mhrivnak about 5 years ago

  • Related to Story #1983: As a user, an importer config change or content removal will cause the next sync to be full added

#3 Updated by ttereshc about 5 years ago

What do you think about adding the force_full option to the platform?
No plugin specific decision is required in this case (just do re-sync no matter what) and potentially it can be useful not only for RPM plugin.

#4 Updated by ipanova@redhat.com about 5 years ago

I agree with Tanya.
In case we decide to that in platform a note should be written in platform that there is new optional importer config parameter usable by on all importer plugin types.

#5 Updated by ipanova@redhat.com about 5 years ago

  • Groomed changed from No to Yes

#6 Updated by mhrivnak about 5 years ago

That sounds great.

#7 Updated by mhrivnak about 5 years ago

  • Sprint/Milestone set to 22

#8 Updated by mhrivnak almost 5 years ago

  • Sprint/Milestone changed from 22 to 23

#9 Updated by ipanova@redhat.com almost 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fdobrovo

#10 Updated by fdobrovo almost 5 years ago

  • Checklist item Document a new importer setting set to Done
  • Checklist item Add code to use the new setting set to Done
  • Checklist item Add the new setting to pulp-admin set to Done
  • Checklist item Add release notes set to Done

#12 Updated by fdobrovo almost 5 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#13 Updated by fdobrovo almost 5 years ago

  • Platform Release set to 2.10.0

#14 Updated by semyers almost 5 years ago

  • Status changed from MODIFIED to 5

#15 Updated by Ichimonji10 almost 5 years ago

  • Status changed from 5 to 6

Verified against a Pulp 2.10.0 system provisioned this morning.

(pulp-smash2) [ichimonji10@beech:pulp-smash]$ python -m unittest2 pulp_smash.tests.rpm.cli.test_sync
...
----------------------------------------------------------------------
Ran 3 tests in 135.682s

OK
(pulp-smash2) [ichimonji10@beech:pulp-smash]$ git grep 1982
pulp_smash/tests/rpm/cli/test_sync.py:    This test case targets `Pulp #1982`_ and `Pulp Smash #353`_. The test
pulp_smash/tests/rpm/cli/test_sync.py:    .. _Pulp #1982: https://pulp.plan.io/issues/1982
pulp_smash/tests/rpm/cli/test_sync.py:        if selectors.bug_is_untestable(1982, cfg.version):
pulp_smash/tests/rpm/cli/test_sync.py:            self.skipTest('https://pulp.plan.io/issues/1982')
(pulp-smash2) [ichimonji10@beech:pulp-smash]$ ssh $hostname rpm -qa | sort | grep -i pulp
Warning: Permanently added 'ibm-x3550m3-09.lab.eng.brq.redhat.com,10.34.36.133' (ECDSA) to the list of known hosts.
pulp-admin-client-2.10.0-0.4.beta.fc24.noarch
pulp-docker-admin-extensions-2.1.0-0.2.beta.fc24.noarch
pulp-docker-plugins-2.1.0-0.2.beta.fc24.noarch
pulp-ostree-admin-extensions-1.1.3-1.fc24.noarch
pulp-ostree-plugins-1.1.3-1.fc24.noarch
pulp-puppet-admin-extensions-2.10.0-0.1.beta.fc24.noarch
pulp-puppet-plugins-2.10.0-0.1.beta.fc24.noarch
pulp-python-admin-extensions-1.1.3-1.fc24.noarch
pulp-python-plugins-1.1.3-1.fc24.noarch
pulp-rpm-admin-extensions-2.10.0-0.4.beta.fc24.noarch
pulp-rpm-plugins-2.10.0-0.4.beta.fc24.noarch
pulp-selinux-2.10.0-0.4.beta.fc24.noarch
pulp-server-2.10.0-0.4.beta.fc24.noarch
python-kombu-3.0.33-6.pulp.fc24.noarch
python-pulp-bindings-2.10.0-0.4.beta.fc24.noarch
python-pulp-client-lib-2.10.0-0.4.beta.fc24.noarch
python-pulp-common-2.10.0-0.4.beta.fc24.noarch
python-pulp-docker-common-2.1.0-0.2.beta.fc24.noarch
python-pulp-oid_validation-2.10.0-0.4.beta.fc24.noarch
python-pulp-ostree-common-1.1.3-1.fc24.noarch
python-pulp-puppet-common-2.10.0-0.1.beta.fc24.noarch
python-pulp-python-common-1.1.3-1.fc24.noarch
python-pulp-repoauth-2.10.0-0.4.beta.fc24.noarch
python-pulp-rpm-common-2.10.0-0.4.beta.fc24.noarch
python-pulp-streamer-2.10.0-0.4.beta.fc24.noarch

Also see: https://github.com/PulpQE/pulp-smash/pull/380

#16 Updated by semyers almost 5 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE

#18 Updated by bmbouter over 3 years ago

  • Sprint set to Sprint 5

#19 Updated by bmbouter over 3 years ago

  • Sprint/Milestone deleted (23)

#20 Updated by bmbouter about 2 years ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF