Issue #2263
closedFast-forward yum publish skips units if previous publish was cancelled
Description
If a yum publish is triggered on a repo, then cancelled before completion, that publish still sets last_publish on the distributor despite being cancelled.
The next yum publish of that repo will then look for units associated after that last_publish, incorrectly skipping any units which hadn't been published due to the cancellation.
Steps to reproduce:
1. Trigger yum distributor on repo, allow it to succeed.
2. Trigger yum distributor on repo.
3. Cancel task from previous step after it starts but before it publishes the RPMs (may have to be quick)
4. Trigger yum distributor on repo, allow it to succeed.
Actual behavior:
- the task triggered at step 4 completes with "summary": "Skipped: Repository content has not changed since last publish."
- the RPMs associated at step 2 aren't published
Expected behavior:
- the task triggered at step 4 publishes the RPMs associated at step 2
Note: because yum publish of a small repo will be quite fast, I used a patch like this while reproducing it:
diff --git a/plugins/pulp_rpm/plugins/distributors/yum/publish.py b/plugins/pulp_rpm/plugins/distributors/yum/publish.py
index cb08d75..5914d58 100644
--- a/plugins/pulp_rpm/plugins/distributors/yum/publish.py
+++ b/plugins/pulp_rpm/plugins/distributors/yum/publish.py
@ -386,6 +386,11
@ class InitRepoMetadataStep(platform_steps.PluginStep):
self.parent.get_checksum_type())
self.parent.repomd_file_context.initialize()
+ logger.info("SLEEPING FOR TEST!!!")
+ import time
+ time.sleep(60)
+ logger.info("END SLEEPING FOR TEST!!!")
+
class CloseRepoMetadataStep(platform_steps.PluginStep):
In practice in our installation we've seen this issue affecting larger repos, where it'd be easier to reproduce since tasks have more time to be cancelled.
Updated by jluza over 7 years ago
Rohan, could you check last_publish attribute of yum distributor after each publish? There was bug which causes last_publish date was updated even if task failed, but I'm sure we already fixed the bug (in our fork and also in upstream).
Updated by rmcgover over 7 years ago
Yes, last_publish is still set even if the publish is cancelled. If it wasn't, this bug wouldn't happen.
Updated by amacdona@redhat.com over 7 years ago
- Priority changed from Normal to High
- Triaged changed from No to Yes
Updated by ipanova@redhat.com over 7 years ago
We should check if the last_published field gets updated just when the publish succeeds, it should not be touched in case published failed or canceled.
Updated by daviddavis about 7 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to daviddavis
Updated by daviddavis about 7 years ago
Test notes: you'll probably want to apply this patch have the code pause in the publish step:
https://gist.github.com/daviddavis/f05674b81d9376a4cc42eff1e498b7d3
Then when you run your sync from pulp-admin, check your logs for "SLEEPING FOREVER". It'll be when the pulp-admin displays "Initializing repo metadata".
Updated by daviddavis about 7 years ago
- Status changed from ASSIGNED to POST
Added by daviddavis about 7 years ago
Added by daviddavis about 7 years ago
Revision 485645f1 | View on GitHub
Only update last_publish if success and not canceled
Only update the last_publish field is the publish was a success and not canceled.
Updated by daviddavis about 7 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulp:pulp|485645f1a1557240f92b5b95b07b794a3a8c38c1.
Updated by bizhang about 7 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
Updated by bmbouter about 6 years ago
- Sprint changed from Sprint 16 to Sprint 14
Only update last_publish if success and not canceled
Only update the last_publish field is the publish was a success and not canceled.
fixes #2263 https://pulp.plan.io/issues/2263