Project

Profile

Help

Issue #2263

closed

Fast-forward yum publish skips units if previous publish was cancelled

Added by rmcgover over 7 years ago. Updated about 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.10.0
Platform Release:
2.12.1
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 14
Quarter:

Description

If a yum publish is triggered on a repo, then cancelled before completion, that publish still sets last_publish on the distributor despite being cancelled.

The next yum publish of that repo will then look for units associated after that last_publish, incorrectly skipping any units which hadn't been published due to the cancellation.

Steps to reproduce:

1. Trigger yum distributor on repo, allow it to succeed.
2. Trigger yum distributor on repo.
3. Cancel task from previous step after it starts but before it publishes the RPMs (may have to be quick)
4. Trigger yum distributor on repo, allow it to succeed.

Actual behavior:

  • the task triggered at step 4 completes with "summary": "Skipped: Repository content has not changed since last publish."
  • the RPMs associated at step 2 aren't published

Expected behavior:

  • the task triggered at step 4 publishes the RPMs associated at step 2

Note: because yum publish of a small repo will be quite fast, I used a patch like this while reproducing it:

diff --git a/plugins/pulp_rpm/plugins/distributors/yum/publish.py b/plugins/pulp_rpm/plugins/distributors/yum/publish.py
index cb08d75..5914d58 100644
--- a/plugins/pulp_rpm/plugins/distributors/yum/publish.py
+++ b/plugins/pulp_rpm/plugins/distributors/yum/publish.py
@ -386,6 +386,11 @ class InitRepoMetadataStep(platform_steps.PluginStep):
self.parent.get_checksum_type())
self.parent.repomd_file_context.initialize()

+ logger.info("SLEEPING FOR TEST!!!")
+ import time
+ time.sleep(60)
+ logger.info("END SLEEPING FOR TEST!!!")
+

class CloseRepoMetadataStep(platform_steps.PluginStep):

In practice in our installation we've seen this issue affecting larger repos, where it'd be easier to reproduce since tasks have more time to be cancelled.

Actions #1

Updated by jluza over 7 years ago

Rohan, could you check last_publish attribute of yum distributor after each publish? There was bug which causes last_publish date was updated even if task failed, but I'm sure we already fixed the bug (in our fork and also in upstream).

Actions #2

Updated by rmcgover over 7 years ago

Yes, last_publish is still set even if the publish is cancelled. If it wasn't, this bug wouldn't happen.

Actions #3

Updated by amacdona@redhat.com over 7 years ago

  • Priority changed from Normal to High
  • Triaged changed from No to Yes
Actions #4

Updated by ipanova@redhat.com over 7 years ago

We should check if the last_published field gets updated just when the publish succeeds, it should not be touched in case published failed or canceled.

Actions #5

Updated by mhrivnak over 7 years ago

  • Sprint/Milestone set to 32
Actions #6

Updated by daviddavis about 7 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to daviddavis
Actions #7

Updated by daviddavis about 7 years ago

Test notes: you'll probably want to apply this patch have the code pause in the publish step:

https://gist.github.com/daviddavis/f05674b81d9376a4cc42eff1e498b7d3

Then when you run your sync from pulp-admin, check your logs for "SLEEPING FOREVER". It'll be when the pulp-admin displays "Initializing repo metadata".

Actions #8

Updated by daviddavis about 7 years ago

  • Status changed from ASSIGNED to POST

Added by daviddavis about 7 years ago

Revision 485645f1 | View on GitHub

Only update last_publish if success and not canceled

Only update the last_publish field is the publish was a success and not canceled.

fixes #2263 https://pulp.plan.io/issues/2263

Added by daviddavis about 7 years ago

Revision 485645f1 | View on GitHub

Only update last_publish if success and not canceled

Only update the last_publish field is the publish was a success and not canceled.

fixes #2263 https://pulp.plan.io/issues/2263

Actions #9

Updated by daviddavis about 7 years ago

  • Status changed from POST to MODIFIED
Actions #11

Updated by bizhang about 7 years ago

  • Platform Release set to 2.12.1
Actions #12

Updated by bizhang about 7 years ago

  • Status changed from MODIFIED to 5
Actions #13

Updated by bizhang about 7 years ago

  • Version changed from 2.10.0 to 2.12.1
Actions #14

Updated by bizhang about 7 years ago

  • Version changed from 2.12.1 to 2.10.0
Actions #15

Updated by bizhang about 7 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #16

Updated by bmbouter about 6 years ago

  • Sprint set to Sprint 16
Actions #17

Updated by bmbouter about 6 years ago

  • Sprint changed from Sprint 16 to Sprint 14
Actions #18

Updated by bmbouter about 6 years ago

  • Sprint/Milestone deleted (32)
Actions #19

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF