Project

Profile

Help

Issue #2791

closed

Rsync publish includes units which have not yet been published via the predistributor

Added by dalley almost 7 years ago. Updated about 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
2.13.3
OS:
Triaged:
Yes
Groomed:
Yes
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Two specific special cases will cause units to be included in a publish which are not meant to be - units which have not been published via the predistributor.

1. Deleting a unit will cause a full republish of all units, including units which have not yet been published by the yum predistributor

dnf download cowsay
phttp POST https://localhost/pulp/api/v2/repositories/zoo/distributors/ < rsync_distributor.json
pulp-admin rpm repo sync run --repo-id zoo
phttp POST https://localhost/pulp/api/v2/repositories/zoo/actions/publish/ id=yum_distributor
phttp POST https://localhost/pulp/api/v2/repositories/zoo/actions/publish/ id=my_rpm_rsync_distributor
pulp-admin rpm repo remove rpm --repo-id zoo --str-eq 'filename=kangaroo-0.2-1.noarch.rpm'
phttp POST https://localhost/pulp/api/v2/repositories/zoo/actions/publish/ id=yum_distributor
pulp-admin rpm repo uploads rpm --repo-id zoo --file cowsay-3.04-4.fc25.noarch.rpm
phttp POST https://localhost/pulp/api/v2/repositories/zoo/actions/publish/ id=my_rpm_rsync_distributor

Result: cowsay RPM is published via rsync despite not having been published by the yum predistributor

2. On the very first rsync publish, if a predistributor publish has occurred it will do a full publish of all units, including those not yet published via the yum predistributor


dnf download cowsay
phttp POST https://localhost/pulp/api/v2/repositories/zoo/distributors/ < rsync_distributor.json
phttp POST https://localhost/pulp/api/v2/repositories/zoo/actions/publish/ id=yum_distributor
pulp-admin rpm repo uploads rpm --repo-id zoo --file cowsay-3.04-4.fc25.noarch.rpm
phttp POST https://localhost/pulp/api/v2/repositories/zoo/actions/publish/ id=my_rpm_rsync_distributor

Result: cowsay RPM is published via rsync despite not having been published by the yum predistributor

The culprit is this code in pulp/server/plugins/rsync/publish.py:


.....
.....
.....
        if self.is_fastforward():
            start_date = self.last_published
            end_date = None
            if self.predistributor:
                end_date = self.predistributor["last_publish"]
            date_filter = self.create_date_range_filter(start_date=start_date, end_date=end_date)
        else:
            date_filter = None
.....
.....
.....

def is_fastforward(self):
     ....
     ....
     ....
     return last_published and ((last_deleted and last_published > last_deleted) or not last_deleted) and not force_full and not delete

If it is the first publish or if a unit has been or is being deleted, is_fastforward() resolves to False, and the date filter is set to "None". Therefore, it is not respecting the date of the predistributor publish.

Actions #1

Updated by dalley almost 7 years ago

  • Status changed from ASSIGNED to POST
Actions #2

Updated by dalley almost 7 years ago

  • Description updated (diff)
  • Groomed changed from No to Yes
Actions #3

Updated by ttereshc almost 7 years ago

  • Triaged changed from No to Yes

Added by dalley almost 7 years ago

Revision 82651e04 | View on GitHub

Fix rsync publishing units it shouldn't

Fixes an issue with rsync logic where some units which have not yet been published by the predistributor are included with rsync publishes.

closes #2791 https://pulp.plan.io/issues/2791

Actions #5

Updated by dalley almost 7 years ago

  • Status changed from POST to MODIFIED

Added by dalley almost 7 years ago

Revision a9c6d55b | View on GitHub

Fix rsync publishing units it shouldn't

Fixes an issue with rsync logic where some units which have not yet been published by the predistributor are included with rsync publishes.

closes #2791 https://pulp.plan.io/issues/2791

(cherry picked from commit 82651e04a3ea2308a6c6a38a70128abe058a6f05)

Actions #7

Updated by pcreech almost 7 years ago

  • Platform Release set to 2.13.3
Actions #8

Updated by pcreech over 6 years ago

  • Status changed from MODIFIED to 5
Actions #9

Updated by pcreech over 6 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #10

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF