Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #2844

closed

Rsync publish for RPM repo fails to run in fast forward mode

Added by Ichimonji10 over 7 years ago. Updated over 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
dalley
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Master
Platform Release:
2.13.3
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 21
Quarter:

Description

To reproduce, do the following:

1. Create a yum repository with a yum and rsync distributor.
2. Add some content to the repository.
3. Publish with the yum and rsync distributors.
4. Publish with the yum and rsync distributors again.

The second publish with the rsync distributor should be a fast-forward publish, as no new units were added to the repository between the first and second publishes. Unfortunately, the second publish appears to be a full publish. This issue cropped up three months ago and was fixed in Pulp 2.13.1. See issue #2666. It has re-appeared in the Pulp 2.14 nightlies.

To figure out whether an issue is a full or fast-forward publish, one can examine the asynchronous task reports generated by Pulp. Specifically, one can do the following:

1. Publish an RPM repository with its rsync distributor. Capture the returned call report.
2. Recursively walk through all of the tasks referenced from this call report, and pick out the spawned task with a task_type of pulp.server.managers.repo.publish.publish.
3. Inspect the spawned task's steps, as listed out in ['result']['details']. The Unit query step (rpm, drpm, srpm) step should list a value of zero in its num_processed field.

Here's an abbreviated example of a pulp.server.managers.repo.publish.publish task:

{'snip': 'snippety snip!',
 'result': {'_ns': 'repo_publish_results',
            'completed': '2017-06-26T18:03:38Z',
            'details': [{'description': '',
                         'details': '',
                         'error_details': [],
                         'items_total': 2,
                         'num_failures': 0,
                         'num_processed': 2,
                         'num_success': 2,
                         'state': 'FINISHED',
                         'step_id': 'a5b6e8e1-5d6e-4991-8930-e729e1709cc8',
                         'step_type': 'Unit query step (rpm, drpm, srpm)'},
                        {'description': 'Rsync files to remote destination',
                         'details': '',
                         'error_details': [],
                         'items_total': 1,
                         'num_failures': 0,
                         'num_processed': 1,
                         'num_success': 1,
                         'state': 'FINISHED',
                         'step_id': '78e1a4ae-0340-4786-b7a6-35c5bc93a506',
                         'step_type': 'Rsync step (origin)'},
                        {'description': 'Rsync files to remote destination',
                         'details': '',
                         'error_details': [],
                         'items_total': 1,
                         'num_failures': 0,
                         'num_processed': 1,
                         'num_success': 1,
                         'state': 'FINISHED',
                         'step_id': 'd2a0212a-57eb-4d33-9c1c-75449c734251',
                         'step_type': 'Rsync step (content)'},
                        {'description': 'Rsync files to remote destination',
                         'details': '',
                         'error_details': [],
                         'items_total': 1,
                         'num_failures': 0,
                         'num_processed': 1,
                         'num_success': 1,
                         'state': 'FINISHED',
                         'step_id': 'b0f8dc3c-1dfc-46f4-9def-54cccbd4b22a',
                         'step_type': 'Rsync step (repodata)'},
                        {'description': '',
                         'details': '',
                         'error_details': [],
                         'items_total': 1,
                         'num_failures': 0,
                         'num_processed': 1,
                         'num_success': 1,
                         'state': 'FINISHED',
                         'step_id': '0785cbfa-9e7a-4d05-8037-a69738eedc17',
                         'step_type': 'UpdateLastPredistDate'}],
            'distributor_id': '9c0e0d42-93dc-4335-9168-06a0d292a77c',
            'distributor_type_id': 'rpm_rsync_distributor',
            'error_message': None,
            'exception': None,
            'id': '59514c7a14059a093629fb86',
            'repo_id': '54b37808-1e7c-43d5-a06d-4e6415dfa3ac',
            'result': 'success',
            'started': '2017-06-26T18:03:36Z',
            'summary': {'Rsync step (content)': 'FINISHED',
                        'Rsync step (origin)': 'FINISHED',
                        'Rsync step (repodata)': 'FINISHED',
                        'Unit query step (rpm, drpm, srpm)': 'FINISHED',
                        'UpdateLastPredistDate': 'FINISHED'},
            'traceback': None},
 'spawned_tasks': [],
 'start_time': '2017-06-26T18:03:36Z',
 'state': 'finished',
 'tags': ['pulp:repository:54b37808-1e7c-43d5-a06d-4e6415dfa3ac',
          'pulp:action:publish'],
 'task_id': '05cef649-ae80-4a78-b1af-18926b4dcc8c',
 'task_type': 'pulp.server.managers.repo.publish.publish',
 'traceback': None,
 'worker_name': 'reserved_resource_worker-0@rhel-7-pulp-2-14'}

Notice the 'num_processed': 2 line? That's correct for an initial rsync publish. It's incorrect for a fast-forward publish. (I think it indicates that two units were copied to the remote host.) For a fast-forward publish, that value should be zero. For more on this, see: Pulp #2666 Note #15 and subsequent comments.

This issue can also be reproduced with Pulp Smash:

python -m unittest pulp_smash.tests.rpm.api_v2.test_rsync_distributor.PublishTwiceTestCase

Related issues

Copied from RPM Support - Issue #2666: Rsync publish for RPM repo fails to run in fast forward modeCLOSED - CURRENTRELEASEipanova@redhat.comActions
Actions
Send by e-mail Copy link

Also available in: Atom PDF