Issue #2844
Updated by Ichimonji10 over 7 years ago
To reproduce, do the following:
1. Create a yum repository with a yum and rsync distributor.
2. Add some content to the repository.
3. Publish with the yum and rsync distributors.
4. Publish with the yum and rsync distributors again.
The second publish with the rsync distributor should be a fast-forward publish, as no new units were added to the repository between the first and second publishes. Unfortunately, the second publish appears to be a full publish. This issue cropped up three months ago and was fixed in Pulp 2.13.1. See "issue #2666":https://pulp.plan.io/issues/2666. It has re-appeared in the Pulp 2.14 nightlies.
To figure out whether an issue is a full or fast-forward publish, one can examine the asynchronous task reports generated by Pulp. Specifically, one can do the following:
1. Publish an RPM repository with its rsync distributor. Capture the returned call report.
2. Recursively walk through all of the tasks referenced from this call report, and pick out the spawned task with a <code>task_type</code> of <code>pulp.server.managers.repo.publish.publish</code>.
3. Inspect the spawned task's steps, as listed out in <code>['result']['details']</code>. The <code>Unit query step (rpm, drpm, srpm)</code> step should list a value of zero in its <code>num_processed</code> field.
Here's an abbreviated example of a <code>pulp.server.managers.repo.publish.publish</code> task:
<pre><code class="python">{'snip': 'snippety snip!',
'result': {'_ns': 'repo_publish_results',
'completed': '2017-06-26T18:03:38Z',
'details': [{'description': '',
'details': '',
'error_details': [],
'items_total': 2,
'num_failures': 0,
'num_processed': 2,
'num_success': 2,
'state': 'FINISHED',
'step_id': 'a5b6e8e1-5d6e-4991-8930-e729e1709cc8',
'step_type': 'Unit query step (rpm, drpm, srpm)'},
{'description': 'Rsync files to remote destination',
'details': '',
'error_details': [],
'items_total': 1,
'num_failures': 0,
'num_processed': 1,
'num_success': 1,
'state': 'FINISHED',
'step_id': '78e1a4ae-0340-4786-b7a6-35c5bc93a506',
'step_type': 'Rsync step (origin)'},
{'description': 'Rsync files to remote destination',
'details': '',
'error_details': [],
'items_total': 1,
'num_failures': 0,
'num_processed': 1,
'num_success': 1,
'state': 'FINISHED',
'step_id': 'd2a0212a-57eb-4d33-9c1c-75449c734251',
'step_type': 'Rsync step (content)'},
{'description': 'Rsync files to remote destination',
'details': '',
'error_details': [],
'items_total': 1,
'num_failures': 0,
'num_processed': 1,
'num_success': 1,
'state': 'FINISHED',
'step_id': 'b0f8dc3c-1dfc-46f4-9def-54cccbd4b22a',
'step_type': 'Rsync step (repodata)'},
{'description': '',
'details': '',
'error_details': [],
'items_total': 1,
'num_failures': 0,
'num_processed': 1,
'num_success': 1,
'state': 'FINISHED',
'step_id': '0785cbfa-9e7a-4d05-8037-a69738eedc17',
'step_type': 'UpdateLastPredistDate'}],
'distributor_id': '9c0e0d42-93dc-4335-9168-06a0d292a77c',
'distributor_type_id': 'rpm_rsync_distributor',
'error_message': None,
'exception': None,
'id': '59514c7a14059a093629fb86',
'repo_id': '54b37808-1e7c-43d5-a06d-4e6415dfa3ac',
'result': 'success',
'started': '2017-06-26T18:03:36Z',
'summary': {'Rsync step (content)': 'FINISHED',
'Rsync step (origin)': 'FINISHED',
'Rsync step (repodata)': 'FINISHED',
'Unit query step (rpm, drpm, srpm)': 'FINISHED',
'UpdateLastPredistDate': 'FINISHED'},
'traceback': None},
'spawned_tasks': [],
'start_time': '2017-06-26T18:03:36Z',
'state': 'finished',
'tags': ['pulp:repository:54b37808-1e7c-43d5-a06d-4e6415dfa3ac',
'pulp:action:publish'],
'task_id': '05cef649-ae80-4a78-b1af-18926b4dcc8c',
'task_type': 'pulp.server.managers.repo.publish.publish',
'traceback': None,
'worker_name': 'reserved_resource_worker-0@rhel-7-pulp-2-14'}
</code></pre>
Notice the <code>'num_processed': 2</code> line? That's correct for an initial rsync publish. It's incorrect for a fast-forward publish. (I think it indicates that two units were copied to the remote host.) For a fast-forward publish, that value should be zero. For more on this, see: "Pulp #2666 Note #15":https://pulp.plan.io/issues/2666#note-15 and subsequent comments.
This issue can also be reproduced with Pulp Smash:
<pre>python -m unittest pulp_smash.tests.rpm.api_v2.test_rsync_distributor.PublishTwiceTestCase</pre>
.