Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2017-11-07T10:34:51ZPulp
Planio RPM Support - Issue #3120 (CLOSED - CURRENTRELEASE): Race condition on saving a distribution unit...https://pulp.plan.io/issues/31202017-11-07T10:34:51Zttereshcttereshc@redhat.com
<pre><code>Nov 6 14:36:30 test-machine pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [aa7bf120] (11567-79808) Tried to save duplicate unique keys (insertDocument :: caused by :: 11000 E11000 duplicate key error index: pulp_database.units_distribution.$distribution_id_1_family_1_variant_1_version_1_arch_1 dup key: { : "ks-Repo-Server-1.1.1-x86_64", : "Repo", : "Server", : "1.1.1", : "x86_64" })
Nov 6 14:36:30 test-machine pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [aa7bf120] (11567-79808) Traceback (most recent call last):
Nov 6 14:36:30 test-machine pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [aa7bf120] (11567-79808) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 277, in run
Nov 6 14:36:30 test-machine pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [aa7bf120] (11567-79808) dist_sync.run()
Nov 6 14:36:30 test-machine pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [aa7bf120] (11567-79808) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 128, in run
Nov 6 14:36:30 test-machine pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [aa7bf120] (11567-79808) self._run(tmp_dir)
Nov 6 14:36:30 test-machine pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [aa7bf120] (11567-79808) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 186, in _run
Nov 6 14:36:30 test-machine pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [aa7bf120] (11567-79808) unit.save()
Nov 6 14:36:30 test-machine pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [aa7bf120] (11567-79808) File "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 390, in save
Nov 6 14:36:30 test-machine pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [aa7bf120] (11567-79808) raise NotUniqueError(message %% unicode(err))
Nov 6 14:36:30 test-machine pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [aa7bf120] (11567-79808) NotUniqueError: Tried to save duplicate unique keys (insertDocument :: caused by :: 11000 E11000 duplicate key error index: pulp_database.units_distribution.$distribution_id_1_family_1_variant_1_version_1_arch_1 dup key: { : "ks-Repo-Server-1.1.1-x86_64", : "Repo", : "Server", : "1.1.1", : "x86_64" })
</code></pre>
<p>Race condition happens <a href="https://github.com/pulp/pulp_rpm/blob/pulp-rpm-2.14.2-1/plugins/pulp_rpm/plugins/importers/yum/parse/treeinfo.py#L186" class="external">here</a> when multiple copies of the same repo are synced in parallel (for Katello case - multiple CVs of the repo are synced to "smart proxy")</p> RPM Support - Issue #3115 (CLOSED - CURRENTRELEASE): One sample request in API doc for regenerate...https://pulp.plan.io/issues/31152017-11-03T11:05:55Zpmoravec@redhat.compmoravec@redhat.com
<p>In <a href="https://docs.pulpproject.org/dev-guide/integration/rest-api/consumer/applicability.html" class="external">https://docs.pulpproject.org/dev-guide/integration/rest-api/consumer/applicability.html</a> , section "Generate Content Applicability for Updated Repositories", there is Sample request:</p>
<pre><code>{
"repo_criteria": {
"filters": {"id": {"$in": ["test-repo", "test-errata"]}},
"parallel": true
}
}
</code></pre>
<p>However "parallel" is not a part of "repo_criteria" but a parameter on the same level. That Sample request should be:</p>
<pre><code>{
"repo_criteria": {
"filters": {"id": {"$in": ["test-repo", "test-errata"]}},
},
"parallel": true
}
</code></pre> RPM Support - Issue #3100 (CLOSED - CURRENTRELEASE): Removal of existing iso units doesn't work i...https://pulp.plan.io/issues/31002017-10-26T14:40:35Zdaviddavis
<p>This is similar to <a class="issue tracker-1 status-11 priority-6 priority-default closed" title="Issue: ISO repo doesn't correctly handle updates to files for content already in Pulp (CLOSED - CURRENTRELEASE)" href="https://pulp.plan.io/issues/3047">#3047</a> and <a class="issue tracker-1 status-11 priority-7 priority-high2 closed" title="Issue: ISO repo does not handle updates to files on manifest during re-sync correctly (CLOSED - CURRENTRELEASE)" href="https://pulp.plan.io/issues/2773">#2773</a>. We added code to remove duplicate ISOs however, this code doesn't work after one iteration. Looks like a generator type was being used instead of a list:</p>
<p><a href="https://github.com/pulp/pulp_rpm/blob/23492381f285f2799aa894f8834e3fbc40b3ac72/plugins/pulp_rpm/plugins/importers/iso/sync.py#L333" class="external">https://github.com/pulp/pulp_rpm/blob/23492381f285f2799aa894f8834e3fbc40b3ac72/plugins/pulp_rpm/plugins/importers/iso/sync.py#L333</a></p>
<p>Steps to reproduce:</p>
<pre><code>export REPO1='iso-1'
export REPO2='iso-2'
mkdir /tmp/iso
cd /tmp/iso
echo "test1" > 1.iso
echo "test2" > 2.iso
echo 1.iso,`sha256sum 1.iso | awk '{ print $1 }'`,`stat -L -c '%s' 1.iso` > PULP_MANIFEST
echo 2.iso,`sha256sum 2.iso | awk '{ print $1 }'`,`stat -L -c '%s' 2.iso` >> PULP_MANIFEST
pulp-admin iso repo create --repo-id $REPO1 --feed file:///tmp/iso
pulp-admin iso repo sync run --repo-id $REPO1
pulp-admin iso repo create --repo-id $REPO2 --feed file:///tmp/iso
pulp-admin iso repo sync run --repo-id $REPO2
echo "blah1" > 1.iso
echo "blah2" > 2.iso
echo 1.iso,`sha256sum 1.iso | awk '{ print $1 }'`,`stat -L -c '%s' 1.iso` > PULP_MANIFEST
echo 2.iso,`sha256sum 2.iso | awk '{ print $1 }'`,`stat -L -c '%s' 2.iso` >> PULP_MANIFEST
pulp-admin iso repo sync run --repo-id $REPO1
pulp-admin iso repo sync run --repo-id $REPO2
pulp-admin iso repo list
</code></pre>
<p>You'll see that the repos have 3 units instead of 2. One of the existing dupe units is removed but the other is not.</p> Pulp - Issue #3090 (CLOSED - CURRENTRELEASE): Uploading an invalid rpm produces an error message:...https://pulp.plan.io/issues/30902017-10-20T15:46:28Zdaviddavis
<p>This is a regression that was originally fixed in <a href="https://pulp.plan.io/issues/2543" class="external">https://pulp.plan.io/issues/2543</a>. See comments below for the commit that broke this.</p>
<p>Steps to reproduce:</p>
<pre><code>touch meerkat-2.1.0.noarch.rpm
pulp-admin rpm repo uploads rpm -f meerkat-2.1.0.noarch.rpm --repo-id zoo
</code></pre>
<p>I'd expect an error message along the lines of "couldn't parse rpm". Instead, the error message is:</p>
<pre><code>Task Failed
The importer yum_importer indicated a failed response when uploading rpm unit to
repository test.
unexpected error occurred importing uploaded file: 'primary'
</code></pre> Pulp - Issue #3069 (CLOSED - CURRENTRELEASE): NotFound exception if consumer is deleted when its ...https://pulp.plan.io/issues/30692017-10-13T17:28:15Zttereshcttereshc@redhat.com
<p>Occasionally, if a user is deleting a large number of hosts, one may fail during the run and be retried. If a host is deleted but its qpid queue is already gone, the delete will fail. Instead, the delete should continue, since it's trying to delete something that isn't there.</p>
<pre><code>no such queue: pulp.agent.6d89102a-c6d7-4d52-920a-f6811f4ddb17
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 87, in dispatch
return handler(request, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/views/decorators.py", line 241, in _auth_decorator
return _verify_auth(self, operation, super_user_only, method, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/views/decorators.py", line 195, in _verify_auth
value = method(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/webservices/views/consumers.py", line 258, in delete
response = manager.unregister(consumer_id)
File "/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/cud.py", line 115, in unregister
agent_consumer.unregister(consumer_id)
File "/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/agent.py", line 56, in unregister
agent.consumer.unregister(context)
File "/usr/lib/python2.7/site-packages/pulp/server/agent/direct/pulpagent.py", line 125, in unregister
consumer.unregister()
File "/usr/lib/python2.7/site-packages/gofer/rmi/stub.py", line 91, in __call__
return self.send(request)
File "/usr/lib/python2.7/site-packages/gofer/common.py", line 234, in sfn
return fn(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/gofer/rmi/stub.py", line 133, in __send
return self.__policy(request)
File "/usr/lib/python2.7/site-packages/gofer/common.py", line 273, in _fn
return fn(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/gofer/rmi/policy.py", line 279, in __call__
return trigger()
File "/usr/lib/python2.7/site-packages/gofer/rmi/policy.py", line 370, in __call__
return self._send()
File "/usr/lib/python2.7/site-packages/gofer/rmi/policy.py", line 337, in _send
data=self._policy.data)
File "/usr/lib/python2.7/site-packages/gofer/messaging/adapter/model.py", line 39, in _fn
return fn(*args, **keywords)
File "/usr/lib/python2.7/site-packages/gofer/messaging/adapter/model.py", line 848, in send
self._impl.send(address, signed, ttl)
File "/usr/lib/python2.7/site-packages/gofer/messaging/adapter/qpid/reliability.py", line 38, in _fn
raise NotFound(*e.args)
NotFound: no such queue: pulp.agent.6d89102a-c6d7-4d52-920a-f6811f4ddb17
</code></pre> RPM Support - Issue #3047 (CLOSED - CURRENTRELEASE): ISO repo doesn't correctly handle updates to...https://pulp.plan.io/issues/30472017-09-28T19:30:47Zdaviddavis
<p>In an earlier issue we handled updates to files in ISO repos when the file was being uploaded or resynced:</p>
<p><a href="https://pulp.plan.io/issues/2773" class="external">https://pulp.plan.io/issues/2773</a></p>
<p>I forgot to handle the case though where the file already exists in pulp and gets added to a repo. Steps to reproduce:</p>
<p>1. Create 2 two repos and sync them both. I recommend using file paths as feeds.<br>
2. Edit an iso in the repo and make sure you update the PULP_MANIFEST file<br>
3. Re-sync repo 1<br>
4. Re-sync repo 2</p>
<p>Expected results:</p>
<p>Each repo has 1 iso unit.</p>
<p>Actual results:</p>
<p>The first repo has 1 unit while the second repo has 2 units. This is because the code that does the association of existing content to repo 2 was never updated in <a class="issue tracker-1 status-11 priority-7 priority-high2 closed" title="Issue: ISO repo does not handle updates to files on manifest during re-sync correctly (CLOSED - CURRENTRELEASE)" href="https://pulp.plan.io/issues/2773">#2773</a>.</p>