Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2020-04-20T17:20:48ZPulp
Planio Migration Plugin - Issue #6531 (CLOSED - WORKSFORME): Advisory conflict happens when syncing a mi...https://pulp.plan.io/issues/65312020-04-20T17:20:48Zipanova@redhat.comipanova@redhat.com
<p>I have migrated a repo from pulp2 that initially got synced from <a href="https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-references-updateinfo/" class="external">https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-references-updateinfo/</a></p>
<p>Then i tried to used the migrated repo's importer aka remote and sync failed</p>
<pre><code>$ http GET $BASE_ADDR/pulp/api/v3/tasks/e3fc9b56-0c3d-4b86-9fca-7f2a8909cc6a/
HTTP/1.1 200 OK
Allow: GET, PATCH, DELETE, HEAD, OPTIONS
Connection: close
Content-Length: 2824
Content-Type: application/json
Date: Mon, 20 Apr 2020 17:06:15 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"child_tasks": [],
"created_resources": [],
"error": {
"description": "Incoming and existing advisories have the same id and timestamp but different and intersecting package lists. At least one of them is wrong. Advisory id: RHEA-2012:0056",
"traceback": " File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 886, in perform_job\n rv = job.perform()\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 664, in perform\n self._result = self._execute()\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 670, in _execute\n return self.func(*self.args, **self.kwargs)\n File \"/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/synchronizing.py\", line 167, in synchronize\n dv.create()\n File \"/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/declarative_version.py\", line 149, in create\n loop.run_until_complete(pipeline)\n File \"/home/vagrant/devel/pulpcore/pulpcore/app/models/repository.py\", line 767, in __exit__\n repository.finalize_new_version(self)\n File \"/home/vagrant/devel/pulp_rpm/pulp_rpm/app/models/repository.py\", line 139, in finalize_new_version\n resolve_advisories(new_version, previous_version)\n File \"/home/vagrant/devel/pulp_rpm/pulp_rpm/app/advisory.py\", line 66, in resolve_advisories\n previous_advisory, added_advisory\n File \"/home/vagrant/devel/pulp_rpm/pulp_rpm/app/advisory.py\", line 139, in resolve_advisory_conflict\n raise AdvisoryConflict(_('Incoming and existing advisories have the same id and '\n"
},
"finished_at": "2020-04-20T17:06:13.537381Z",
"name": "pulp_rpm.app.tasks.synchronizing.synchronize",
"parent_task": null,
"progress_reports": [
{
"code": "parsing.comps",
"done": 3,
"message": "Parsed Comps",
"state": "completed",
"suffix": null,
"total": 3
},
{
"code": "parsing.advisories",
"done": 4,
"message": "Parsed Advisories",
"state": "completed",
"suffix": null,
"total": 4
},
{
"code": "parsing.packages",
"done": 35,
"message": "Parsed Packages",
"state": "completed",
"suffix": null,
"total": 35
},
{
"code": "downloading.metadata",
"done": 5,
"message": "Downloading Metadata Files",
"state": "completed",
"suffix": null,
"total": null
},
{
"code": "downloading.artifacts",
"done": 0,
"message": "Downloading Artifacts",
"state": "completed",
"suffix": null,
"total": null
},
{
"code": "associating.content",
"done": 4,
"message": "Associating Content",
"state": "completed",
"suffix": null,
"total": null
}
],
"pulp_created": "2020-04-20T17:06:09.040275Z",
"pulp_href": "/pulp/api/v3/tasks/e3fc9b56-0c3d-4b86-9fca-7f2a8909cc6a/",
"reserved_resources_record": [
"/pulp/api/v3/repositories/rpm/rpm/574e3ae3-079d-41f9-9b63-e71f01ff6420/",
"/pulp/api/v3/remotes/rpm/rpm/c2d1df25-2f09-4fe1-8a7f-ce768d540e67/"
],
"started_at": "2020-04-20T17:06:09.152171Z",
"state": "failed",
"task_group": null,
"worker": "/pulp/api/v3/workers/4e91fcc4-cc02-482b-8998-6d9d1c45e5bc/"
}
</code></pre> RPM Support - Issue #5831 (CLOSED - WORKSFORME): RPM file arrays sometimes have directive in firs...https://pulp.plan.io/issues/58312019-12-03T20:47:05Ziballou
<p>For example:</p>
<pre><code>["dir", "/lib/modules/3.10.0-1062.el7.x86_64/kernel/sound/usb/", "line6"]
</code></pre>
<p>vs.</p>
<pre><code>["/lib/modules/3.10.0-1062.el7.x86_64/kernel/net/netfilter/", "nft_masq.ko.xz"]
</code></pre>
<p>I've also seen "ghost" in the first index of the array which makes it look like a directive for the %files list (<a href="http://ftp.rpm.org/max-rpm/s1-rpm-inside-files-list-directives.html" class="external">http://ftp.rpm.org/max-rpm/s1-rpm-inside-files-list-directives.html</a>).</p>
<p>It seems like a bug that a file without a directive has a 2-entry array but a file with a directive has a 3 entry array. Perhaps they could all have prefixes?</p> RPM Support - Issue #5305 (CLOSED - WORKSFORME): When 'updateinfo' missing in repomd.xml sync tas...https://pulp.plan.io/issues/53052019-08-20T14:22:22Zppicka
<p>without <data type="updateinfo"> in repomd.xml in repository sync task will fail as we are checking updateinfo url which is not set up : <a href="https://github.com/pulp/pulp_rpm/blob/master/pulp_rpm/app/tasks/synchronizing.py#L299" class="external">https://github.com/pulp/pulp_rpm/blob/master/pulp_rpm/app/tasks/synchronizing.py#L299</a></p>
<p>We should change the way of how we checking record type when sync or use 'null' values to skip conditionals.</p>
<pre><code>"description": "local variable 'updateinfo_url' referenced before assignment",
"traceback": " File \"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py\", line 822, in perform_job\n rv = job.perform()\n File \"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py\", line 605, in perform\n self._result = self._execute()\n File \"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py\", line 611, in _execute\n return self.func(*self.args, **self.kwargs)\n File \"/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/synchronizing.py\", line 73, in synchronize\n dv.create()\n File \"/home/vagrant/devel/pulpcore-plugin/pulpcore/plugin/stages/declarative_version.py\", line 169, in create\n loop.run_until_complete(pipeline)\n File \"/usr/lib64/python3.7/asyncio/base_events.py\", line 568, in run_until_complete\n return future.result()\n File \"/home/vagrant/devel/pulpcore-plugin/pulpcore/plugin/stages/api.py\", line 209, in create_pipeline\n await asyncio.gather(*futures)\n File \"/home/vagrant/devel/pulpcore-plugin/pulpcore/plugin/stages/api.py\", line 43, in __call__\n await self.run()\n File \"/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/synchronizing.py\", line 308, in run\n elif results[0].url == updateinfo_url:\n"
</code></pre> RPM Support - Issue #5085 (CLOSED - WORKSFORME): unable to sync EPEL7 RPM remote: "No declared ar...https://pulp.plan.io/issues/50852019-07-04T15:33:06Zrbicker
<p>Hi Guys</p>
<p>I am unable to sync the Extra Packages for Enterprise Linux (EPEL) 7 remote with a repository. The tasks ends with the error below. I tested different policies (immediate, on_demand) and with or without validation. The result is always the same. Other RPM remotes are syncing fine. Pulp 3 was installed using the ansible installation method.</p>
<p>If you need me to do any further testing please let me know.</p>
<p>Thanks<br>
---</p>
<p>Steps to reproduce:</p>
<pre><code class="text syntaxhl" data-language="text">http POST http://localhost:24817/pulp/api/v3/remotes/rpm/rpm/ name='epel7-x86_64' url='https://dl.fedoraproject.org/pub/epel/7/x86_64/' policy='on_demand'
http POST http://localhost:24817/pulp/api/v3/repositories/ name='epel7-x86_64'
export REMOTE_HREF=$(http :24817/pulp/api/v3/remotes/rpm/rpm/ | jq -r '.results[] | select(.name == "epel7-x86_64") | ._href')
export REPO_HREF=$(http :24817/pulp/api/v3/repositories/ | jq -r '.results[] | select(.name == "epel7-x86_64") | ._href')
http POST :24817${REMOTE_HREF}sync/ repository=$REPO_HREF
</code></pre>
<p>Result:</p>
<pre><code class="json syntaxhl" data-language="json"><span class="p">{</span><span class="w">
</span><span class="nl">"_created"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2019-07-04T14:47:04.436912Z"</span><span class="p">,</span><span class="w">
</span><span class="nl">"_href"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/tasks/1309ad44-3111-4cf4-a4c9-1243295818fb/"</span><span class="p">,</span><span class="w">
</span><span class="nl">"created_resources"</span><span class="p">:</span><span class="w"> </span><span class="p">[],</span><span class="w">
</span><span class="nl">"error"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"code"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"No declared artifact with relative path </span><span class="se">\"\"</span><span class="s2"> for content </span><span class="se">\"</span><span class="s2"><Package: cowsay></span><span class="se">\"</span><span class="s2">"</span><span class="p">,</span><span class="w">
</span><span class="nl">"traceback"</span><span class="p">:</span><span class="w"> </span><span class="s2">" File </span><span class="se">\"</span><span class="s2">/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py</span><span class="se">\"</span><span class="s2">, line 812, in perform_job</span><span class="se">\n</span><span class="s2"> rv = job.perform()</span><span class="se">\n</span><span class="s2"> File </span><span class="se">\"</span><span class="s2">/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py</span><span class="se">\"</span><span class="s2">, line 588, in perform</span><span class="se">\n</span><span class="s2"> self._result = self._execute()</span><span class="se">\n</span><span class="s2"> File </span><span class="se">\"</span><span class="s2">/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py</span><span class="se">\"</span><span class="s2">, line 594, in _execute</span><span class="se">\n</span><span class="s2"> return self.func(*self.args, **self.kwargs)</span><span class="se">\n</span><span class="s2"> File </span><span class="se">\"</span><span class="s2">/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py</span><span class="se">\"</span><span class="s2">, line 67, in synchronize</span><span class="se">\n</span><span class="s2"> dv.create()</span><span class="se">\n</span><span class="s2"> File </span><span class="se">\"</span><span class="s2">/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py</span><span class="se">\"</span><span class="s2">, line 169, in create</span><span class="se">\n</span><span class="s2"> loop.run_until_complete(pipeline)</span><span class="se">\n</span><span class="s2"> File </span><span class="se">\"</span><span class="s2">/usr/lib64/python3.6/asyncio/base_events.py</span><span class="se">\"</span><span class="s2">, line 484, in run_until_complete</span><span class="se">\n</span><span class="s2"> return future.result()</span><span class="se">\n</span><span class="s2"> File </span><span class="se">\"</span><span class="s2">/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py</span><span class="se">\"</span><span class="s2">, line 209, in create_pipeline</span><span class="se">\n</span><span class="s2"> await asyncio.gather(*futures)</span><span class="se">\n</span><span class="s2"> File </span><span class="se">\"</span><span class="s2">/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py</span><span class="se">\"</span><span class="s2">, line 43, in __call__</span><span class="se">\n</span><span class="s2"> await self.run()</span><span class="se">\n</span><span class="s2"> File </span><span class="se">\"</span><span class="s2">/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py</span><span class="se">\"</span><span class="s2">, line 217, in run</span><span class="se">\n</span><span class="s2"> RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch))</span><span class="se">\n</span><span class="s2"> File </span><span class="se">\"</span><span class="s2">/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py</span><span class="se">\"</span><span class="s2">, line 260, in _needed_remote_artifacts</span><span class="se">\n</span><span class="s2"> c=d_content.content))</span><span class="se">\n</span><span class="s2">"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"finished_at"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2019-07-04T14:47:31.442835Z"</span><span class="p">,</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"pulp_rpm.app.tasks.synchronizing.synchronize"</span><span class="p">,</span><span class="w">
</span><span class="nl">"non_fatal_errors"</span><span class="p">:</span><span class="w"> </span><span class="p">[],</span><span class="w">
</span><span class="nl">"parent"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"progress_reports"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"done"</span><span class="p">:</span><span class="w"> </span><span class="mi">5</span><span class="p">,</span><span class="w">
</span><span class="nl">"message"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Downloading and Parsing Metadata"</span><span class="p">,</span><span class="w">
</span><span class="nl">"state"</span><span class="p">:</span><span class="w"> </span><span class="s2">"canceled"</span><span class="p">,</span><span class="w">
</span><span class="nl">"suffix"</span><span class="p">:</span><span class="w"> </span><span class="s2">""</span><span class="p">,</span><span class="w">
</span><span class="nl">"task"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/tasks/1309ad44-3111-4cf4-a4c9-1243295818fb/"</span><span class="p">,</span><span class="w">
</span><span class="nl">"total"</span><span class="p">:</span><span class="w"> </span><span class="mi">5</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"done"</span><span class="p">:</span><span class="w"> </span><span class="mi">206</span><span class="p">,</span><span class="w">
</span><span class="nl">"message"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Downloading Artifacts"</span><span class="p">,</span><span class="w">
</span><span class="nl">"state"</span><span class="p">:</span><span class="w"> </span><span class="s2">"canceled"</span><span class="p">,</span><span class="w">
</span><span class="nl">"suffix"</span><span class="p">:</span><span class="w"> </span><span class="s2">""</span><span class="p">,</span><span class="w">
</span><span class="nl">"task"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/tasks/1309ad44-3111-4cf4-a4c9-1243295818fb/"</span><span class="p">,</span><span class="w">
</span><span class="nl">"total"</span><span class="p">:</span><span class="w"> </span><span class="mi">206</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"done"</span><span class="p">:</span><span class="w"> </span><span class="mi">5502</span><span class="p">,</span><span class="w">
</span><span class="nl">"message"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Associating Content"</span><span class="p">,</span><span class="w">
</span><span class="nl">"state"</span><span class="p">:</span><span class="w"> </span><span class="s2">"canceled"</span><span class="p">,</span><span class="w">
</span><span class="nl">"suffix"</span><span class="p">:</span><span class="w"> </span><span class="s2">""</span><span class="p">,</span><span class="w">
</span><span class="nl">"task"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/tasks/1309ad44-3111-4cf4-a4c9-1243295818fb/"</span><span class="p">,</span><span class="w">
</span><span class="nl">"total"</span><span class="p">:</span><span class="w"> </span><span class="mi">5502</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">],</span><span class="w">
</span><span class="nl">"spawned_tasks"</span><span class="p">:</span><span class="w"> </span><span class="p">[],</span><span class="w">
</span><span class="nl">"started_at"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2019-07-04T14:47:04.512453Z"</span><span class="p">,</span><span class="w">
</span><span class="nl">"state"</span><span class="p">:</span><span class="w"> </span><span class="s2">"failed"</span><span class="p">,</span><span class="w">
</span><span class="nl">"worker"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/workers/22d38b77-2067-4811-a52e-7e90f98d4122/"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre> Pulp - Issue #4679 (CLOSED - WORKSFORME): Custom settings not propogatinghttps://pulp.plan.io/issues/46792019-04-12T08:24:00Ztustvold
<p>I'm trying to configure the S3 storage backend, however, am running into an issue with settings not propogating correctly.</p>
<p>I have set <code>DJANGO_SETTINGS_MODULE</code> to <code>pulpcore.app.settings</code> as an environment variable, and additionally have set <code>PULP_SETTINGS</code> to <code>/var/lib/pulp/settings.yaml</code> which is where I have set the variables. The templated contents of which are below:</p>
<pre><code class="yaml syntaxhl" data-language="yaml"><span class="na">default</span><span class="pi">:</span>
<span class="c1"># Django</span>
<span class="na">SECRET_KEY</span><span class="pi">:</span> <span class="s1">'</span><span class="s">${PULP_SECRET}'</span>
<span class="na">DEBUG</span><span class="pi">:</span> <span class="s">True</span>
<span class="na">CONTENT_HOST</span><span class="pi">:</span> <span class="s1">'</span><span class="s">${CONTENT_HOST}'</span>
<span class="c1"># Redis</span>
<span class="na">REDIS_HOST</span><span class="pi">:</span> <span class="s1">'</span><span class="s">${REDIS_HOST}'</span>
<span class="na">REDIS_PORT</span><span class="pi">:</span> <span class="s">${REDIS_PORT}</span>
<span class="na">REDIS_PASSWORD</span><span class="pi">:</span> <span class="s1">'</span><span class="s">${REDIS_PASSWORD}'</span>
<span class="c1"># Storage</span>
<span class="na">AWS_ACCESS_KEY_ID</span><span class="pi">:</span> <span class="s1">'</span><span class="s">${S3_ACCESS_KEY_ID}'</span>
<span class="na">AWS_SECRET_ACCESS_KEY</span><span class="pi">:</span> <span class="s1">'</span><span class="s">${S3_SECRET_ACCESS_KEY}'</span>
<span class="na">AWS_STORAGE_BUCKET_NAME</span><span class="pi">:</span> <span class="s1">'</span><span class="s">${S3_STORAGE_BUCKET_NAME}'</span>
<span class="na">AWS_DEFAULT_ACL</span><span class="pi">:</span>
<span class="na">AWS_AUTO_CREATE_BUCKET</span><span class="pi">:</span> <span class="s">False</span>
<span class="na">DEFAULT_FILE_STORAGE</span><span class="pi">:</span> <span class="s">storages.backends.s3boto3.S3Boto3Storage</span>
<span class="na">MEDIA_ROOT</span><span class="pi">:</span> <span class="s1">'</span><span class="s">'</span>
<span class="c1"># Postgres</span>
<span class="na">DATABASES</span><span class="pi">:</span>
<span class="na">default</span><span class="pi">:</span>
<span class="na">CONN_MAX_AGE</span><span class="pi">:</span> <span class="m">0</span>
<span class="na">ENGINE</span><span class="pi">:</span> <span class="s">django.db.backends.postgresql_psycopg2</span>
<span class="na">NAME</span><span class="pi">:</span> <span class="s1">'</span><span class="s">${PGSQL_DB}'</span>
<span class="na">USER</span><span class="pi">:</span> <span class="s1">'</span><span class="s">${PGSQL_USER}'</span>
<span class="na">PASSWORD</span><span class="pi">:</span> <span class="s1">'</span><span class="s">${PGSQL_PASSWORD}'</span>
<span class="na">HOST</span><span class="pi">:</span> <span class="s1">'</span><span class="s">${PGSQL_HOST}'</span>
<span class="na">PORT</span><span class="pi">:</span> <span class="s">${PGSQL_PORT}</span>
</code></pre>
<p>The problem appears to be that <a href="https://github.com/pulp/pulpcore/blob/master/pulpcore/app/settings.py#L20" class="external">pulpcore.app.settings</a> is following the explicit approach as described <a href="https://github.com/rochacbruno/dynaconf/blob/master/docs/guides/django.md#explicit-mode" class="external">here</a> but is missing the crucial <code>settings.populate_obj(sys.modules[__name__])</code>. This means that only the settings explicitly mentioned in the file are propogated.</p>
<p><em>Caveat: I am new to django and dynaconf so might be missing something incredibly obvious</em></p> Pulp - Issue #4575 (CLOSED - WORKSFORME): Cannot upload file when using django runserverhttps://pulp.plan.io/issues/45752019-03-25T18:45:17Zamacdona@redhat.comaustin@redhat.com
<pre><code>[vagrant@pulp3-source-fedora28 ~]$ http --form POST localhost:8000/pulp/api/v3/artifacts/ file@~/devel/pulp_python/pulpcore-3.0.0b23-py3-none-any.whl
HTTP/1.1 500 Internal Server Error
Content-Length: 27
Content-Type: text/html
Date: Mon, 25 Mar 2019 18:42:35 GMT
Server: WSGIServer/0.2 CPython/3.6.5
Vary: Cookie
X-Frame-Options: SAMEORIGIN
<h1>Server Error (500)</h1>
pulp: django.request:ERROR: Internal Server Error: /pulp/api/v3/artifacts/
Traceback (most recent call last):
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/base.py", line 126, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/base.py", line 124, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/viewsets.py", line 116, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/views.py", line 495, in dispatch
response = self.handle_exception(exc)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/views.py", line 455, in handle_exception
self.raise_uncaught_exception(exc)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/views.py", line 492, in dispatch
response = handler(request, *args, **kwargs)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/mixins.py", line 19, in create
serializer = self.get_serializer(data=request.data)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/request.py", line 212, in data
self._load_data_and_files()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/request.py", line 275, in _load_data_and_files
self._data, self._files = self._parse()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/request.py", line 350, in _parse
parsed = parser.parse(stream, media_type, self.parser_context)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rest_framework/parsers.py", line 113, in parse
data, files = parser.parse()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/http/multipartparser.py", line 227, in parse
content_length, charset, content_type_extra,
File "/home/vagrant/devel/pulpcore/pulpcore/app/files.py", line 60, in new_file
content_type_extra)
File "/home/vagrant/devel/pulpcore/pulpcore/app/files.py", line 16, in __init__
super().__init__(name, content_type, size, charset, content_type_extra)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/files/uploadedfile.py", line 61, in __init__
file = tempfile.NamedTemporaryFile(suffix='.upload' + ext, dir=settings.FILE_UPLOAD_TEMP_DIR)
File "/usr/lib64/python3.6/tempfile.py", line 549, in NamedTemporaryFile
(fd, name) = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
File "/usr/lib64/python3.6/tempfile.py", line 260, in _mkstemp_inner
fd = _os.open(file, flags, 0o600)
FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/pulp/tmp/tmp9hsz2e4m.upload.whl'
</code></pre>
<p>After this, I killed the django runserver and started the wsgi app via systemctl. Upload worked as expected.</p>
<pre><code>[vagrant@pulp3-source-fedora28 ~]$ http --form POST localhost:8000/pulp/api/v3/artifacts/ file@~/devel/pulp_python/pulpcore-3.0.0b23-py3-none-any.whl
HTTP/1.1 201 Created
Allow: GET, POST, HEAD, OPTIONS
Connection: close
Content-Length: 694
Content-Type: application/json
Date: Mon, 25 Mar 2019 18:42:55 GMT
Location: /pulp/api/v3/artifacts/08bfeac4-fb7a-4b7b-9dbd-a1ed0ae278a5/
Server: gunicorn/19.9.0
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"_created": "2019-03-25T18:42:55.199443Z",
"_href": "/pulp/api/v3/artifacts/08bfeac4-fb7a-4b7b-9dbd-a1ed0ae278a5/",
"file": "artifact/0b/50a064be9e7b20e752399486357df586bdac574fac64e62c3ca50bef449dc8",
"md5": "9d8822eea690bd8765f5b0b01e10ca9c",
"sha1": "da80295413a8a73ab171a2bca90896e8f5a4eb11",
"sha224": "9baf50097129d2f48fe861904f029dabefe4117432c27a57281f4896",
"sha256": "0b50a064be9e7b20e752399486357df586bdac574fac64e62c3ca50bef449dc8",
"sha384": "86caad0492aa3a6829327ff2507500152cf2e5aa91a180305782ccea873242428708b0198c2c3bb9a655412245ad0991",
"sha512": "05130b5ef51ba477393c06e5981c3778775f7a26724579de070aa00c710dcc4033c302098dc4b32c017267571ec285dedf782ed81f54234b9fa6f5cb10fac7ff",
"size": 156352
}
</code></pre> RPM Support - Issue #4502 (CLOSED - WORKSFORME): syncing rhel 6 repo results in error "null value...https://pulp.plan.io/issues/45022019-03-05T16:20:44Zjsherril@redhat.comjsherril@redhat.com
<pre><code> "code": null,
"description": "null value in column \"ref_id\" violates not-null constraint\nDETAIL: Failing row contains (1, https://rhn.redhat.com/errata/RHBA-2013-0387.html, null, RHBA-2013:0387, self, 1).\n",
"traceback": " File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 799, in perform_job\n rv = job.perform()\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 600, in perform\n self._result = self._execute()\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 606, in _execute\n return self.func(*self.args, **self.kwargs)\n File \"/home/vagrant/nfs/pulp_rpm/pulp_rpm/app/tasks/synchronizing.py\", line 67, in synchronize\n dv.create()\n File \"/home/vagrant/devel/pulpcore-plugin/pulpcore/plugin/stages/declarative_version.py\", line 169, in create\n loop.run_until_complete(pipeline)\n File \"/usr/lib64/python3.6/asyncio/base_events.py\", line 468, in run_until_complete\n return future.result()\n File \"/home/vagrant/devel/pulpcore-plugin/pulpcore/plugin/stages/api.py\", line 209, in create_pipeline\n await asyncio.gather(*futures)\n File \"/home/vagrant/devel/pulpcore-plugin/pulpcore/plugin/stages/api.py\", line 43, in __call__\n await self.run()\n File \"/home/vagrant/devel/pulpcore-plugin/pulpcore/plugin/stages/content_stages.py\", line 112, in run\n await self._post_save(batch)\n File \"/home/vagrant/nfs/pulp_rpm/pulp_rpm/app/tasks/synchronizing.py\", line 366, in _post_save\n UpdateReference.objects.bulk_create(update_references_to_save)\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py\", line 82, in manager_method\n return getattr(self.get_queryset(), name)(*args, **kwargs)\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py\", line 465, in bulk_create\n ids = self._batched_insert(objs_without_pk, fields, batch_size)\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py\", line 1149, in _batched_insert\n inserted_id = self._insert(item, fields=fields, using=self.db, return_id=True)\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py\", line 1136, in _insert\n return query.get_compiler(using=using).execute_sql(return_id)\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py\", line 1289, in execute_sql\n cursor.execute(sql, params)\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 68, in execute\n return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 77, in _execute_with_wrappers\n return executor(sql, params, many, context)\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute\n return self.cursor.execute(sql, params)\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/utils.py\", line 89, in __exit__\n raise dj_exc_value.with_traceback(traceback) from exc_value\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 85, in _execute\n return self.cursor.execute(sql, params)\n"
</code></pre> File Support - Issue #4452 (CLOSED - WORKSFORME): fetching a published PULP_MANIFEST throws a 404...https://pulp.plan.io/issues/44522019-02-22T16:56:45Zjsherril@redhat.comjsherril@redhat.com
<p>1. Create a repository<br>
2. Create a file publisher<br>
3. Publish the publisher<br>
4. Create a distribution from the publication<br>
5. Fetch PULP_MANIFEST</p>
<p>Receive a 404. According to the docs a PULP_MANIFEST should be created by default for all file publishes (unless the filename has been configured differently in the publisher).</p> File Support - Issue #3013 (CLOSED - WORKSFORME): Content is not downloaded after switching from ...https://pulp.plan.io/issues/30132017-09-15T00:28:44Zttereshcttereshc@redhat.com
<p>Steps to reproduce:</p>
<ul>
<li>create repo</li>
<li>create importer with on_demand policy</li>
<li>sync</li>
<li>update importer, set policy to immediate</li>
<li>sync again</li>
</ul>
<p>Observe that no files were downloaded.</p>
<p>My guess is that importer <a href="https://github.com/pulp/pulp_file/blob/cac0d02e81de5634df387308fa7f49809f2cdcd2/pulp_file/app/models.py#L152" class="external">looks into <code>pulp_file_filecontent</code> only and builds <code>inventory</code> based on the data from it</a> without checking an existence of corresponding artifact in case of immediate policy.</p> Pulp - Issue #2504 (CLOSED - WORKSFORME): errata applicability fails if errata packagelist doesn'...https://pulp.plan.io/issues/25042016-12-21T17:36:25Zjsherril@redhat.comjsherril@redhat.com
<p>If you sync a repo with an errata that does not have a package list such as this one here:</p>
<p><a href="https://github.com/Katello/runcible/blob/master/test/fixtures/repositories/zoo5/updateinfo.xml#L3-L9" class="external">https://github.com/Katello/runcible/blob/master/test/fixtures/repositories/zoo5/updateinfo.xml#L3-L9</a></p>
<p>and then try to generate applicability for a consumer, you will get an error:</p>
<pre><code>"traceback"=>"Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 488, in __call__
return super(Task, self).__call__(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
return super(PulpTask, self).__call__(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
return self.run(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/applicability.py", line 107, in regenerate_applicability_for_consumers
manager.regenerate_applicability(profile_hash, content_type, profile_id, repo_id)
File "/usr/lib/python2.7/site-packages/pulp/server/managers/consumer/applicability.py", line 257, in regenerate_applicability
profiler_conduit)
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/profilers/yum.py", line 65, in calculate_applicable_units
bound_repo_id, config, conduit)}
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/profilers/yum.py", line 212, in _calculate_applicable_units
unit, profile_lookup_table, available_rpm_nevras)
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/profilers/yum.py", line 314, in _is_errata_applicable
errata_rpms = YumProfiler._get_rpms_from_errata(errata)
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/profilers/yum.py", line 292, in _get_rpms_from_errata
for pkgs in errata.metadata['pkglist']:
TypeError: 'NoneType' object is not iterable
</code></pre>
<p>This used to work (on at least 2.8.7), but no longer does on 2.9.3 an newer.</p> RPM Support - Issue #2280 (CLOSED - WORKSFORME): Error is not reported correctly when size verifi...https://pulp.plan.io/issues/22802016-09-23T15:04:37Zttereshcttereshc@redhat.com
<p>Create and sync DRPM repo with `validate` option enabled for importer and the feed for which size verification of DRPMs will fail, for now it is <a href="https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/" class="external">https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/</a></p>
<p>Tested on Pulp 2.9.3.</p>
<pre><code>$ pulp-admin rpm repo create --repo-id drpm --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/drpm/ --validate true
</code></pre>
<p>Sync task will just succeed, in the logs there will be a traceback.</p>
<pre><code>$ pulp-admin rpm repo sync run --repo-id drpm
+----------------------------------------------------------------------+
Synchronizing Repository [drpm]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[\]
... completed
Downloading repository content...
[-]
[==================================================] 100%
RPMs: 5/5 items
Delta RPMs: 0/4 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[-]
... completed
Cleaning duplicate packages...
[-]
... completed
Task Succeeded
</code></pre>
<pre><code>Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) 'DRPM' object has no attribute 'name'
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) Traceback (most recent call last):
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) File "/usr/lib/python2.7/site-packages/nectar/downloaders/base.py", line 145, in _fire_event_to_listener
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) event_listener_callback(*args, **kwargs)
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/listener.py", line 194, in download_succeeded
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) super(RPMListener, self).download_succeeded(report)
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/listener.py", line 81, in download_succeeded
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) self._verify_size(unit, report)
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/listener.py", line 120, in _verify_size
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) constants.NAME: unit.name,
Sep 22 21:51:15 dev pulp[3891]: nectar.downloaders.base:ERROR: (3891-96512) AttributeError: 'DRPM' object has no attribute 'name'
</code></pre>
<p>Expected result, example of the same error reporting for srpm:</p>
<pre><code>$ pulp-admin rpm repo sync run --repo-id srpm
+----------------------------------------------------------------------+
Synchronizing Repository [srpm]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[\]
... completed
Downloading repository content...
[-]
[==================================================] 100%
RPMs: 3/3 items
Delta RPMs: 0/0 items
... completed
Individual package errors encountered during sync:
Package: test-srpm01
Error: The size did not match the value specified in the
repository metadata.
Package: test-srpm03
Error: The size did not match the value specified in the
repository metadata.
Package: test-srpm02
Error: The size did not match the value specified in the
repository metadata.
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[\]
... completed
Cleaning duplicate packages...
[-]
... completed
Task Succeeded
</code></pre>
<p>JFYI, the importer `--validate` option is enabled by default in Katello <a href="https://github.com/Katello/katello/pull/4747/files" class="external">https://github.com/Katello/katello/pull/4747/files</a></p> Puppet Support - Issue #1676 (CLOSED - WORKSFORME): OOM when orphan remove on large puppet repohttps://pulp.plan.io/issues/16762016-02-15T16:06:45Zcduryeecduryee@redhat.com
<p>I synced down the puppet forge repo and then ran an orphan removal. This resulted in an OOM both times I attempted it.</p>
<p>I only have two repos right now in my Pulp instance, one with puppet forge and one with nine puppet modules.</p>
<p>repro steps:</p>
<ul>
<li>create a puppet repo with feed URL of puppet forge</li>
<li>sync (takes awhile)</li>
<li>run orphan removal</li>
</ul>
<p>My vm has 3GB of mem, fwiw.</p> OSTree Support - Issue #1609 (CLOSED - WORKSFORME): RPM and OSTree content not available over htt...https://pulp.plan.io/issues/16092016-02-01T13:58:35Zplackoplacko@redhat.com
<p>Steps to reproduce:</p>
<pre><code>* create rpm or ostree repository
* publish
* files should be available on https://server.com/pulp/repos or https://server.com/pulp/ostree respectively
</code></pre>
<p>On pulp 2.8 only, works fine for 2.7.</p> Pulp - Issue #649 (CLOSED - WORKSFORME): https://repos.fedorapeople.org/repos/pulp/pulp/rhel-pulp...https://pulp.plan.io/issues/6492015-02-28T23:17:49Zjon@soh.rejon@soh.re
<p>Description of problem:<br>
The .repo file does not work with a fresh Centos 7 box. Specifically $releasever on Centos7 resolves as "7" where as the pulp repo is expecting it to resolve to "7Server", because of this, people are unable to follow the getting started guide on <a href="https://pulp-user-guide.readthedocs.org/en/2.5-release/installation.html" class="external">https://pulp-user-guide.readthedocs.org/en/2.5-release/installation.html</a>.</p>
<p>The fix is, to edit the .repo file from $releasever to "$releasever"Server</p>
<p>Version-Release number of selected component (if applicable):</p>
<p>How reproducible:<br>
100%</p>
<p>Steps to Reproduce:<br>
1. Build a new Centos7 Box<br>
2. Wget the repo<br>
3. Yum update and watch the url fail</p>
<p>Actual results:<br>
<a href="https://repos.fedorapeople.org/repos/pulp/pulp/stable/2/7/x86_64/repodata/repomd.xml" class="external">https://repos.fedorapeople.org/repos/pulp/pulp/stable/2/7/x86_64/repodata/repomd.xml</a>: [Errno 14] HTTPS Error 404 - Not Found</p>
<p>Expected results:<br>
pulp-2-stable | 3.6 kB 00:00:00<br>
(1/2): pulp-2-stable/7/x86_64/group_gz | 521 B 00:00:00<br>
(2/2): pulp-2-stable/7/x86_64/primary_db | 22 kB 00:00:00</p>
<p>Additional info:</p>
<p>Document URL:</p>
<p>Section Number and Name:</p>
<p>Describe the issue:</p>
<p>Suggestions for improvement:</p>
<p>Additional information:</p>
<p>Description of problem:</p>
<p>Version-Release number of selected component (if applicable):</p>
<p>How reproducible:</p>
<p>Steps to Reproduce:<br>
1.<br>
2.<br>
3.</p>
<p>Actual results:</p>
<p>Expected results:</p>
<p>Additional info:</p>
<p>Document URL:</p>
<p>Section Number and Name:</p>
<p>Describe the issue:</p>
<p>Suggestions for improvement:</p>
<p>Additional information:</p>
<p>+ This bug was cloned from <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1178241" class="external">Bugzilla Bug #1178241</a> +</p> Pulp - Issue #573 (CLOSED - WORKSFORME): nodes.conf is blank by default, but the config loader do...https://pulp.plan.io/issues/5732015-02-28T22:39:05Zrbarlow
<p>Description of problem:<br>
I noticed today that I could not run pulp-dev.py -I because my nodes.conf has the ca_path setting commented out.</p>
<p>Version-Release number of selected component (if applicable):<br>
master</p>
<p>How reproducible:<br>
Every time.</p>
<p>Steps to Reproduce:<br>
1. Comment out all of nodes.conf.<br>
2. Run sudo ./pulp-dev -I</p>
<p>Actual results:<br>
<snip><br>
Traceback (most recent call last):<br>
File "<string>", line 2, in <module><br>
File "/home/rbarlow/devel/pulp/nodes/common/pulp_node/resources.py", line 43, in node_configuration<br>
cfg.validate(NODE_SCHEMA)<br>
File "/home/rbarlow/devel/pulp/common/pulp/common/config.py", line 296, in validate<br>
return v.validate(self)<br>
File "/home/rbarlow/devel/pulp/common/pulp/common/config.py", line 360, in validate<br>
s.validate(section)<br>
File "/home/rbarlow/devel/pulp/common/pulp/common/config.py", line 476, in validate<br>
self.validproperty(section, property)<br>
File "/home/rbarlow/devel/pulp/common/pulp/common/config.py", line 494, in validproperty<br>
raise pe<br>
pulp.common.config.PropertyNotFound: Required property "main.ca_path", not found</p>
<p>Expected results:<br>
The ca_path should be allowed to be undefined in nodes.conf.</p>
<p>+ This bug was cloned from <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1150754" class="external">Bugzilla Bug #1150754</a> +</p>