Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-09-02T14:53:23ZPulp
Planio RPM Support - Issue #9335 (CLOSED - DUPLICATE): Huge memory consumption when performing depsolvin...https://pulp.plan.io/issues/93352021-09-02T14:53:23Zdalleydalley@redhat.comRPM Support - Issue #9107 (CLOSED - CURRENTRELEASE): filelists and changelog metadata is not pars...https://pulp.plan.io/issues/91072021-07-17T22:47:54Zoptiz0r
<p>Versions:</p>
<ul>
<li>Foreman 2.5.2</li>
<li>Katello 4.1.1</li>
<li>pulp-rpm-3.13.3</li>
</ul>
<p>Upstream: <a href="http://lon.mirror.rackspace.com/almalinux/8/BaseOS/x86_64/os/repodata/" class="external">http://lon.mirror.rackspace.com/almalinux/8/BaseOS/x86_64/os/repodata/</a>.</p>
<p>I am seeing filelists not being parsed correctly for Alma Linux repositories. Within Katello UI, I have checked a random sample of packages from all of the Alma repositories I have sync'd (18 in total, x86_64, sources and debug-x86_64 for each of baseos, appstream, highavailability, powertools, extras and devel), and none of them list any files. For comparison, CentOS 8 Stream RPMS do list files.</p>
<p>The generated filelist served up by katello contains each of the packages in the repo but with no package contents. For example, the upstream baseos filelist is 2MB whereas the generated filelist for the katello-hosted repo is just 105KB and looks like the below:</p>
<pre><code class="xml syntaxhl" data-language="xml"><span class="cp"><?xml version="1.0" encoding="UTF-8"?></span>
<span class="nt"><filelists</span> <span class="na">xmlns=</span><span class="s">"http://linux.duke.edu/metadata/filelists"</span> <span class="na">packages=</span><span class="s">"1985"</span><span class="nt">></span>
<span class="nt"><package</span> <span class="na">pkgid=</span><span class="s">"59c2172d6e423d8adc3248c1983146492471678afccdfae8ab0f66e18a1aaaa5"</span> <span class="na">name=</span><span class="s">"kernel-debug"</span> <span class="na">arch=</span><span class="s">"x86_64"</span><span class="nt">></span>
<span class="nt"><version</span> <span class="na">epoch=</span><span class="s">"0"</span> <span class="na">ver=</span><span class="s">"4.18.0"</span> <span class="na">rel=</span><span class="s">"305.el8"</span><span class="nt">/></span>
<span class="nt"></package></span>
<span class="nt"><package</span> <span class="na">pkgid=</span><span class="s">"c8120b541261d0ad425f369b1da5eef0aaad5883760ad0e149137027d102207c"</span> <span class="na">name=</span><span class="s">"avahi"</span> <span class="na">arch=</span><span class="s">"i686"</span><span class="nt">></span>
<span class="nt"><version</span> <span class="na">epoch=</span><span class="s">"0"</span> <span class="na">ver=</span><span class="s">"0.7"</span> <span class="na">rel=</span><span class="s">"20.el8"</span><span class="nt">/></span>
<span class="nt"></package></span>
</code></pre>
<p>This is breaking system installs which use the local mirrored content, because packages included in the kickstart base package set have dependencies specified using files, and these are all unsatisfied due to the empty filelists. I have triggered resyncs from upstream mirrors without any change in behaviour.</p>
<p>May be related to similar-looking issue <a class="issue tracker-1 status-11 priority-6 priority-default closed" title="Issue: Filelists and changlogs not always parsed properly (CLOSED - CURRENTRELEASE)" href="https://pulp.plan.io/issues/8955">#8955</a> ?</p> RPM Support - Issue #8944 (CLOSED - CURRENTRELEASE): Cannot sync RHEL6 repos from cdn.redhat.comhttps://pulp.plan.io/issues/89442021-06-21T22:37:08Zsskracic@redhat.comsskracic@redhat.com
<ul>
<li>a containerized pulp based on pulp-ci-centos image with:</li>
</ul>
<p>pulpcore-3.13.0
pulp-rpm-3.13.0</p>
<p>I am not sure whether it's a Pulp bug or a CDN data error, but RHEL6 i386 and x86_64
repo syncing repeatedly fails with this error. The repositories are synced with mirror=True
and the distribution is set to point to repository (so that the latest repo is always distributed).</p>
<pre><code>Repo: Red Hat Enterprise Linux 6 Server from RHUI (RPMs) (6Server-i386)
Start Time: 2021-06-21 22:24:04
Finish Time: 2021-06-21 22:28:18
Elapsed Time: 0:04:13.581718
Result: Error
Exception: Package id from primary metadata (42003b3621ebf25f9b5fe25b0d2c4f2024958a9d), does not match package id from filelists, other metadata (dcacfd1c930cba9ee112c1055e7a1e94bedd7f3a)
Traceback: File "/usr/local/lib/python3.6/site-packages/rq/worker.py", line 1013, in perform_job
rv job.perform()
File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 709, in perform
self._result = self._execute()
File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 732, in _execute
result = self.func(*self.args, **self.kwargs)
File "/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py", line 395, in synchronize
version = dv.create()
File "/src/pulpcore/pulpcore/plugin/stages/declarative_version.py", line 149, in create
loop.run_until_complete(pipeline)
File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
return future.result()
File "/src/pulpcore/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
await asyncio.gather(*futures)
File "/src/pulpcore/pulpcore/plugin/stages/api.py", line 43, in __call__
await self.run()
File "/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py", line 674, in run
await self.parse_repository_metadata(repomd, repomd_files, file_extension)
File "/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py", line 719, in parse_repository_metadata
file_extension=file_extension,
File "/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py", line 987, in parse_packages
).format(pkgid, pkgid_extra)
</code></pre> RPM Support - Issue #7786 (CLOSED - WORKSFORME): Pulp_rpm sync reports "completed" but does not u...https://pulp.plan.io/issues/77862020-11-04T07:51:35ZAant
<p>This is a copy of <a href="https://pulp.plan.io/issues/7535?next_issue_id=7534" class="external">https://pulp.plan.io/issues/7535?next_issue_id=7534</a>. The issue persists.
Even if I delete and recreate the publication and distribution after synchronization I still experience this issue.
This is a very severe bug as it makes me believe that my systems are up to date while they are not.</p>
<a name="What-would-be-the-proper-workflow-to-resynchronise-a-repo-after-it-is-created"></a>
<h2 >What would be the proper workflow to resynchronise a repo after it is created?<a href="#What-would-be-the-proper-workflow-to-resynchronise-a-repo-after-it-is-created" class="wiki-anchor">¶</a></h2>
<p>I have set up Pulpcore v3.6.3 with Pulp_rpm v3.6.2 - later updated both to 3.7.0.
I created the following remote:</p>
<pre><code>{
"pulp_href": "/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/",
"pulp_created": "2020-08-18T08:47:19.183836Z",
"name": "centos8_base",
"url": "http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/",
"ca_cert": null,
"client_cert": null,
"client_key": null,
"tls_validation": true,
"proxy_url": "http://<proxy server>",
"username": null,
"password": null,
"pulp_last_updated": "2020-08-18T08:47:19.183855Z",
"download_concurrency": 10,
"policy": "immediate",
"sles_auth_token": null
}
</code></pre>
<p>and this repository:</p>
<pre><code>{
"pulp_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/",
"pulp_created": "2020-08-18T08:47:17.704104Z",
"versions_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/versions/",
"latest_version_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/versions/1/",
"name": "centos8_base",
"description": null,
"remote": null,
"metadata_signing_service": null,
"retain_package_versions": 0
}
</code></pre>
<p>If I sync, it reports success:</p>
<pre><code>http --auth admin --auth-type basic :9000/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/sync/ remote=/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/
http: password for admin@localhost:9000:
HTTP/1.1 202 Accepted
Allow: POST, OPTIONS
Connection: close
Content-Length: 67
Content-Type: application/json
Date: Mon, 21 Sep 2020 08:10:11 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"task": "/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/"
}
</code></pre>
<p>Let's take a closer look at the task:</p>
<pre><code> http --auth admin --auth-type basic :9000/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/
http: password for admin@localhost:9000:
HTTP/1.1 200 OK
Allow: GET, PATCH, DELETE, HEAD, OPTIONS
Connection: close
Content-Length: 743
Content-Type: application/json
Date: Mon, 21 Sep 2020 08:10:29 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"child_tasks": [],
"created_resources": [],
"error": null,
"finished_at": "2020-09-21T08:10:11.585792Z",
"name": "pulp_rpm.app.tasks.synchronizing.synchronize",
"parent_task": null,
"progress_reports": [
{
"code": "optimizing.sync",
"done": 1,
"message": "Optimizing Sync",
"state": "completed",
"suffix": null,
"total": null
}
],
"pulp_created": "2020-09-21T08:10:11.257796Z",
"pulp_href": "/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/",
"reserved_resources_record": [
"/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/",
"/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/"
],
"started_at": "2020-09-21T08:10:11.507912Z",
"state": "completed",
"task_group": null,
"worker": "/pulp/api/v3/workers/80ca382e-fdec-4eef-94b4-6c1c67433511/"
}
</code></pre>
<p>Let's use this repo on the same machine:</p>
<pre><code>cat /etc/yum.repos.d/CentOS-Base.repo
[Base]
name=CentOS-$releasever - Base
enabled=1
baseurl=https://<myurl>/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
</code></pre>
<p>Now if I do <code>yum check-update</code> it does not find any package to update.
If I change <code>baseurl=</code> to <code>http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/</code> (which is the same url I have in the remote) and I do <code>yum check-update</code>, then it finds 28 outdated packages. How is it possible?</p> RPM Support - Issue #7535 (CLOSED - WORKSFORME): Pulp_rpm sync reports "completed" but does not u...https://pulp.plan.io/issues/75352020-09-21T08:28:31ZAant
<p>I have set up Pulpcore v3.6.3 with Pulp_rpm v3.6.2.
I created the following remote:</p>
<pre><code>{
"pulp_href": "/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/",
"pulp_created": "2020-08-18T08:47:19.183836Z",
"name": "centos8_base",
"url": "http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/",
"ca_cert": null,
"client_cert": null,
"client_key": null,
"tls_validation": true,
"proxy_url": "http://<proxy server>",
"username": null,
"password": null,
"pulp_last_updated": "2020-08-18T08:47:19.183855Z",
"download_concurrency": 10,
"policy": "immediate",
"sles_auth_token": null
}
</code></pre>
<p>and this repository:</p>
<pre><code>{
"pulp_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/",
"pulp_created": "2020-08-18T08:47:17.704104Z",
"versions_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/versions/",
"latest_version_href": "/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/versions/1/",
"name": "centos8_base",
"description": null,
"remote": null,
"metadata_signing_service": null,
"retain_package_versions": 0
}
</code></pre>
<p>If I sync, it reports success:</p>
<pre><code>http --auth admin --auth-type basic :9000/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/sync/ remote=/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/
http: password for admin@localhost:9000:
HTTP/1.1 202 Accepted
Allow: POST, OPTIONS
Connection: close
Content-Length: 67
Content-Type: application/json
Date: Mon, 21 Sep 2020 08:10:11 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"task": "/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/"
}
</code></pre>
<p>Let's take a closer look at the task:</p>
<pre><code> http --auth admin --auth-type basic :9000/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/
http: password for admin@localhost:9000:
HTTP/1.1 200 OK
Allow: GET, PATCH, DELETE, HEAD, OPTIONS
Connection: close
Content-Length: 743
Content-Type: application/json
Date: Mon, 21 Sep 2020 08:10:29 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"child_tasks": [],
"created_resources": [],
"error": null,
"finished_at": "2020-09-21T08:10:11.585792Z",
"name": "pulp_rpm.app.tasks.synchronizing.synchronize",
"parent_task": null,
"progress_reports": [
{
"code": "optimizing.sync",
"done": 1,
"message": "Optimizing Sync",
"state": "completed",
"suffix": null,
"total": null
}
],
"pulp_created": "2020-09-21T08:10:11.257796Z",
"pulp_href": "/pulp/api/v3/tasks/655d2e7b-2783-4496-a851-3bcd8d5e442e/",
"reserved_resources_record": [
"/pulp/api/v3/remotes/rpm/rpm/d8fb524c-e73b-4afc-ba2e-05eea7f5b6a8/",
"/pulp/api/v3/repositories/rpm/rpm/7696b222-3788-4754-b4a6-747e64228d7a/"
],
"started_at": "2020-09-21T08:10:11.507912Z",
"state": "completed",
"task_group": null,
"worker": "/pulp/api/v3/workers/80ca382e-fdec-4eef-94b4-6c1c67433511/"
}
</code></pre>
<p>Let's use this repo on the same machine:</p>
<pre><code>cat /etc/yum.repos.d/CentOS-Base.repo
[Base]
name=CentOS-$releasever - Base
enabled=1
baseurl=https://<myurl>/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
</code></pre>
<p>Now if I do <code>yum check-update</code> it does not find any package to update.
If I change <code>baseurl=</code> to <code>http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/</code> (which is the same url I have in the remote) and I do <code>yum check-update</code>, then it finds 28 outdated packages. How is it possible?</p> RPM Support - Issue #6861 (CLOSED - CURRENTRELEASE): Migration from 3.2.0 to 3.3.z is brokenhttps://pulp.plan.io/issues/68612020-05-29T16:37:58Zdalleydalley@redhat.com
<p>In trying to resolve the migration issues between 3.3.0 and 3.3.1, we accidentally introduced a new blocking issue between 3.2.0 and 3.3.z.</p>
<p>Both migration 0005 and 0009 need to be modified, again, such that they both specify that 'last_sync_revision_number' is nullable.</p> RPM Support - Issue #6743 (CLOSED - CURRENTRELEASE): Installations migrating from 3.3.0 to 3.3.1 ...https://pulp.plan.io/issues/67432020-05-14T19:27:32Zdalleydalley@redhat.com
<p>By modifying a prior migration, we accidentally created a mismatch between the schema we expect and the schema Django seems to interpret and enforce, presumably due to some kind of optimization being employed.</p>
<p>It seems that because Django sees that migration 0005 made the 'last_sync_revision_number' column nullable (it didn't, it was modified afterwards) [0], it ignores when migration 0009 tells Django to perform that migration later. Thus the 'last_sync_revision_number' column remains non-nullable.</p>
<p>This impacts Pulp users by throwing the following error when creating new RPM repositories.</p>
<pre><code>django.db.utils.IntegrityError: null value in column "last_sync_revision_number" violates not-null constraint
</code></pre>
<p>[0] <a href="https://github.com/pulp/pulp_rpm/pull/1694/files#diff-8882c765a6805ada6c8a8bafe3e3e5b8" class="external">https://github.com/pulp/pulp_rpm/pull/1694/files#diff-8882c765a6805ada6c8a8bafe3e3e5b8</a></p> Pulp - Issue #6438 (CLOSED - WORKSFORME): Pulp 3.2.1 not able to sync to redhathttps://pulp.plan.io/issues/64382020-04-02T13:47:18Zbinlinf0
<p>After upgrading from 3.1 to 3.2.1, we no longer able to sync to redhat repo. Below is the errors:</p>
<p>"error": {
"description": "[PEM: NO_START_LINE] no start line (_ssl.c:3947)",
"traceback": " File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rq/worker.py", line 884, in perform_job\n rv = job.perform()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rq/job.py", line 664, in perform\n self._result = self._execute()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rq/job.py", line 670, in _execute\n return self.func(*self.args, **self.kwargs)\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 126, in synchronize\n treeinfo = get_treeinfo_data(remote)\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulp_rpm/app/kickstart/treeinfo.py", line 21, in get_treeinfo_data\n downloader = remote.get_downloader(url=urljoin(remote_url, namespace))\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/app/models/repository.py", line 287, in get_downloader\n return self.download_factory.build(url, **kwargs)\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/app/models/repository.py", line 245, in download_factory\n self._download_factory = DownloaderFactory(self)\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/download/factory.py", line 68, in <strong>init</strong>\n self._session = self._make_aiohttp_session_from_remote()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/download/factory.py", line 85, in _make_aiohttp_session_from_remote\n sslcontext = ssl.create_default_context(cadata=self._remote.ca_cert)\n File "/opt/python/3.7.3/lib64/python3.7/ssl.py", line 573, in create_default_context\n context.load_verify_locations(cafile, capath, cadata)\n"</p>
<p>The works fine before updating to 3.2.1. We also tried overwrite all cert in pulp 3.2.1 and it didn't help.</p> Pulp - Issue #6154 (CLOSED - CURRENTRELEASE): Redirect to S3 in pulp-content-app is broken in 3.1.0https://pulp.plan.io/issues/61542020-02-13T14:29:13Zosapryki
<p>pulp-content-app returns redirect to invalid S3 URL:</p>
<pre><code>https://bucket.s3.amazonaws.com/artifact/xx/xxx?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=********&X-Amz-Date=20200213T141229Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=********?response-content-disposition=attachment;+filename=pulp-artifact-filename
</code></pre>
<p><code>response-content-disposition</code> is attached to the URL that already has query string which gives as a result invalid URL.</p>
<p>Please refer to for the correct example of setting Response-Content-Disposition parameter with django-storages:</p>
<p><a href="https://github.com/ansible/galaxy/blob/devel/galaxy/api/download/views.py#L64" class="external">https://github.com/ansible/galaxy/blob/devel/galaxy/api/download/views.py#L64</a></p> Pulp - Issue #5673 (CLOSED - CURRENTRELEASE): Resource reservations are not cleaned up if worker ...https://pulp.plan.io/issues/56732019-11-06T17:29:15Zosapryki
<p>It pulp worker is killed while executing task that has reserved resources, resources are not cleaned up.<br>
All subsequent task that use any of the reserved resources are assigned to the same worker (which is dead).</p>
<p>Steps to reproduce:</p>
<p>1. Spawn import_collection task from pulp_ansible (T1).<br>
2. While task is running kill worker (W1).<br>
3. Start another worker (W2)<br>
4. Spawn import_collection task from pulp_ansible (T2)</p>
<p>Expected behavior:</p>
<p>Task T2 is assigned on worker W2 or Cancelled if assigned on W1 before cleanup is performed.</p>
<p>Actual behavior:</p>
<p>Task T2 is assigned on worker W1 and remains in waiting state forever.<br>
Resources are not cleaned up.</p>
<p>Environment:</p>
<p>pulpcore + pulp_ansible.<br>
1 worker</p> Pulp - Issue #4929 (CLOSED - WORKSFORME): Pulp 2.18 having issues to start workers celery and com...https://pulp.plan.io/issues/49292019-06-07T04:36:54Zymadav
<p>Team,</p>
<p>We upgraded pulp version from 2.12 to 2.18 and all the services are up and running but,worker services in worker server are not able to start and it is failing to communicate to rabbitmq server.we have below package versions installed in worker for celery and pulp,also posting the pulp errors.Please help us to fix this as this is production we are running out of the time.uploaded the message file for reference,please let me know if anything else is required.</p>
<p>We have earlier python2-celery-4.2.1-3.el7.noarch version installed,even with that we were unable to start the services,so using python2-celery-4.0.2-7.el7.noarch.Server is running with RHEL 7.6</p>
<p>Mostly i see these errors</p>
<p>Jun 07 04:17:21 ip-10-12-111-237. celery[319]: from billiard.compat import get_fdmax, close_open_fds<br>
Jun 07 04:17:21 ip-10-12-111-237. celery[319]: ImportError: cannot import name get_fdmax</p>
<p>Installed packaged<br>
rpm <del>qa | grep -i celery<br>
python2-celery-4.0.2-7.el7.noarch<br>
[ymadav@ip-10-12-111-237 ~]$ rpm -qa | egrep -i "amqp|celery|billiard|kombu"<br>
python2-celery-4.0.2-7.el7.noarch<br>
python2-kombu-4.0.2-11.el7.noarch<br>
python-gofer-amqp-2.12.5-1.el7.noarch<br>
python2-amqp-2.2.2-5.el7.noarch<br>
python2-billiard-3.5.0.3-4.el7.x86_64<br>
[ymadav@ip-10-12-111-237 ~]$ rpm -qa | grep -i pulp</del>*<br>
python-pulp-python-common-2.0.3-1.el7.noarch<br>
python-pulp-rpm-common-2.18.1-1.el7.noarch<br>
pulp-rpm-plugins-2.18.1-1.el7.noarch<br>
python-pulp-puppet-common-2.18.1-1.el7.noarch<br>
pulp-selinux-2.18.1-2.el7.noarch<br>
pulp-puppet-tools-2.18.1-1.el7.noarch<br>
python-pulp-manifest-2.16.4-1.el7.noarch<br>
python-pulp-docker-common-3.2.2-1.el7.noarch<br>
pulp-docker-plugins-3.2.2-1.el7.noarch<br>
pulp-python-plugins-2.0.3-1.el7.noarch<br>
python-pulp-deb-common-1.8.0-1.el7.noarch<br>
python-pulp-repoauth-2.18.1-2.el7.noarch<br>
python2-solv-0.6.34-2.pulp.el7.x86_64<br>
pulp-deb-plugins-1.8.0-1.el7.noarch<br>
python-pulp-ostree-common-1.4.0-1.el7.noarch<br>
pulp-puppet-plugins-2.18.1-1.el7.noarch<br>
python-pulp-oid_validation-2.18.1-2.el7.noarch<br>
pulp-server-2.18.1-2.el7.noarch<br>
python-pulp-common-2.18.1-2.el7.noarch<br>
libsolv-0.6.34-2.pulp.el7.x86_64<br>
pulp-ostree-plugins-1.4.0-1.el7.noarch</p>
<pre><code>sudo systemctl list-units | grep -i pulp
● pulp_worker-0.service loaded failed failed Pulp Worker #0
● pulp_worker-1.service loaded failed failed Pulp Worker #1
● pulp_worker-2.service loaded failed failed Pulp Worker #2
● pulp_worker-3.service loaded failed failed Pulp Worker #3
pulp_workers.service loaded active exited Pulp Celery Workers
</code></pre>
<p>Jun 07 04:17:21 ip-10-12-111-237. celery[319]: File "/usr/lib/python2.7/site-packages/celery/utils/log.py", line 19, in <module><br>
Jun 07 04:17:21 ip-10-12-111-237. celery[319]: from .term import colored<br>
Jun 07 04:17:21 ip-10-12-111-237. celery[319]: File "/usr/lib/python2.7/site-packages/celery/utils/term.py", line 11, in <module><br>
Jun 07 04:17:21 ip-10-12-111-237. celery[319]: from celery.platforms import isatty<br>
Jun 07 04:17:21 ip-10-12-111-237. celery[319]: File "/usr/lib/python2.7/site-packages/celery/platforms.py", line 21, in <module><br>
Jun 07 04:17:21 ip-10-12-111-237. celery[319]: from billiard.compat import get_fdmax, close_open_fds<br>
Jun 07 04:17:21 ip-10-12-111-237. celery[319]: ImportError: cannot import name get_fdmax<br>
Jun 07 04:17:21 ip-10-12-111-237. systemd[1]: pulp_worker-0.service: main process exited, code=exited, status=1/FAILURE<br>
Jun 07 04:17:21 ip-10-12-111-237. systemd[1]: Unit pulp_worker-0.service entered failed state.<br>
Jun 07 04:17:21 ip-10-12-111-237. systemd[1]: pulp_worker-0.service failed.<br>
<img src="https://pulp.plan.io/attachments/download/518583/clipboard-201906070629-xt6og.png" alt=""></p>
<p>Thanks,<br>
-Yash</p> Pulp - Issue #4226 (CLOSED - DUPLICATE): Upgrade from 2.17 to 2.18 failedhttps://pulp.plan.io/issues/42262018-12-05T07:19:37ZPoil
<p>Hi,</p>
<p>Nothing in the release note about how to upgrade, so I've first only do a "yum update"<br>
When starting the service I have this error</p>
<pre><code>Dec 5 07:14:53 repos pulp: pulp.server.webservices.application:ERROR: (5296-72992) The database has not been migrated to the current version. Run pulp-manage-db and restart the application.
Dec 5 07:14:53 repos pulp: pulp.server.webservices.application:ERROR: (5296-72992) Traceback (most recent call last):
Dec 5 07:14:53 repos pulp: pulp.server.webservices.application:ERROR: (5296-72992) File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 111, in wsgi_application
Dec 5 07:14:53 repos pulp: pulp.server.webservices.application:ERROR: (5296-72992) _initialize_web_services()
Dec 5 07:14:53 repos pulp: pulp.server.webservices.application:ERROR: (5296-72992) File "/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py", line 74, in _initialize_web_services
Dec 5 07:14:53 repos pulp: pulp.server.webservices.application:ERROR: (5296-72992) migration_models.check_package_versions()
Dec 5 07:14:53 repos pulp: pulp.server.webservices.application:ERROR: (5296-72992) File "/usr/lib/python2.7/site-packages/pulp/server/db/migrate/models.py", line 314, in check_package_versions
Dec 5 07:14:53 repos pulp: pulp.server.webservices.application:ERROR: (5296-72992) raise Exception(error_message)
Dec 5 07:14:53 repos pulp: pulp.server.webservices.application:ERROR: (5296-72992) InitializationException: The database has not been migrated to the current version. Run pulp-manage-db and restart the application
</code></pre>
<p>So I've run "sudo -u apache pulp-manage-db"<br>
And I've this error</p>
<pre><code>Applying pulp.server.db.migrations version 29
*******************************************************************************
Applying migration pulp.server.db.migrations.0029_applicability_schema_change failed.
Halting migrations due to a migration failure.
command SON([('dropIndexes', u'repo_profile_applicability'), ('index', 'profile_hash_-1_repo_id_-1')]) on namespace pulp_database.$cmd failed: index not found with name [profile_hash_-1_repo_id_-1]
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 239, in main
return _auto_manage_db(options)
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 306, in _auto_manage_db
migrate_database(options)
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 135, in migrate_database
update_current_version=not options.test)
File "/usr/lib/python2.7/site-packages/pulp/server/db/migrate/models.py", line 189, in apply_migration
migration.migrate()
File "/usr/lib/python2.7/site-packages/pulp/server/db/migrations/0029_applicability_schema_change.py", line 52, in migrate
rpa_collection.drop_index("profile_hash_-1_repo_id_-1")
File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 1456, in drop_index
allowable_errors=["ns not found"])
File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 205, in _command
read_concern=read_concern)
File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 211, in command
read_concern)
File "/usr/lib64/python2.7/site-packages/pymongo/network.py", line 100, in command
helpers._check_command_response(response_doc, msg, allowable_errors)
File "/usr/lib64/python2.7/site-packages/pymongo/helpers.py", line 196, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
OperationFailure: command SON([('dropIndexes', u'repo_profile_applicability'), ('index', 'profile_hash_-1_repo_id_-1')]) on namespace pulp_database.$cmd failed: index not found with name [profile_hash_-1_repo_id_-1]
</code></pre> Pulp - Issue #3611 (CLOSED - NOTABUG): Pulp Data Migrated but failed to access through URLhttps://pulp.plan.io/issues/36112018-04-27T04:32:27Zrams
<p>Hi,</p>
<p>We migrated pulp-server from NTT env to AWS env.<br>
We bcaked up data in following way<br>
- /var/lib/pulp: tar this data and transferred<br>
-/var/lib/mongodb - tar this data and transferred</p>
<p>on AWS instance we installed pulp-server (2.6.2), mongo db (version 2.4.14) and pyhton (2.6.6)<br>
untar the content into same file system as in previous environment. Started all services.<br>
Ran sudo -u apache pulp-manage-db command on empty db<br>
then restored data</p>
<p>but not able to access repositories via web browser</p>
<p>httpd, qpidd, pulp_workers, pulp_celerybeat, pulp_resource_manager serivces are up and running</p> Python Support - Issue #2334 (CLOSED - NOTABUG): Python egg uploads failhttps://pulp.plan.io/issues/23342016-10-13T15:21:02ZIchimonji10jerebear@protonmail.com
<p>Let's say one decides to upload a Python egg with pulp-admin. The following script for doing so succeeds:</p>
<pre><code class="bash syntaxhl" data-language="bash">wget https://repos.fedorapeople.org/pulp/pulp/fixtures/python/packages/source/s/shelf-reader/shelf-reader-0.1.tar.gz
pulp-admin python repo create <span class="nt">--repo-id</span> foo
pulp-admin python repo upload <span class="nt">--repo-id</span> foo <span class="nt">--file</span> shelf-reader-0.1.tar.gz
</code></pre>
<p>On the other hand, let's say one uploads directly via the API. Doing so is a four-step process:</p>
<p>1. Create an upload request.<br>
2. Make one or more PUT requests, sending a chunk of the file each time.<br>
3. Import the file into a repository.<br>
4. Close the upload request.</p>
<p>This has worked since at least Pulp 2.8. Unfortunately, the code for accomplishing step 3 has changed in Pulp 2.11. A POST request to <code>/{repo_href}/actions/import_upload/</code> with the following body will succeed in Pulp 2.8, 2.9 and 2.10:</p>
<pre><code class="json syntaxhl" data-language="json"><span class="p">{</span><span class="w">
</span><span class="nl">"unit_key"</span><span class="p">:</span><span class="w"> </span><span class="p">{},</span><span class="w">
</span><span class="nl">"unit_type_id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"python_package"</span><span class="p">,</span><span class="w">
</span><span class="nl">"upload_id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"..."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre>
<p>In Pulp 2.11, this request will instead cause Pulp to respond with the following failed task:</p>
<pre><code>{'_href': '/pulp/api/v2/tasks/37de3871-e878-4abc-9c6f-0d39c86cd4bb/',
'_id': {'$oid': '57ff9b425137beaefc1f9d0b'},
'_ns': 'task_status',
'error': {'code': 'PLP0000',
'data': {},
'description': 'Pulp exception occurred: PulpExecutionException',
'sub_errors': []},
'exception': None,
'finish_time': '2016-10-13T14:33:38Z',
'id': '57ff9b425137beaefc1f9d0b',
'progress_report': {},
'queue': 'reserved_resource_worker-0@localhost.localdomain.dq',
'result': None,
'spawned_tasks': [],
'start_time': '2016-10-13T14:33:38Z',
'state': 'error',
'tags': ['pulp:repository:a962599e-2dfd-4560-8dfa-d6d391d141fa',
'pulp:action:import_upload'],
'task_id': '37de3871-e878-4abc-9c6f-0d39c86cd4bb',
'task_type': 'pulp.server.managers.content.upload.import_uploaded_unit',
'traceback': 'Traceback (most recent call last):\n'
' File "/usr/lib/python2.7/site-packages/celery/app/trace.py", '
'line 240, in trace_task\n'
' R = retval = fun(*args, **kwargs)\n'
' File '
'"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", '
'line 488, in __call__\n'
' return super(Task, self).__call__(*args, **kwargs)\n'
' File '
'"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", '
'line 103, in __call__\n'
' return super(PulpTask, self).__call__(*args, **kwargs)\n'
' File "/usr/lib/python2.7/site-packages/celery/app/trace.py", '
'line 437, in __protected_call__\n'
' return self.run(*args, **kwargs)\n'
' File '
'"/usr/lib/python2.7/site-packages/pulp/server/managers/content/upload.py", '
'line 218, in import_uploaded_unit\n'
' unit_metadata, file_path, conduit, call_config)\n'
' File '
'"/usr/lib/python2.7/site-packages/pulp_python/plugins/importers/importer.py", '
'line 191, in upload_unit\n'
' new_file_path = os.path.join(working_dir, '
"unit_key['filename'])\n"
'PulpExecutionException: Pulp exception occurred: '
'PulpExecutionException\n',
'worker_name': 'reserved_resource_worker-0@localhost.localdomain'}
</code></pre>
<p>Here's a snippet from journalctl:</p>
<pre><code>Oct 13 10:33:38 localhost.localdomain pulp[7917]: pulp.server.managers.content.upload:ERROR: (7917-13792) Error from the importer while importing uploaded unit to repository [a962599e-2dfd-4560-8dfa-d6d391d141fa]
Oct 13 10:33:38 localhost.localdomain pulp[7917]: pulp.server.managers.content.upload:ERROR: (7917-13792) Traceback (most recent call last):
Oct 13 10:33:38 localhost.localdomain pulp[7917]: pulp.server.managers.content.upload:ERROR: (7917-13792) File "/usr/lib/python2.7/site-packages/pulp/server/managers/content/upload.py", line 218, in import_uploaded_unit
Oct 13 10:33:38 localhost.localdomain pulp[7917]: pulp.server.managers.content.upload:ERROR: (7917-13792) unit_metadata, file_path, conduit, call_config)
Oct 13 10:33:38 localhost.localdomain pulp[7917]: pulp.server.managers.content.upload:ERROR: (7917-13792) File "/usr/lib/python2.7/site-packages/pulp_python/plugins/importers/importer.py", line 191, in upload_unit
Oct 13 10:33:38 localhost.localdomain pulp[7917]: pulp.server.managers.content.upload:ERROR: (7917-13792) new_file_path = os.path.join(working_dir, unit_key['filename'])
Oct 13 10:33:38 localhost.localdomain pulp[7917]: pulp.server.managers.content.upload:ERROR: (7917-13792) KeyError: 'filename'
Oct 13 10:33:38 localhost.localdomain pulp[7917]: pulp.server.async.tasks:INFO: Task failed : [fe698de5-9172-4994-ad37-d0c14188b505]
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) Task pulp.server.managers.content.upload.import_uploaded_unit[fe698de5-9172-4994-ad37-d0c14188b505] raised unexpected: PulpExecutionException(KeyError('filename',),)
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) Traceback (most recent call last):
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) R = retval = fun(*args, **kwargs)
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 488, in __call__
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) return super(Task, self).__call__(*args, **kwargs)
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) return super(PulpTask, self).__call__(*args, **kwargs)
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) return self.run(*args, **kwargs)
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) File "/usr/lib/python2.7/site-packages/pulp/server/managers/content/upload.py", line 218, in import_uploaded_unit
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) unit_metadata, file_path, conduit, call_config)
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) File "/usr/lib/python2.7/site-packages/pulp_python/plugins/importers/importer.py", line 191, in upload_unit
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) new_file_path = os.path.join(working_dir, unit_key['filename'])
Oct 13 10:33:38 localhost.localdomain pulp[7629]: celery.worker.job:ERROR: (7629-13792) PulpExecutionException: Pulp exception occurred: PulpExecutionException
</code></pre>
<p>The culprit seems to be that Pulp now expects the following POST request body:</p>
<pre><code class="json syntaxhl" data-language="json"><span class="p">{</span><span class="w">
</span><span class="nl">"unit_key"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="nl">"filename"</span><span class="p">:</span><span class="w"> </span><span class="s2">"shelf-reader-0.1.tar.gz"</span><span class="p">},</span><span class="w">
</span><span class="nl">"unit_type_id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"python_package"</span><span class="p">,</span><span class="w">
</span><span class="nl">"upload_id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"..."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre>
<p>This change is backward-incompatible. It also seems unnecessary, as the filename can be derived from the package metadata.</p>
<p>If this information is needed when uploading packages, it should be documented in one of the following locations:</p>
<ul>
<li><a href="http://docs.pulpproject.org/dev-guide/integration/rest-api/content/upload.html#import-into-a-repository" class="external">http://docs.pulpproject.org/dev-guide/integration/rest-api/content/upload.html#import-into-a-repository</a></li>
<li><a href="http://docs.pulpproject.org/plugins/pulp_python/user-docs/getting_started.html#upload-a-python-package" class="external">http://docs.pulpproject.org/plugins/pulp_python/user-docs/getting_started.html#upload-a-python-package</a></li>
</ul> Python Support - Issue #1916 (CLOSED - NOTABUG): Unable to install packages with `.` in the name ...https://pulp.plan.io/issues/19162016-05-13T17:46:49Znwittstocknate.wittstock@urbanairship.com
<p>The latest pip patch version updated a dependency that changes how `.` is normalized in package names. Unfortunately this causes issues with pulp_python and similar tools (such as devpi). We can confirm that our pulp is affected by attempting to install the package `zope.interface`. This works correctly with pip 8.1.1, but fails with pip 8.1.2, which claims the package can't be found in the index.</p>
<p>See <a href="https://github.com/pypa/pip/issues/3666" class="external">https://github.com/pypa/pip/issues/3666</a> for the issue with pip where this is discussed; the maintainers have closed the issue, and are taking the stance that tools like devpi and pulp will need to be updated to fix.</p>
<p>Currently, we've worked around the issue by pinning pip==8.1.1, but this is a short-term fix.</p>
<p>Let me know if I can provide any additional information. Thanks!</p>