Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-12-21T17:23:07ZPulp
Planio RPM Support - Issue #9655 (CLOSED - DUPLICATE): requesting lots of packages with a subset of fiel...https://pulp.plan.io/issues/96552021-12-21T17:23:07Zjsherril@redhat.comjsherril@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2312":<a href="https://github.com/pulp/pulp_rpm/issues/2312" class="external">https://github.com/pulp/pulp_rpm/issues/2312</a></p>
<hr>
<p>When we make a request like this:</p>
<pre><code>/pulp/api/v3/content/rpm/packages/?fields=pulp_href%2Cname%2Cversion%2Crelease%2Carch%2Cepoch%2Csummary%2Cis_modular%2Crpm_sourcerpm%2Clocation_href%2CpkgId&limit=1&offset=1000&repository_version=%2Fpulp%2Fapi%2Fv3%2Frepositories%2Frpm%2Frpm%2Fc6b93206-22bb-4c58-ba07-828c32326330%2Fversions%2F1%2F
</code></pre>
<p>where some subset of fields is requested, we're still seeing all of the fields being loaded from the database. The result is that the query takes much much longer than it should</p>
<p>Testing on the rhel7 repo, it took about ~100 seconds on my machine to fetch all 32K packages specifying the fields above. If I comment out the filelists, changelogs, provides/requires from the model and serializers, that drops down to ~33 seconds. This is a huge improvement.</p>
<p>Reading this: <a href="https://stackoverflow.com/questions/53319787/how-can-i-select-specific-fields-in-django-rest-framework" class="external">https://stackoverflow.com/questions/53319787/how-can-i-select-specific-fields-in-django-rest-framework</a></p>
<p>it looks like its may be possible to exclude certain fields from the query itself fairly easily.</p> RPM Support - Issue #9651 (CLOSED - DUPLICATE): Sync creates publication but no new repository ve...https://pulp.plan.io/issues/96512021-12-21T09:37:32Zmgoddard
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2311":<a href="https://github.com/pulp/pulp_rpm/issues/2311" class="external">https://github.com/pulp/pulp_rpm/issues/2311</a></p>
<hr>
<p>I have a nightly job that uses Ansible Squeezer modules to synchronise, publish and distribute some repositories. Every few days I hit an error like this:</p>
<pre><code>Found multiple matches for publication ({'repository_version': '/pulp/api/v3/repositories/rpm/rpm/b9eb112b-28c3-46bc-a40e-594531b54925/versions/4/
</code></pre>
<p>I have verified that this is the case. There is one publication created after the last successful sync, and another created today for the same version.</p>
<pre><code> pulp rpm publication list --repository-version /pulp/api/v3/repositories/rpm/rpm/b9eb112b-28c3-46bc-a40e-594531b54925/versions/4/
[
{
"pulp_href": "/pulp/api/v3/publications/rpm/rpm/21680308-1fc4-4bea-a5fc-1e3c609533f1/",
"pulp_created": "2021-12-21T02:31:44.243390Z",
"repository_version": "/pulp/api/v3/repositories/rpm/rpm/b9eb112b-28c3-46bc-a40e-594531b54925/versions/4/",
"repository": "/pulp/api/v3/repositories/rpm/rpm/b9eb112b-28c3-46bc-a40e-594531b54925/",
"metadata_checksum_type": "unknown",
"package_checksum_type": "unknown",
"gpgcheck": 0,
"repo_gpgcheck": 1,
"sqlite_metadata": true
},
{
"pulp_href": "/pulp/api/v3/publications/rpm/rpm/4a0c6a75-bb86-4b96-bb6c-6d0f08763847/",
"pulp_created": "2021-12-15T02:31:07.260722Z",
"repository_version": "/pulp/api/v3/repositories/rpm/rpm/b9eb112b-28c3-46bc-a40e-594531b54925/versions/4/",
"repository": "/pulp/api/v3/repositories/rpm/rpm/b9eb112b-28c3-46bc-a40e-594531b54925/",
"metadata_checksum_type": "unknown",
"package_checksum_type": "unknown",
"gpgcheck": 0,
"repo_gpgcheck": 1,
"sqlite_metadata": true
}
]
</code></pre>
<p>I checked the sync task from today, and it completed successfully. However, it lists the new publication as a created resource, but no new repo version.</p>
<pre><code> {
"pulp_href": "/pulp/api/v3/tasks/5e132510-89cb-4224-9966-d1f22d49a4e1/",
"pulp_created": "2021-12-21T02:30:53.723988Z",
"state": "completed",
"name": "pulp_rpm.app.tasks.synchronizing.synchronize",
"logging_cid": "0a4dc729907842aaa5ba9605e418cdd4",
"started_at": "2021-12-21T02:30:53.801940Z",
"finished_at": "2021-12-21T02:31:44.897653Z",
"error": null,
"worker": "/pulp/api/v3/workers/605f92b7-9b71-4039-a3de-0af017d86651/",
"parent_task": null,
"child_tasks": [],
"task_group": null,
"progress_reports": [
{
"message": "Downloading Metadata Files",
"code": "sync.downloading.metadata",
"state": "completed",
"total": null,
"done": 8,
"suffix": null
},
{
"message": "Downloading Artifacts",
"code": "sync.downloading.artifacts",
"state": "completed",
"total": null,
"done": 297,
"suffix": null
},
{
"message": "Associating Content",
"code": "associating.content",
"state": "completed",
"total": null,
"done": 0,
"suffix": null
},
{
"message": "Parsed Packages",
"code": "sync.parsing.packages",
"state": "completed",
"total": null,
"done": 299,
"suffix": null
},
{
"message": "Un-Associating Content",
"code": "unassociating.content",
"state": "completed",
"total": null,
"done": 0,
"suffix": null
}
],
"created_resources": [
"/pulp/api/v3/publications/rpm/rpm/21680308-1fc4-4bea-a5fc-1e3c609533f1/"
],
"reserved_resources_record": [
"/pulp/api/v3/repositories/rpm/rpm/b9eb112b-28c3-46bc-a40e-594531b54925/",
"shared:/pulp/api/v3/remotes/rpm/rpm/7b6bc03e-787e-4266-ba33-425c4f9e540b/"
]
},
</code></pre>
<p>Comparing with another sync task, I see a repository version listed in the created_resources instead.</p>
<p>Here is one of the affected repos:</p>
<pre><code>{
"pulp_href": "/pulp/api/v3/repositories/rpm/rpm/b9eb112b-28c3-46bc-a40e-594531b54925/",
"pulp_created": "2021-11-19T13:21:20.971989Z",
"versions_href": "/pulp/api/v3/repositories/rpm/rpm/b9eb112b-28c3-46bc-a40e-594531b54925/versions/",
"pulp_labels": {},
"latest_version_href": "/pulp/api/v3/repositories/rpm/rpm/b9eb112b-28c3-46bc-a40e-594531b54925/versions/4/",
"name": "CentOS Stream 8 - NFV OpenvSwitch",
"description": null,
"retain_repo_versions": null,
"remote": null,
"autopublish": false,
"metadata_signing_service": null,
"retain_package_versions": 0,
"metadata_checksum_type": null,
"package_checksum_type": null,
"gpgcheck": 0,
"repo_gpgcheck": 0,
"sqlite_metadata": false
}
</code></pre>
<p>And the corresponding remote:</p>
<pre><code> {
"pulp_href": "/pulp/api/v3/remotes/rpm/rpm/7b6bc03e-787e-4266-ba33-425c4f9e540b/",
"pulp_created": "2021-11-19T13:21:41.147452Z",
"name": "CentOS Stream 8 - NFV OpenvSwitch-remote",
"url": "http://mirrorlist.centos.org/?release=8-stream&arch=x86_64&repo=nfv-openvswitch-2",
"ca_cert": null,
"client_cert": null,
"tls_validation": true,
"proxy_url": null,
"pulp_labels": {},
"pulp_last_updated": "2021-11-19T13:21:41.147492Z",
"download_concurrency": null,
"max_retries": null,
"policy": "immediate",
"total_timeout": null,
"connect_timeout": null,
"sock_connect_timeout": null,
"sock_read_timeout": null,
"headers": null,
"rate_limit": null,
"sles_auth_token": null
},
</code></pre>
<p>I'm using <code>policy: immediate</code> and <code>sync_policy: mirror_complete</code> when syncing.</p>
<p>Versions:</p>
<pre><code> {
"component": "core",
"version": "3.16.0"
},
{
"component": "rpm",
"version": "3.16.1"
},
{
"component": "file",
"version": "1.10.1"
},
{
"component": "deb",
"version": "2.16.0"
},
{
"component": "container",
"version": "2.9.0"
},
{
"component": "certguard",
"version": "1.5.1"
}
</code></pre> RPM Support - Issue #9647 (CLOSED - DUPLICATE): ULN-remotes HTTP-Proxy use brokenhttps://pulp.plan.io/issues/96472021-12-20T09:33:07Zmbucher
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2322":<a href="https://github.com/pulp/pulp_rpm/issues/2322" class="external">https://github.com/pulp/pulp_rpm/issues/2322</a></p>
<hr>
<p>Using ULN-Remotes, using HTTP-proxy does not work.</p>
<pre><code>Cannot connect to host linux-update.oracle.com:443 ssl:default [Connect call failed ('138.1.51.46', 443)
</code></pre> RPM Support - Task #9633 (CLOSED - DUPLICATE): Use repo priorities in the dependency solverhttps://pulp.plan.io/issues/96332021-12-12T06:09:47Zdalleydalley@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2309":<a href="https://github.com/pulp/pulp_rpm/issues/2309" class="external">https://github.com/pulp/pulp_rpm/issues/2309</a></p>
<hr>
<p>We ought to be setting repo priorities such that for every set of copies, any matching RPMs present in the same repository are prioritzed over ones in other repos.</p> RPM Support - Issue #9631 (CLOSED - DUPLICATE): aiohttp.client_exceptions.ClientPayloadErrorhttps://pulp.plan.io/issues/96312021-12-10T13:17:21Zjanr7
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2308":<a href="https://github.com/pulp/pulp_rpm/issues/2308" class="external">https://github.com/pulp/pulp_rpm/issues/2308</a></p>
<hr>
<p>Hi All</p>
<p>Pulp 3.16.1 on docker.</p>
<p>The Sync from remote never completes.
If you could please assist.</p>
<p>We are seeing Ansible error:</p>
<pre><code>An exception occurred during task execution. To see the full traceback, use -vvv. The error was: Exception: Task failed to complete. (failed; A file located at the url https://rmt.abcgroup.net/repo/SUSE/Updates/SLE-SERVER/12-SP5/x86_64/update/src/kernel-source-4.12.14-122.98.1.src.rpm failed validation due to checksum.)
failed: [localhost] (item={'name': 'SLES12-SP5-Updates', 'backend': 'https://rmt.abcgroup.net/repo/SUSE/Updates/SLE-SERVER/12-SP5/x86_64/update/', 'base': 'SLE/SLES12/SP5/updates'}) => {"ansible_index_var": "s_idx", "ansible_loop_var": "item", "changed": false, "item": {"backend": "https://rmt.abcgroup.net/repo/SUSE/Updates/SLE-SERVER/12-SP5/x86_64/update/", "base": "SLE/SLES12/SP5/updates", "name": "SLES12-SP5-Updates"}, "msg": "Task failed to complete. (failed; A file located at the url https://rmt.abcgroup.net/repo/SUSE/Updates/SLE-SERVER/12-SP5/x86_64/update/src/kernel-source-4.12.14-122.98.1.src.rpm failed validation due to checksum.)", "s_idx": 3}
</code></pre>
<p>On remote: rmt.abcgroup.net</p>
<a name="sysctl-agrep-aio"></a>
<h1 >sysctl -a|grep aio<a href="#sysctl-agrep-aio" class="wiki-anchor">¶</a></h1>
<p>fs.aio-max-nr = 65536
fs.aio-nr = 2080</p>
<p>On pulp server</p>
<a name="sysctl-agrep-aio-2"></a>
<h1 >sysctl -a|grep aio<a href="#sysctl-agrep-aio-2" class="wiki-anchor">¶</a></h1>
<p>fs.aio-max-nr = 65536
fs.aio-nr = 2</p>
<p>================
Attached is the Docker pulp log.
docker logs pulp</p>
<p>aio reported errors for various syncs(tries) of the same repo.</p>
<p>The sha256 for the RPMs are correct on the remote server's repodata.</p>
<pre><code>Backing off download_wrapper(...) for 0.7s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [6efc07543b0c4821922029cf5cda3e90]: backoff:INFO: Backing off download_wrapper(...) for 0.7s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 3.5s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [6efc07543b0c4821922029cf5cda3e90]: backoff:INFO: Backing off download_wrapper(...) for 3.5s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 0.4s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [896ae2851414447db9e177f69c677185]: backoff:INFO: Backing off download_wrapper(...) for 0.4s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 0.3s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [896ae2851414447db9e177f69c677185]: backoff:INFO: Backing off download_wrapper(...) for 0.3s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 1.8s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [896ae2851414447db9e177f69c677185]: backoff:INFO: Backing off download_wrapper(...) for 1.8s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 0.9s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [896ae2851414447db9e177f69c677185]: backoff:INFO: Backing off download_wrapper(...) for 0.9s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 3.1s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [896ae2851414447db9e177f69c677185]: backoff:INFO: Backing off download_wrapper(...) for 3.1s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 1.1s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [896ae2851414447db9e177f69c677185]: backoff:INFO: Backing off download_wrapper(...) for 1.1s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 1.6s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [896ae2851414447db9e177f69c677185]: backoff:INFO: Backing off download_wrapper(...) for 1.6s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 2.5s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [896ae2851414447db9e177f69c677185]: backoff:INFO: Backing off download_wrapper(...) for 2.5s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 0.8s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [5f8f6f6814aa494da5817dc65d65dca2]: backoff:INFO: Backing off download_wrapper(...) for 0.8s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 0.8s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [5f8f6f6814aa494da5817dc65d65dca2]: backoff:INFO: Backing off download_wrapper(...) for 0.8s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 0.9s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [5f8f6f6814aa494da5817dc65d65dca2]: backoff:INFO: Backing off download_wrapper(...) for 0.9s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 0.4s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [5f8f6f6814aa494da5817dc65d65dca2]: backoff:INFO: Backing off download_wrapper(...) for 0.4s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 3.8s (aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected)
pulp [5f8f6f6814aa494da5817dc65d65dca2]: backoff:INFO: Backing off download_wrapper(...) for 3.8s (aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected)
Backing off download_wrapper(...) for 1.2s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [5f8f6f6814aa494da5817dc65d65dca2]: backoff:INFO: Backing off download_wrapper(...) for 1.2s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 6.6s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [5f8f6f6814aa494da5817dc65d65dca2]: backoff:INFO: Backing off download_wrapper(...) for 6.6s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Backing off download_wrapper(...) for 6.3s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [5f8f6f6814aa494da5817dc65d65dca2]: backoff:INFO: Backing off download_wrapper(...) for 6.3s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
Giving up download_wrapper(...) after 5 tries (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
pulp [5f8f6f6814aa494da5817dc65d65dca2]: backoff:ERROR: Giving up download_wrapper(...) after 5 tries (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed)
</code></pre>
<hr>
<p>The failures reported soon after the reported aio errors: (see attached log file)</p>
<pre><code>pulp [2fa84b10f57c41f39635eddffc89ee0a]: pulpcore.tasking.pulpcore_worker:INFO: Task 086ebba6-1f81-471c-9f60-47eda47969a6 failed ()
pulp [2fa84b10f57c41f39635eddffc89ee0a]: pulpcore.tasking.pulpcore_worker:INFO: File "/usr/local/lib/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py", line 370, in _perform_task
--
pulp [6efc07543b0c4821922029cf5cda3e90]: pulpcore.tasking.pulpcore_worker:INFO: Task c375f81f-7b64-4356-983b-b8729048094f failed (A file located at the url https://rmt.abcgroup.net/repo/SUSE/Updates/SLE-SERVER/12-SP5/x86_64/update/noarch/kernel-source-azure-4.12.14-16.76.2.noarch.rpm failed validation due to checksum.)
pulp [6efc07543b0c4821922029cf5cda3e90]: pulpcore.tasking.pulpcore_worker:INFO: File "/usr/local/lib/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py", line 370, in _perform_task
</code></pre>
<a name="--"></a>
<h2 >--<a href="#--" class="wiki-anchor">¶</a></h2>
<p>Installation information:</p>
<pre><code>pulp rpm remote show --name "remote_SLES12-SP5-Updates"
"download_concurrency": 2, (tried 1, 4, 10 - same result)
</code></pre>
<p>pulp --version
pulp3 command line interface, version 0.12.0</p>
<p>pip freeze
appdirs==1.4.3
asn1crypto==0.24.0
Babel==2.8.0
beautifulsoup4==4.10.0
bs4==0.0.1
certifi==2021.5.30
cffi==1.13.2
chardet==4.0.0
click==8.0.1
colorama==0.4.4
commonmark==0.9.1
contextlib2==21.6.0
cryptography==2.8
dataclasses==0.8
httpie==1.0.3
idna==2.10
importlib-metadata==4.6.1
iotop==0.6
isc==2.0
Jinja2==2.10.1
MarkupSafe==1.0
packaging==20.3
pip-search==0.0.7
ply==3.10
pulp-cli==0.12.0
py==1.8.1
pyasn1==0.4.2
pycparser==2.17
Pygments==2.6.1
pyOpenSSL==19.0.0
pyparsing==2.4.7
pytz==2020.5
PyYAML==5.4.1
requests==2.25.1
rich==10.15.2
schema==0.7.4
six==1.14.0
soupsieve==2.3.1
toml==0.10.2
typing-extensions==3.10.0.0
urllib3==1.26.6
zipp==3.5.0</p>
<p>pulp status
"versions": [
{
"component": "core",
"version": "3.16.1"
},
{
"component": "rpm",
"version": "3.16.1"
},
{
"component": "python",
"version": "3.5.2"
},
{
"component": "file",
"version": "1.10.1"
},
{
"component": "deb",
"version": "2.16.0"
},
{
"component": "container",
"version": "2.9.1"
},
{
"component": "certguard",
"version": "1.5.1"
},
{
"component": "ansible",
"version": "0.10.1"
}</p>
<p>Docker server</p>
<pre><code>cat /etc/os-release
NAME="SLES"
VERSION="15-SP3"
VERSION_ID="15.3"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP3"
</code></pre>
<p>Thanks so much for the support.
Jan</p> RPM Support - Issue #9619 (CLOSED - DUPLICATE): OpenAPI schema for ModulemdDefault is incorrecthttps://pulp.plan.io/issues/96192021-12-08T16:59:02Zdkliban@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2307":<a href="https://github.com/pulp/pulp_rpm/issues/2307" class="external">https://github.com/pulp/pulp_rpm/issues/2307</a></p>
<hr>
<p>The open api schema for ModulemdDefaults is incorrect. When trying to generate a client with openapi-generator-cli 5.3.0, the following exception is emitted:</p>
<pre><code>Exception in thread "main" java.lang.RuntimeException: Could not generate api file for 'ContentModulemdDefaults'
</code></pre> RPM Support - Issue #9605 (CLOSED - DUPLICATE): regression in pulp-rpm 3.14.6https://pulp.plan.io/issues/96052021-12-06T14:54:59Zvchepkov
<p>The original fix for issue <a href="https://pulp.plan.io/issues/8875" class="external">https://pulp.plan.io/issues/8875</a> fixed the problem, but recent changes broke functionality again</p>
<pre><code>Dec 6 05:05:50 foreman901 pulpcore-worker-6: pulp [19375194-2146-4216-8e9d-9c53e3188190]: backoff:ERROR: Giving up download_wrapper(...) after 1 tries (aiohttp.client_exceptions.ClientResponseError: 403, message='Forbidden', url=URL('http://amazonlinux.us-east-1.amazonaws.com/blobstore/0f3098b2c2be0b1b3a4421f1187386bf35e1325faf11a3efff568caf06b1cd2e/java-11-amazon-corretto-headless-11.0.13+8-1.amzn2.x86_64.rpm'))
Dec 6 05:05:51 foreman901 pulpcore-worker-6: pulp [19375194-2146-4216-8e9d-9c53e3188190]: pulpcore.tasking.pulpcore_worker:INFO: Task 786abd82-c009-4ddc-b5ba-f10ecf3e0d97 failed (403, message='Forbidden', url=URL('http://amazonlinux.us-east-1.amazonaws.com/blobstore/0f3098b2c2be0b1b3a4421f1187386bf35e1325faf11a3efff568caf06b1cd2e/java-11-amazon-corretto-headless-11.0.13+8-1.amzn2.x86_64.rpm'))
</code></pre>
<p>downgrading from python3-pulp-rpm-3.14.6-2.el7.noarch to python3-pulp-rpm-3.14.3-1.el7.noarch restored successful synchronization</p> RPM Support - Issue #9587 (CLOSED - NOTABUG): manually upload rpm packageshttps://pulp.plan.io/issues/95872021-11-25T16:23:51ZTiagodCC
<p>Hello All</p>
<p>I am currently developing an Ansible playbook to upload custom RPM packages to specific pulp repositories. It uses the pulp.squeezer collection and some API calls.<br>
For this I have used the following documentations: <a href="https://docs.pulpproject.org/pulp_rpm/workflows/upload.html" class="external">https://docs.pulpproject.org/pulp_rpm/workflows/upload.html</a> & <a href="https://docs.pulpproject.org/pulp_rpm/restapi.html#operation/" class="external">https://docs.pulpproject.org/pulp_rpm/restapi.html#operation/</a></p>
<p>Following the upload of the file, the creation of rpm content and the association of the content with the desired repository, the change to the repository is published and distributed.<br>
However, when Pulp is invoked via the browser, it becomes apparent that the package is not displayed with the actual package name, but with its "location_href" from the metadata.</p>
<p>Below you can find a snippet from the metadata of one of the affected packages:</p>
<pre><code><package type="rpm">
<name>amsd</name>
<arch>x86_64</arch>
<version epoch="0" ver="2.5.0" rel="1675.24.rhel8"/>
<checksum type="sha256" pkgid="YES">554b4534999d9e0b778c8c1825f1ae098fe858b36daeaa5ac7c38b81804ffb4a</checksum>
<summary>Agentless Management Service</summary>
<description>This package contains the helper daemon that provides information for iLO5
embedded health and alerting.</description>
<packager>Hewlett-Packard Enterprise</packager>
<url>http://www.hpe.com/go/proliantlinux</url>
<time file="1637837726" build="1630381441"/>
<size package="3026011" installed="8620992" archive="8632796"/>
<location href="Packages/f/fcbdc174-f1f4-4277-8913-3e2ce843e028"/>
<format>
<rpm:license>MIT and BSD</rpm:license>
<rpm:vendor>Hewlett-Packard Enterprise</rpm:vendor>
<rpm:group>System Environment/Daemons</rpm:group>
<rpm:buildhost>ablrh7ex6416.sde.rdlabs.hpecorp.net</rpm:buildhost>
<rpm:sourcerpm>amsd-2.5.0-1675.24.rhel8.src.rpm</rpm:sourcerpm>
<rpm:header-range start="872" end="55032"/>
<rpm:provides>
<rpm:entry name="config(amsd)" flags="EQ" epoch="0" ver="2.5.0" rel="1675.24.rhel8"/>
<rpm:entry name="amsd(x86-64)" flags="EQ" epoch="0" ver="2.5.0" rel="1675.24.rhel8"/>
<rpm:entry name="amsd" flags="EQ" epoch="0" ver="2.5.0" rel="1675.24.rhel8"/>
</rpm:provides>
<rpm:requires>
<rpm:entry name="libc.so.6(GLIBC_2.17)(64bit)"/>
<rpm:entry name="/bin/sh"/>
<rpm:entry name="/bin/sh" pre="1"/>
<rpm:entry name="systemd"/>
<rpm:entry name="rtld(GNU_HASH)"/>
<rpm:entry name="pciutils"/>
<rpm:entry name="lsof"/>
<rpm:entry name="libsystemd.so.0(LIBSYSTEMD_209)(64bit)"/>
<rpm:entry name="libsystemd.so.0()(64bit)"/>
<rpm:entry name="libstdc++.so.6(GLIBCXX_3.4.21)(64bit)"/>
<rpm:entry name="libstdc++.so.6(GLIBCXX_3.4.20)(64bit)"/>
<rpm:entry name="libstdc++.so.6(GLIBCXX_3.4)(64bit)"/>
<rpm:entry name="libstdc++.so.6(CXXABI_1.3)(64bit)"/>
<rpm:entry name="libstdc++.so.6()(64bit)"/>
<rpm:entry name="librt.so.1(GLIBC_2.2.5)(64bit)"/>
<rpm:entry name="librt.so.1()(64bit)"/>
<rpm:entry name="librpmio.so.8()(64bit)"/>
<rpm:entry name="librpm.so.8()(64bit)"/>
<rpm:entry name="libpthread.so.0(GLIBC_2.2.5)(64bit)"/>
<rpm:entry name="libpthread.so.0()(64bit)"/>
<rpm:entry name="libm.so.6(GLIBC_2.2.5)(64bit)"/>
<rpm:entry name="libm.so.6()(64bit)"/>
<rpm:entry name="libjson-c.so.4()(64bit)"/>
<rpm:entry name="libgcc_s.so.1(GCC_3.0)(64bit)"/>
<rpm:entry name="libgcc_s.so.1()(64bit)"/>
<rpm:entry name="libdl.so.2(GLIBC_2.2.5)(64bit)"/>
<rpm:entry name="libdl.so.2()(64bit)"/>
<rpm:entry name="coreutils"/>
<rpm:entry name="bash"/>
<rpm:entry name="/bin/sh"/>
</rpm:requires>
<rpm:conflicts>
<rpm:entry name="hp-ams"/>
</rpm:conflicts>
<file>/sbin/smad_rev</file>
<file>/sbin/smad</file>
<file>/sbin/mr_cpqScsi_rev</file>
<file>/sbin/mr_cpqScsi</file>
<file>/sbin/cpqiScsi</file>
<file>/sbin/cpqScsi_rev</file>
<file>/sbin/cpqScsi</file>
<file>/sbin/cpqIde_rev</file>
<file>/sbin/cpqIde</file>
<file>/sbin/cpqFca_rev</file>
<file>/sbin/cpqFca</file>
<file>/sbin/amsd_rev</file>
<file>/sbin/amsd</file>
<file>/sbin/ahslog</file>
<file>/etc/sysconfig/smad_rev</file>
<file>/etc/sysconfig/smad</file>
<file>/etc/sysconfig/mr_cpqScsi_rev</file>
<file>/etc/sysconfig/mr_cpqScsi</file>
<file>/etc/sysconfig/cpqiScsi</file>
<file>/etc/sysconfig/cpqScsi_rev</file>
<file>/etc/sysconfig/cpqScsi</file>
<file>/etc/sysconfig/cpqIde_rev</file>
<file>/etc/sysconfig/cpqIde</file>
<file>/etc/sysconfig/cpqFca_rev</file>
<file>/etc/sysconfig/cpqFca</file>
<file>/etc/sysconfig/amsd_rev</file>
<file>/etc/sysconfig/amsd</file>
<file>/etc/sysconfig/ahslog</file>
</format>
</package>
</code></pre>
<p>The package is located at (as mentioned in the metadata). The desired place would look something like this: </p>
<p>The package can be found and installed via clients, but this makes it more difficult to understand.<br>
I unfortunately didn't find anything in the documentation about this.</p>
<p>I therefore wanted to ask what the reason for this is and how to fix it.</p>
<p>Thanks in advance for the answer.</p> RPM Support - Task #9585 (CLOSED - DUPLICATE): Sub-tree-only sync breaks our model of how syncs a...https://pulp.plan.io/issues/95852021-11-23T21:00:08Zdalleydalley@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2306":<a href="https://github.com/pulp/pulp_rpm/issues/2306" class="external">https://github.com/pulp/pulp_rpm/issues/2306</a></p>
<hr>
<p>See: <a href="https://pulp.plan.io/issues/9565" class="external">https://pulp.plan.io/issues/9565</a></p>
<p>We haven't considered the possibility of a repository having a .treeinfo but no top-level repodata. We need to investigate whether this breaks anything (for example: autopublish), and potentially add a new test fixture to exercise it.</p> RPM Support - Issue #9567 (CLOSED - DUPLICATE): More fault tollerant metadata parsinghttps://pulp.plan.io/issues/95672021-11-12T13:22:17Zholger.hees
<p>This fixes parsing metadata for repositories where the filelists does not contain package entries like for gitlab repos.</p>
<p>Check <a href="https://packages.gitlab.com/gitlab/gitlab-ce/scientific/7/x86_64/repodata/repomd.xml" class="external">https://packages.gitlab.com/gitlab/gitlab-ce/scientific/7/x86_64/repodata/repomd.xml</a></p>
<p>there you find a filelists document which contains only</p>
<p>< ? xml version="1.0" encoding="UTF-8" ? >
< filelists xmlns="<a href="http://linux.duke.edu/metadata/filelists" class="external">http://linux.duke.edu/metadata/filelists</a>" packages="0" / ></p>
<p><a href="https://github.com/pulp/pulp_rpm/pull/2174" class="external">https://github.com/pulp/pulp_rpm/pull/2174</a></p> RPM Support - Issue #9565 (CLOSED - DUPLICATE): Subtree only sync is failinghttps://pulp.plan.io/issues/95652021-11-12T13:20:26Zholger.hees
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2318":<a href="https://github.com/pulp/pulp_rpm/issues/2318" class="external">https://github.com/pulp/pulp_rpm/issues/2318</a></p>
<hr>
<p>In case that only a subtree is synced, it can happen that the PRIMARY_REPO key does not exists in repo_sync_results and the sync failed with accessing a not existing key at the end.</p>
<p><a href="https://github.com/pulp/pulp_rpm/pull/2172" class="external">https://github.com/pulp/pulp_rpm/pull/2172</a></p> RPM Support - Issue #9559 (MODIFIED): Unable to sync Fedora 35 repositoryhttps://pulp.plan.io/issues/95592021-11-08T02:09:29Zhyu
<p>Cloned from <a href="https://bugzilla.redhat.com/show_bug.cgi?id=2020473" class="external">https://bugzilla.redhat.com/show_bug.cgi?id=2020473</a></p>
<h2>Description of problem:
When publishing the Fedora 35 repository, Pulp failed with the following error:</h2>
<h2>celery.app.trace:ERROR: [56fc62c9] (24505-91520) Task pulp.server.managers.repo.publish.publish[56fc62c9-29b0-41bf-a820-ba3625ef4a00] raised unexpected: TemplateSyntaxError(u'Empty variable tag on line 548',)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) Traceback (most recent call last):
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
celery.app.trace:ERROR: [56fc62c9] (24505-91520) R = retval = fun(*args, **kwargs)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 688, in <strong>call</strong>
celery.app.trace:ERROR: [56fc62c9] (24505-91520) return super(Task, self).<strong>call</strong>(*args, **kwargs)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 110, in <strong>call</strong>
celery.app.trace:ERROR: [56fc62c9] (24505-91520) return super(PulpTask, self).<strong>call</strong>(*args, **kwargs)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in <strong>protected_call</strong>
celery.app.trace:ERROR: [56fc62c9] (24505-91520) return self.run(*args, **kwargs)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 1142, in publish
celery.app.trace:ERROR: [56fc62c9] (24505-91520) result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 1251, in check_publish
celery.app.trace:ERROR: [56fc62c9] (24505-91520) result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 1303, in _do_publish
celery.app.trace:ERROR: [56fc62c9] (24505-91520) publish_report = publish_repo(transfer_repo, conduit, call_config)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 901, in wrap_f
celery.app.trace:ERROR: [56fc62c9] (24505-91520) return f(*args, **kwargs)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 185, in publish_repo
celery.app.trace:ERROR: [56fc62c9] (24505-91520) return self._publisher.process_lifecycle()
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 573, in process_lifecycle
celery.app.trace:ERROR: [56fc62c9] (24505-91520) super(PluginStep, self).process_lifecycle()
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 164, in process_lifecycle
celery.app.trace:ERROR: [56fc62c9] (24505-91520) step.process()
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 240, in process
celery.app.trace:ERROR: [56fc62c9] (24505-91520) self._process_block(item=item)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 302, in _process_block
celery.app.trace:ERROR: [56fc62c9] (24505-91520) self.process_main(item=item)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 500, in process_main
celery.app.trace:ERROR: [56fc62c9] (24505-91520) context.add_unit_metadata(unit)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/metadata/filelists.py", line 42, in add_unit_metadata
celery.app.trace:ERROR: [56fc62c9] (24505-91520) self.metadata_file_handle.write(unit.render_filelists(self.checksum_type))
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py", line 869, in render_filelists
celery.app.trace:ERROR: [56fc62c9] (24505-91520) return self._render(metadata, context)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py", line 884, in _render
celery.app.trace:ERROR: [56fc62c9] (24505-91520) t = Template(template)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/django/template/base.py", line 191, in <strong>init</strong>
celery.app.trace:ERROR: [56fc62c9] (24505-91520) self.nodelist = self.compile_nodelist()
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/django/template/base.py", line 230, in compile_nodelist
celery.app.trace:ERROR: [56fc62c9] (24505-91520) return parser.parse()
celery.app.trace:ERROR: [56fc62c9] (24505-91520) File "/usr/lib/python2.7/site-packages/django/template/base.py", line 482, in parse
celery.app.trace:ERROR: [56fc62c9] (24505-91520) raise self.error(token, 'Empty variable tag on line %d' % token.lineno)
celery.app.trace:ERROR: [56fc62c9] (24505-91520) TemplateSyntaxError: Empty variable tag on line 548</h2>
<p>This is because one of the rpm in the repo has as "{{" filename causing the Django template syntax error.</p>
<p>In "6dc8dae1be904c2613d5aa3667dacd2554d05077eb1ce4296b6edfa3c4db3a46-filelists.xml.gz"
/usr/lib/.build-id
/usr/lib/.build-id/42
/usr/lib/.build-id/42/5049f5a9e0d1ac25c21de795f28fe886bfa0ca
/usr/lib64/R/library/rlang
/usr/lib64/R/library/rlang/DESCRIPTION
/usr/lib64/R/library/rlang/INDEX
/usr/lib64/R/library/rlang/LICENSE
/usr/lib64/R/library/rlang/Meta
snip...
/usr/lib64/R/library/rlang/help/wref_key.html
/usr/lib64/R/library/rlang/help/wref_value.html
/usr/lib64/R/library/rlang/help/zap.html
/usr/lib64/R/library/rlang/help/zap_srcref.html
/usr/lib64/R/library/rlang/help/{{.html <================= TemplateSyntaxError: Empty variable tag on line 548\n",
/usr/lib64/R/library/rlang/help/{{}}.html <=================</p>
<p>$ rpm -qlp R-rlang-0.4.11-3.fc35.x86_64.rpm | grep "{{"
/usr/lib64/R/library/rlang/help/{{.html
/usr/lib64/R/library/rlang/help/{{}}.html</p>
<p>help]$ cat {{.html</p>
<p>help]$ cat {{}}.html</p>
<p>Steps to Reproduce:</p>
<ol>
<li>Create a custom repository call fedora 35 and add the following feed url</li>
</ol>
<p><a href="https://dl.fedoraproject.org/pub/fedora/linux/releases/35/Everything/x86_64/os/" class="external">https://dl.fedoraproject.org/pub/fedora/linux/releases/35/Everything/x86_64/os/</a></p>
<ol start="2">
<li>Sync the repo</li>
</ol>
<p>Actual results:
Failed to publish repo</p>
<p>Expected results:
Repo can be published successfully</p>
<p>Additional info:
Tested on Satellite 6.10 and the repository can be synced successfully without issue.</p> RPM Support - Issue #9553 (MODIFIED): Publishing repository with large metadata may consume high ...https://pulp.plan.io/issues/95532021-11-03T06:10:34Zhyu
<p>Pulp is consuming high memory when publishing RHEL 7 repository. This is happening when Pulp is calculating the checksum of the metadata. It reads the whole metadata file into memory at once to calculates the checksum. For example, the other.xml.gz (compressed ) for RHEL 7 repository is about 837MB size. Reading the entire file into memory will cause Pulp worker to consume more than 1GB for RAM.</p>
<p><a href="https://github.com/pulp/pulp/blob/2-master/server/pulp/plugins/util/metadata_writer.py#L99-L101" class="external">https://github.com/pulp/pulp/blob/2-master/server/pulp/plugins/util/metadata_writer.py#L99-L101</a>.</p>
<p>How to reproduce:</p>
<ol>
<li>Sync the RHEL 7 repository.</li>
<li>After that manually force full publish it and run the below command to observe the memory usage.</li>
</ol>
<p>watch -n 1 'ps -aux | grep resource_worker</p>
<ol start="3">
<li>The memory usage should be stable between 200MB to 350MB all the time, but will suddenly go up to about 1.1GB for about 3 seconds (around finalizing the publish rpms step) then back to 200MB+.</li>
</ol> RPM Support - Story #9484 (CLOSED - DUPLICATE): delete rpm content via api or command linehttps://pulp.plan.io/issues/94842021-10-04T10:31:36ZTiagodCC
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2303":<a href="https://github.com/pulp/pulp_rpm/issues/2303" class="external">https://github.com/pulp/pulp_rpm/issues/2303</a></p>
<hr>
<p>I use specific custom repositories that aren't synced to publish packages from third party software vendors.</p>
<p>Because of this, I wanted to ask how I can delete existing RPM packages (content).
When I try to remove the Artifact, I get the following error:
"The Artifact cannot be deleted because it is associated with Content."
However, I cannot remove the package:
"Method "DELETE" not allowed."</p>
<p>I unfortunately didn't find anything in the documentation about this.</p>
<p>Thanks in advance for the answer.</p>
<p>Translated with <a href="http://www.DeepL.com/Translator" class="external">www.DeepL.com/Translator</a> (free version)</p> RPM Support - Issue #9471 (CLOSED - NOTABUG): Subrepos aren't being published properlyhttps://pulp.plan.io/issues/94712021-09-29T14:39:46Zdalleydalley@redhat.com
<pre><code>(pulp) [vagrant@pulp3-source-fedora34 devel]$ http GET :24816/pulp/content/test/
HTTP/1.1 200 OK
Content-Length: 645
Content-Type: text/html
Date: Wed, 29 Sep 2021 14:38:32 GMT
Server: Python/3.9 aiohttp/3.7.4.post0
X-PULP-CACHE: MISS
<!DOCTYPE html>
<html>
<body>
<ul>
<li><a href=".treeinfo">.treeinfo</a></li>
<li><a href="AppStream/">AppStream/</a></li>
<li><a href="Packages/">Packages/</a></li>
<li><a href="config.repo">config.repo</a></li>
<li><a href="images/">images/</a></li>
<li><a href="repodata/">repodata/</a></li>
</ul>
</body>
</html>
(pulp) [vagrant@pulp3-source-fedora34 devel]$ http GET :24816/pulp/content/test/Appstream/
HTTP/1.1 404 Not Found
Content-Length: 14
Content-Type: text/plain; charset=utf-8
Date: Wed, 29 Sep 2021 14:38:40 GMT
Server: Python/3.9 aiohttp/3.7.4.post0
404: Not Found
</code></pre>