Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-09-23T09:38:55ZPulp
Planio Pulp - Issue #9434 (CLOSED - CURRENTRELEASE): Unable to switch distribution from repository to re...https://pulp.plan.io/issues/94342021-09-23T09:38:55Zsskracic@redhat.comsskracic@redhat.com
<pre><code>pulp_core: 3.14.5
pulp_container: 2.7.1
</code></pre>
<p>In RHUI workflow we support pre-syncing the content and then making it available (say at GA) with
by simply repointing the distribution. This works correctly in pulp_rpm, but pulp_container raises an
error when a REST request is issued to update a distribution. Given this distribution:</p>
<pre><code>{
"base_path": "base_path",
"content_guard": "/pulp/api/v3/contentguards/container/content_redirect/5fbef8f9-cfce-4bca-a376-c46403fcafc5/",
"description": null,
"name": "name",
"namespace": "/pulp/api/v3/pulp_container/namespaces/2c0429a6-faf4-46fd-9cbe-c441da1ac5e9/",
"private": false,
"pulp_created": "2021-09-22T14:18:23.653633Z",
"pulp_href": "/pulp/api/v3/distributions/container/container/3a8537e8-6e16-498c-bea2-ee4837bcf768/",
"pulp_labels": {},
"registry_path": "localhost:24817/base_path",
"repository": null,
"repository_version": "/pulp/api/v3/repositories/container/container/cadd407b-411c-4597-9b2a-ab278a480dca/versions/1/"
}
</code></pre>
<p>Now, I want to change this distribution to always point to the latest repository version instead of a fixed one, so I'm trying
to populate the <strong>repository</strong> field:</p>
<pre><code>curl -u admin:****** -X PUT -H 'Content-type: application/json' -d '{"name": "name", "base_path": "base_path", "repository": "/pulp/api/v3/repositories/container/container/cadd407b-411c-4597-9b2a-ab278a480dca/"}' http://localhost:24817/pulp/api/v3/distributions/container/container/3a8537e8-6e16-498c-bea2-ee4837bcf768/
{"non_field_errors":["Only one of the attributes 'repository' and 'repository_version' may be used simultaneously."]}[
</code></pre>
<p>It doesn't make a difference if I supply "repository_version": null in the request body, or when using PATCH HTTP method.</p> RPM Support - Issue #9398 (CLOSED - CURRENTRELEASE): No optimized syncs after a change to repo's ...https://pulp.plan.io/issues/93982021-09-15T09:18:32Zsskracic@redhat.comsskracic@redhat.com
<p>My environment:</p>
<p>pulpcore - 3.14.5
pulp-rpm - 3.14.2
RHEL 8</p>
<p>I am fighting a weird performance issue. After a change to a remote that is attached to some repository (for example when a client certificate is changed), all further sync attempts for that repository involves reloading of the repository metadata and parsing it, even when no new content is available and no new repository version is created afterwards. This repeats for any subsequent sync attempt of that repo, and is a significant resource hog (eg. RHEL7 repos carry 30K+ packages). A sample task with progress reports looks like this:</p>
<pre><code> {
"pulp_href": "/pulp/api/v3/tasks/a9ec4ba1-a804-483d-b9ae-02874a8f19ad/",
"pulp_created": "2021-09-15T08:43:25.974820Z",
"state": "running",
"name": "pulp_rpm.app.tasks.synchronizing.synchronize",
"logging_cid": "4bd0de408bce49f2932790a4c6075e5e",
"started_at": "2021-09-15T09:03:54.732570Z",
"finished_at": null,
"error": null,
"worker": "/pulp/api/v3/workers/0c1e8308-37a8-43e3-8a9b-fb91ebdc18b6/",
"parent_task": null,
"child_tasks": [],
"task_group": null,
"progress_reports": [
{
"message": "Downloading Metadata Files",
"code": "sync.downloading.metadata",
"state": "completed",
"total": null,
"done": 11,
"suffix": null
},
{
"message": "Downloading Artifacts",
"code": "sync.downloading.artifacts",
"state": "running",
"total": null,
"done": 0,
"suffix": null
},
{
"message": "Associating Content",
"code": "associating.content",
"state": "running",
"total": null,
"done": 0,
"suffix": null
},
{
"message": "Parsed Modulemd",
"code": "sync.parsing.modulemds",
"state": "completed",
"total": 374,
"done": 374,
"suffix": null
},
{
"message": "Parsed Modulemd-defaults",
"code": "sync.parsing.modulemd_defaults",
"state": "completed",
"total": 45,
"done": 45,
"suffix": null
},
{
"message": "Parsed Packages",
"code": "sync.parsing.packages",
"state": "completed",
"total": 19122,
"done": 19122,
"suffix": null
},
{
"message": "Parsed Advisories",
"code": "sync.parsing.advisories",
"state": "completed",
"total": 1566,
"done": 1566,
"suffix": null
},
{
"message": "Parsed Comps",
"code": "sync.parsing.comps",
"state": "completed",
"total": 67,
"done": 67,
"suffix": null
}
],
"created_resources": [
null
],
"reserved_resources_record": [
"/pulp/api/v3/remotes/rpm/rpm/9b11994b-f7ea-4df3-a7ed-094f4b183a6c/",
"/pulp/api/v3/repositories/rpm/rpm/c610ead3-8517-47c4-9ed8-d9dc9cc731ee/"
]
},
</code></pre> RPM Support - Issue #8944 (CLOSED - CURRENTRELEASE): Cannot sync RHEL6 repos from cdn.redhat.comhttps://pulp.plan.io/issues/89442021-06-21T22:37:08Zsskracic@redhat.comsskracic@redhat.com
<ul>
<li>a containerized pulp based on pulp-ci-centos image with:</li>
</ul>
<p>pulpcore-3.13.0
pulp-rpm-3.13.0</p>
<p>I am not sure whether it's a Pulp bug or a CDN data error, but RHEL6 i386 and x86_64
repo syncing repeatedly fails with this error. The repositories are synced with mirror=True
and the distribution is set to point to repository (so that the latest repo is always distributed).</p>
<pre><code>Repo: Red Hat Enterprise Linux 6 Server from RHUI (RPMs) (6Server-i386)
Start Time: 2021-06-21 22:24:04
Finish Time: 2021-06-21 22:28:18
Elapsed Time: 0:04:13.581718
Result: Error
Exception: Package id from primary metadata (42003b3621ebf25f9b5fe25b0d2c4f2024958a9d), does not match package id from filelists, other metadata (dcacfd1c930cba9ee112c1055e7a1e94bedd7f3a)
Traceback: File "/usr/local/lib/python3.6/site-packages/rq/worker.py", line 1013, in perform_job
rv job.perform()
File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 709, in perform
self._result = self._execute()
File "/usr/local/lib/python3.6/site-packages/rq/job.py", line 732, in _execute
result = self.func(*self.args, **self.kwargs)
File "/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py", line 395, in synchronize
version = dv.create()
File "/src/pulpcore/pulpcore/plugin/stages/declarative_version.py", line 149, in create
loop.run_until_complete(pipeline)
File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
return future.result()
File "/src/pulpcore/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
await asyncio.gather(*futures)
File "/src/pulpcore/pulpcore/plugin/stages/api.py", line 43, in __call__
await self.run()
File "/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py", line 674, in run
await self.parse_repository_metadata(repomd, repomd_files, file_extension)
File "/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py", line 719, in parse_repository_metadata
file_extension=file_extension,
File "/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py", line 987, in parse_packages
).format(pkgid, pkgid_extra)
</code></pre> RPM Support - Issue #8787 (CLOSED - CURRENTRELEASE): "UnboundLocalError: local variable 'result' ...https://pulp.plan.io/issues/87872021-05-20T11:32:01Zsskracic@redhat.comsskracic@redhat.com
<p>Today I was trying to sync some Red Hat repos with fresh pulp-rpm 3.12.0 release. Everything went well, except for couple of repos that generated this stack trace in the logs:</p>
<pre><code>May 20 11:20:17 pulp3.skracic.net rq[1495551]: pulp [612c4427c121435c9fea0b933098c1cd]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=rhel-8
-for-aarch64-appstream-beta-source-rhui-rpms remote=rhel-8-for-aarch64-appstream-beta-source-rhui-rpms
May 20 11:20:17 pulp3.skracic.net rq[1495559]: pulp [9e8d4c3e69de43b58bb023182fe0db84]: rq.worker:ERROR: Traceback (most recent call last):
May 20 11:20:17 pulp3.skracic.net rq[1495559]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 1013, in perform_job
May 20 11:20:17 pulp3.skracic.net rq[1495559]: rv = job.perform()
May 20 11:20:17 pulp3.skracic.net rq[1495559]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 709, in perform
May 20 11:20:17 pulp3.skracic.net rq[1495559]: self._result = self._execute()
May 20 11:20:17 pulp3.skracic.net rq[1495559]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 732, in _execute
May 20 11:20:17 pulp3.skracic.net rq[1495559]: result = self.func(*self.args, **self.kwargs)
May 20 11:20:17 pulp3.skracic.net rq[1495559]: File "/usr/local/lib/pulp/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py", line 227, in synchronize
May 20 11:20:17 pulp3.skracic.net rq[1495559]: remote_url = fetch_remote_url(remote)
May 20 11:20:17 pulp3.skracic.net rq[1495559]: File "/usr/local/lib/pulp/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py", line 141, in fetch_remote_url
May 20 11:20:17 pulp3.skracic.net rq[1495559]: if get_repomd_file(remote, remote_url):
May 20 11:20:17 pulp3.skracic.net rq[1495559]: File "/usr/local/lib/pulp/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py", line 110, in get_repomd_file
May 20 11:20:17 pulp3.skracic.net rq[1495559]: return result
May 20 11:20:17 pulp3.skracic.net rq[1495559]: UnboundLocalError: local variable 'result' referenced before assignment
</code></pre>
<p>Upon further inspection, it looks like the repos themselves have been "turned off" following the RHEL 8.4 GA, and when tried to download using the command line utilities, 403 Forbidden error code is retrieved.</p>
<p>So it looks like the exception handling should deal with 403 returned from the remote URL.</p>
<p>My Pulp installation is running on RHEL 8, BTW.</p> RPM Support - Story #8681 (CLOSED - CURRENTRELEASE): RFE: auto-publishing step should be visible ...https://pulp.plan.io/issues/86812021-05-03T12:02:16Zsskracic@redhat.comsskracic@redhat.com
<p>When a sync is invoked on a repository with auto-publish enabled, after the actual content is fetched
from the remote, the task spends considerable
amount of time presumably generating metadata, but all progress reports are in state completed.
It would be good if progress report would indicate what the task is currently doing.</p> RPM Support - Story #8673 (CLOSED - DUPLICATE): Auto-publishing should be more fault-toleranthttps://pulp.plan.io/issues/86732021-04-30T14:56:08Zsskracic@redhat.comsskracic@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2273":<a href="https://github.com/pulp/pulp_rpm/issues/2273" class="external">https://github.com/pulp/pulp_rpm/issues/2273</a></p>
<hr>
<p>I admit the title is a bit vague.</p>
<p>During auto-publishing sync of a very large repository (rhel-7-server-rpms), the <code>rq</code> process got killed by oom-killer sometime in the middle of the publishing step. So the new repository version (1) got created, but accompanying
publication did not.</p>
<p>On the subsequent sync runs, the repository did not get published, as no new content was available to sync, hence a new repository version was not created, which in turn should trigger publication and distribution update.</p>
<p>Of course, the repo can still be published and distributed using the non-autopublishing REST API, but I wonder whether the behavior was intended.</p> RPM Support - Issue #8644 (CLOSED - CURRENTRELEASE): Possibly wrong checksum type in repomd.xml o...https://pulp.plan.io/issues/86442021-04-28T09:56:22Zsskracic@redhat.comsskracic@redhat.com
<p>I'm working off the 'master' branch of pulp-rpm trying to get a glimpse of repo autopublish feature - I know it has not yet been released, so I have been warned. Nevertheless, I would like to report a possible issue with checksum type in repomd.xml when autopublish feature is used. Even when both package and metadata checksum types are set to 'sha1', the checksum used in repomd.xml is sha256. Here's a sample for a RHEL 5 repo synced from cdn.redhat.com:</p>
<pre><code><repomd>
<revision>1619601758</revision>
<data type="primary">
<checksum type="sha256">
0f36c55b27e3af5c9c3c13a79e2a438baf9518a98737eeb79481d0f13fd13d2f
</checksum>
<open-checksum type="sha256">
35424b6b8f28b9890e4a69e2fa60a3dc230c5c3a1a55d440ebf95446da21649c
</open-checksum>
...
</code></pre>
<p>rendering it unreadable from RHEL 5 yum clients. However, the publication itself has both checksum types set to 'sha1' as expected, and 'sha1' is indeed used for package checksums in primary.xml.gz - so only the repomd.xml file itself carries the wrong checksums. I'm running Pulp 3 on RHEL8, btw (I couldn't find it in OS menu dropdown).</p> Pulp - Issue #6805 (CLOSED - CURRENTRELEASE): RHUI repo name incorrect for RHEL 8 installs in AWShttps://pulp.plan.io/issues/68052020-05-22T09:43:55Zsskracic@redhat.comsskracic@redhat.com
<p>When installed on a RHEL 8 box running in AWS, pulp-rpm plugin cannot be installed properly due to missing prerequisite RPMs. Those RPMs are delivered through the CodeReady repo, namely <code>codeready-builder-for-rhel-8-rhui-rpms</code>. The repo name is currently spelt wrong.</p>