Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-09-13T15:42:24ZPulp
Planio RPM Support - Issue #9387 (CLOSED - CURRENTRELEASE): Depsolving computes repoclosure once per-rpm...https://pulp.plan.io/issues/93872021-09-13T15:42:24Zggainey
<p>Unlike Pulp2, Pulp3 on-copy depsolving is computing repoclosure on the destination repo once <strong>per RPM copied</strong> . It only takes 1s to compute - but when copying 10K rpms, that adds up, in both CPU and memory.</p> RPM Support - Issue #9107 (CLOSED - CURRENTRELEASE): filelists and changelog metadata is not pars...https://pulp.plan.io/issues/91072021-07-17T22:47:54Zoptiz0r
<p>Versions:</p>
<ul>
<li>Foreman 2.5.2</li>
<li>Katello 4.1.1</li>
<li>pulp-rpm-3.13.3</li>
</ul>
<p>Upstream: <a href="http://lon.mirror.rackspace.com/almalinux/8/BaseOS/x86_64/os/repodata/" class="external">http://lon.mirror.rackspace.com/almalinux/8/BaseOS/x86_64/os/repodata/</a>.</p>
<p>I am seeing filelists not being parsed correctly for Alma Linux repositories. Within Katello UI, I have checked a random sample of packages from all of the Alma repositories I have sync'd (18 in total, x86_64, sources and debug-x86_64 for each of baseos, appstream, highavailability, powertools, extras and devel), and none of them list any files. For comparison, CentOS 8 Stream RPMS do list files.</p>
<p>The generated filelist served up by katello contains each of the packages in the repo but with no package contents. For example, the upstream baseos filelist is 2MB whereas the generated filelist for the katello-hosted repo is just 105KB and looks like the below:</p>
<pre><code class="xml syntaxhl" data-language="xml"><span class="cp"><?xml version="1.0" encoding="UTF-8"?></span>
<span class="nt"><filelists</span> <span class="na">xmlns=</span><span class="s">"http://linux.duke.edu/metadata/filelists"</span> <span class="na">packages=</span><span class="s">"1985"</span><span class="nt">></span>
<span class="nt"><package</span> <span class="na">pkgid=</span><span class="s">"59c2172d6e423d8adc3248c1983146492471678afccdfae8ab0f66e18a1aaaa5"</span> <span class="na">name=</span><span class="s">"kernel-debug"</span> <span class="na">arch=</span><span class="s">"x86_64"</span><span class="nt">></span>
<span class="nt"><version</span> <span class="na">epoch=</span><span class="s">"0"</span> <span class="na">ver=</span><span class="s">"4.18.0"</span> <span class="na">rel=</span><span class="s">"305.el8"</span><span class="nt">/></span>
<span class="nt"></package></span>
<span class="nt"><package</span> <span class="na">pkgid=</span><span class="s">"c8120b541261d0ad425f369b1da5eef0aaad5883760ad0e149137027d102207c"</span> <span class="na">name=</span><span class="s">"avahi"</span> <span class="na">arch=</span><span class="s">"i686"</span><span class="nt">></span>
<span class="nt"><version</span> <span class="na">epoch=</span><span class="s">"0"</span> <span class="na">ver=</span><span class="s">"0.7"</span> <span class="na">rel=</span><span class="s">"20.el8"</span><span class="nt">/></span>
<span class="nt"></package></span>
</code></pre>
<p>This is breaking system installs which use the local mirrored content, because packages included in the kickstart base package set have dependencies specified using files, and these are all unsatisfied due to the empty filelists. I have triggered resyncs from upstream mirrors without any change in behaviour.</p>
<p>May be related to similar-looking issue <a class="issue tracker-1 status-11 priority-6 priority-default closed" title="Issue: Filelists and changlogs not always parsed properly (CLOSED - CURRENTRELEASE)" href="https://pulp.plan.io/issues/8955">#8955</a> ?</p> RPM Support - Issue #7116 (CLOSED - CURRENTRELEASE): Modules in errata not exposed by pulp3https://pulp.plan.io/issues/71162020-07-09T15:43:42Zpaji@redhat.compaji@redhat.com
<ol>
<li>Sync the appstream repo</li>
<li>Get the errata reference associated to errata_id = RHBA-2019:1530 or any other modular errata</li>
<li>run (make sure to update the errata id)</li>
</ol>
<pre><code> curl https://`hostname`/pulp/api/v3/content/rpm/advisories/9ff192a4-906f-4efd-ba29-55222eead0d8/ --cert
/etc/pki/katello/certs/pulp-client.crt --key /etc/pki/katello/private/pulp-client.key | jq
</code></pre>
<ol start="4">
<li>Look for pkglist attribute</li>
</ol>
<p>Expected:
pkglist has both module references and package references.</p>
<p>Actual:
Notice that module entries are missing from the package list</p>
<p>Additional Info:</p>
<pre><code> RHBA-2019:1530 is an errata that has the following module in its package list
<update status="final" from="release-engineering@redhat.com" version="1" type="bugfix">
<id>RHBA-2019:1530</id>
......
<pkglist>
<collection short="rhel-8-for-x86_64-appstream-rpms__8_1_virt">
<name>rhel-8-for-x86_64-appstream-rpms__8_1_virt</name>
<module stream="rhel" version="8000020190529063309" arch="x86_64" name="virt" context="55190bc5" />
........
</update>
</code></pre>
<p>Yet pulp does not show me any module name in the custom json</p> RPM Support - Test #4351 (CLOSED - COMPLETE): modules.yaml reference in repomd.xml does not use s...https://pulp.plan.io/issues/43512019-01-24T14:13:07Zbherring
<p>A report from upstream katello (<a href="https://projects.theforeman.org/issues/25529" class="external">https://projects.theforeman.org/issues/25529</a>) indicates that pulp generates a repomd.xml file that claims to use the checksum type that has been set on the publisher, but in fact is not using that:</p>
<p>In this example it purports to have used a sha1 checksum, but in fact its actually a sha256 checksum:</p>
<pre><code> <data type="modules">
<location href="repodata/824ffe238f202a0612ecbb2b0c0459dc289a7ef47adb1f26406453d41f476449-modules.yaml.gz"/>
<timestamp>1542811864</timestamp>
<size>33</size>
<checksum type="sha1">824ffe238f202a0612ecbb2b0c0459dc289a7ef47adb1f26406453d41f476449</checksum>
<open-size>0</open-size>
<open-checksum type="sha1">da39a3ee5e6b4b0d3255bfef95601890afd80709</open-checksum>
</data>
</code></pre>
<p>To reproduce:</p>
<p>1. create and sync a yum repository<br>
2. set a checksum type of sha1<br>
3. publish the repository</p> RPM Support - Test #4350 (CLOSED - COMPLETE): modules.yaml file is generated on repository with n...https://pulp.plan.io/issues/43502019-01-24T14:12:25Zbherring
<p>It seems that syncing and publishing a repository without any modularity info, still results in a modules.yaml being generated.</p>
<p>See: <a href="https://projects.theforeman.org/issues/25529" class="external">https://projects.theforeman.org/issues/25529</a> for more information</p> RPM Support - Test #4269 (CLOSED - COMPLETE): Recursive and conservative recursive copyhttps://pulp.plan.io/issues/42692018-12-14T17:36:04Zmilan
<a name="Description"></a>
<h2 >Description<a href="#Description" class="wiki-anchor">¶</a></h2>
<p>With the <a href="https://github.com/pulp/pulp_rpm/pull/1226" class="external">fix</a> for the Issue <a class="issue tracker-1 status-11 priority-6 priority-default closed" title="Issue: Regression Pulp 2.17.1: recursive copy of RPMs does not copy partially resolvable dependencies (CLOSED - CURRENTRELEASE)" href="https://pulp.plan.io/issues/4152">#4152</a>, the default behaviour of recursive copy changes back to the state before 2.17, rendering the test case <a href="https://github.com/PulpQE/Pulp-2-Tests/blob/c9277928c2788bced8d0ae3f630a46b6162fc4c9/pulp_2_tests/tests/rpm/api_v2/test_rich_weak_dependencies.py#L183" class="external">CopyRecursiveUnitsTestCase</a> broken.<br>
The behaviour specified in the <a href="https://github.com/PulpQE/pulp-smash/issues/1090" class="external">pulp smash issue 1090</a>, as requested in the Issue <a class="issue tracker-3 status-11 priority-6 priority-default closed" title="Story: Support more conservative dependency solving (CLOSED - CURRENTRELEASE)" href="https://pulp.plan.io/issues/2478">#2478</a>, is now optional and can be triggered by providing the <code>"recursive_conservative": true</code> config override instead of the default <code>"recursive": true</code>.<br>
With that setting, the test case <a href="https://github.com/PulpQE/Pulp-2-Tests/blob/c9277928c2788bced8d0ae3f630a46b6162fc4c9/pulp_2_tests/tests/rpm/api_v2/test_rich_weak_dependencies.py#L183" class="external">CopyRecursiveUnitsTestCase</a> should agin pass as is.</p>
<a name="Proposed-solution"></a>
<h2 >Proposed solution<a href="#Proposed-solution" class="wiki-anchor">¶</a></h2>
<p>Split the test case <a href="https://github.com/PulpQE/Pulp-2-Tests/blob/c9277928c2788bced8d0ae3f630a46b6162fc4c9/pulp_2_tests/tests/rpm/api_v2/test_rich_weak_dependencies.py#L183" class="external">CopyRecursiveUnitsTestCase</a> such that it asserts:</p>
<ul>
<li>a failure in the terms described in <a href="https://github.com/PulpQE/pulp-smash/issues/1090" class="external">pulp smash issue 1090</a> with the default <code>"recursive": true</code> config override setting</li>
<li>a success once the non-default <code>"recursive_conservative": true</code> config override setting is provided</li>
</ul> RPM Support - Issue #1476 (CLOSED - WONTFIX): The RPM models are indexing a lot of fields that do...https://pulp.plan.io/issues/14762016-01-05T20:54:34Zrbarlow
<p>The RPM models are setting up <a href="https://github.com/pulp/pulp_rpm/blob/bbaddd8f7c0b65914c2c01327f4b5f9370228fcf/plugins/pulp_rpm/plugins/db/models.py#L366-L367" class="external">these indices</a> :</p>
<pre><code>"name", "epoch", "version", "release", "arch", "filename", "checksum", "checksumtype", "version_sort_index"
</code></pre>
<p>Almost all of these do not make sense, and they each will significantly increase the RAM required of the MongoDB server and reduce write performance. I'll go one by one and explain each:</p>
<p>name: The name is also indexed by the first field of the unit key uniqueness constraint that is set up on line 369 below, so this index is redundant.</p>
<p>epoch: Why would users search by epoch?</p>
<p>release: Why would users search by release?</p>
<p>arch: Why would users search by arch?</p>
<p>filename: I could see a case for this one, especially with a contains, but I'm still a little skeptical. I'm a maybe on it.</p>
<p>checksum: This can be a useful way to look up units, so I think it's OK.</p>
<p>checksumtype: Why would users search with this?</p>
<p>version_sort_index: This is also the first field of the compound index of version_sort_index and release_sort_index, so it is redundant like the name index.</p>
<p>I would mark this easy fix or even just make a pull request, but if we've made a release with this we'll need a migration to drop the indices. I haven't done the research to determine if we've released with these indices. If we haven't released, I'd consider this a 2.8 blocker.</p> RPM Support - Story #1314 (CLOSED - WONTFIX): Extend whitelist importer option to optionally incl...https://pulp.plan.io/issues/13142015-10-15T17:48:33Zbmbouterbmbouter@redhat.com
<p>This story builds off of the whitelist importer option introduced in <a class="issue tracker-3 status-9 priority-6 priority-default closed" title="Story: As a user, I can specify a whitelist of packages for sync (CLOSED - WONTFIX)" href="https://pulp.plan.io/issues/206">#206</a> and adds an option to include dependencies. In <a class="issue tracker-3 status-9 priority-6 priority-default closed" title="Story: As a user, I can specify a whitelist of packages for sync (CLOSED - WONTFIX)" href="https://pulp.plan.io/issues/206">#206</a> the 'whitelist' attribute expects a list as the value. This story maintains the list-style, but introduces a dictionary style value with the keys "packages" which contains the list of names similar to the value of <a class="issue tracker-3 status-9 priority-6 priority-default closed" title="Story: As a user, I can specify a whitelist of packages for sync (CLOSED - WONTFIX)" href="https://pulp.plan.io/issues/206">#206</a> and "include_deps" which will be a boolean to enable the inclusion of dependencies of whitelist items. If include_deps is not specified it defaults to false.</p>
<pre><code>{
"whitelist": {
"packages": [
"foo",
"bar"
],
"include_deps": true
}
}
</code></pre>
<p>Dependency solving should be done in its own logic that is separate from the rpm sync code itself. It will need to be called from the sync code which implements <a class="issue tracker-3 status-9 priority-6 priority-default closed" title="Story: As a user, I can specify a whitelist of packages for sync (CLOSED - WONTFIX)" href="https://pulp.plan.io/issues/206">#206</a>.</p> RPM Support - Issue #1301 (CLOSED - NOTABUG): Error producing repo metadatahttps://pulp.plan.io/issues/13012015-10-06T15:45:45Zlfisher047@gmail.comlfisher047@gmail.com
<p>When applying <a href="https://github.com/pulp/pulp_rpm/pull/720/files" class="external">https://github.com/pulp/pulp_rpm/pull/720/files</a> to my pulp install on a centos 6 systen, I am unable to use the produced repo.</p>
<p>I run yum update and get the error TypeError: Parsing primary.xml error: XML declaration allowed only at start of the document.</p>
<p>Sets to repoduce:</p>
<p>Install pulp on rhel/centos 6.<br>
Apply change request above.<br>
Sync any yum repo via pulp<br>
Create yum.repo.d/newfile.repo that points to created repo from pulp.<br>
Run yum update.</p>
<p>Should get the error above.</p> RPM Support - Refactor #1013 (CLOSED - WONTFIX): Reduce sync time spent processing metadata up-fronthttps://pulp.plan.io/issues/10132015-05-29T18:44:56Zmhrivnakmhrivnak@redhat.com
<p>The rpm sync workflow spends a lot of time early on parsing metadata (filelist.xml and other.xml) and writing it to disk in an indexed data structure. Actual downloading of files doesn't happen until afterward. This can take multiple minutes.</p>
<p>There are several options for improving this workflow.</p>
<ul>
<li>start downloading before indexing is complete. This would require a new queue of completed downloads, and an additional thread that finishes processing and saving them once the indexing is complete.</li>
<li>Only index the rpms that will actually be downloaded. This would not help on the first sync (except in rare cases).</li>
<li>use the sqlite file (if available) to get this metadata instead of indexing the data ourselves. The downside here is we'd have to convert it to XML.</li>
<li>there may be other options</li>
</ul>
<p>This refactor is to employ whatever improvements are reasonable to reduce the amount of time spent chewing on metadata before downloading starts.</p> RPM Support - Issue #960 (CLOSED - DUPLICATE): Sync sometimes fetches partial fileshttps://pulp.plan.io/issues/9602015-05-07T18:50:35Zmhrivnakmhrivnak@redhat.com
<p>Rephrased from a downstream bug...</p>
<p>If you sync ~50GB of packages (RHEL 6 + RHEV 3.5)<br>
- on some runs, everything syncs successfully, yet it's fairly common to see a sync error in the RHEVM repo.<br>
- I've seen errors with the below packages this week:<br>
jasperreports-server-pro, "expected_size"=>473115020, "actual_size"=>312700<br>
rhev-guest-tools-iso, 'expected_size': 261383164, 'actual_size': 88080384<br>
- a re-sync is successful</p>
<p>Example error:</p>
<pre><code>{"expected_size"=>119814460,
"error_code"=>"size_mismatch",
"actual_size"=>1110812,
"unit_key"=>
{"name"=>"rhevm-webadmin-portal",
"checksum"=>"fbfa34e6cf6dc4259bc02dacb41c9a40cf88e3cf",
"epoch"=>"0",
"version"=>"3.5.0",
"release"=>"0.32.el6ev",
"arch"=>"noarch",
"checksumtype"=>"sha1"}}
</code></pre>
<p>So somehow it looks like a download is not failing during the transfer when it probably should. It finishes getting an HTTP response, tries to validate the file it got, and the file is too small. If the transfer were interrupted, I would expect to see a connection error of some kind.</p> RPM Support - Task #928 (CLOSED - NOTABUG): Test with SLEShttps://pulp.plan.io/issues/9282015-04-29T13:46:51Zmhrivnakmhrivnak@redhat.com
<p>Test basic yum repo management for suse Enterprise Server. See me for access.</p>
<p>Deliverables</p>
<ul>
<li>sync and publish SLES repos</li>
<li>if possible, install and provision from a pulp-published repo</li>
<li>install software and updates from a pulp-published repo</li>
<li>file issues for any problems or gaps that are discovered</li>
</ul> RPM Support - Task #927 (CLOSED - NOTABUG): Test with openSUSEhttps://pulp.plan.io/issues/9272015-04-29T13:41:02Zmhrivnakmhrivnak@redhat.com
<p>Test basic yum repo management for opensuse.</p>
<p>Deliverables</p>
<ul>
<li>sync and publish opensuse repos</li>
<li>if possible, install and provision from a pulp-published repo</li>
<li>install software and updates from a pulp-published repo</li>
<li>file issues for any problems or gaps that are discovered</li>
</ul> RPM Support - Issue #721 (CLOSED - NOTABUG): Export ISO file retention Limitation.https://pulp.plan.io/issues/7212015-03-02T18:05:20Zbvijaycombvijaycom@gmail.com
<p>Hi Team,</p>
<p>Today I have created pulp server in DMZ subnet and from our master server tried to export as an ISO file. So that from disconnected server I can recreate whenever I need to replicate. However the problem is pulp is always keep n version. Second time if we execute export command the current exported ISO file and directory getting deleted before the new ISO file creation. Is there any way to put retention files (I mean number files) on my own?</p>
<p>My pulp version is</p>
<p>[root@server1]# rpm -q pulp-server<br>
pulp-server-2.5.3-1.el7.noarch</p>
<p>I understand that this export to different directory can be achieved through --export-dir tag.However i just tested the same command and added with --export-dir option.</p>
<p>Now i have 2 issues</p>
<p>1. I have created directory under / called test.I have changed the ownership to apache:apache and executed the export command with --export-dir pointing to this directory.I have received permission denied error.Is there any limitation in pulp that ,pulp will understand only inside /var/lib/pulp directory ?</p>
<p>2. However now i have created the directory called export inside /var/lib/pulp/ and changed the ownership to apache:apache. Now if i run export command pointing to this directory,only rpm's are getting downloaded instead of single ISO file.</p>
<p>Now i am looking for some solutions for the below scenarios in the future releases...</p>
<p>1. Through --export-dir option if we give external directory it should be allowed in pulp for creating ISO file.<br>
2. If i give /var/lip/pulp/export directory in --export-dir tag,why instead ISO single file rpm's are getting downloaded.<br>
3. Also if the customer wants to retain n number of iso exported files under /var/lib/pulp/published/yum/master/export_distributor/ directory itself then there should be command option</p>
<p>Here i have attached the log file.</p>
<p>Thanks,<br>
BVIJAY</p> RPM Support - Task #144 (CLOSED - WORKSFORME): Ensure that DNF works with Pulp rpm reposhttps://pulp.plan.io/issues/1442015-02-05T14:53:01Zcduryeecduryee@redhat.com
<p>With Fedora 22 DNF will be the default instead of yum[0].</p>
<p>This task is to subscribe to a pulp rpm repo and download content with DNF. The output of this task is not necessarily to fix any issues but just to put in bugzilla entries for any breakage.</p>
<p>[0] <a href="http://fedoraproject.org/wiki/Changes/ReplaceYumWithDNF" class="external">http://fedoraproject.org/wiki/Changes/ReplaceYumWithDNF</a></p>