Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2019-03-18T15:12:17ZPulp
Planio Debian Support - Issue #4546 (CLOSED - CURRENTRELEASE): Safely select packages to symlink in comp...https://pulp.plan.io/issues/45462019-03-18T15:12:17Zmdellweg
<p>When publishing an old copy of a Repository, it can happen, that the<br>
component references packages not yet in that version.</p> RPM Support - Issue #4541 (CLOSED - CURRENTRELEASE): Repository publishing duplicates RPM files u...https://pulp.plan.io/issues/45412019-03-14T04:48:16Ztchellomello
<p>Some repositories are getting content duplicated on its repodata. For example, the CentoOS 7 Base repository, gets duplicated symlinks as follows:</p>
<pre><code>[root@pulp ~]# pulp-admin rpm repo list --details --repo-id centos-7-base
+----------------------------------------------------------------------+
RPM Repositories
+----------------------------------------------------------------------+
Id: centos-7-base
Display Name: None
Description: None
Content Unit Counts:
Distribution: 1
Package Category: 11
Package Environment: 10
Package Group: 88
Package Langpacks: 1
Rpm: 10019
Notes:
Scratchpad:
Checksum Type: sha256
Importers:
Config:
Feed: http://centos.mirror.constant.com/7/os/x86_64/
Id: yum_importer
Importer Type Id: yum_importer
Last Override Config:
Last Sync: 2019-03-13T19:58:14Z
Last Updated: 2019-03-13T19:54:40Z
Repo Id: centos-7-base
Scratchpad:
Repomd Revision: 1543161601
Distributors:
Auto Publish: True
Config:
Http: True
Https: True
Relative URL: 7/os/x86_64/
Distributor Type Id: yum_distributor
Id: yum_distributor
Last Override Config:
Last Publish: 2019-03-14T04:17:40Z
Last Updated: 2019-03-13T19:54:40Z
Repo Id: centos-7-base
Scratchpad:
Auto Publish: False
Config:
Http: True
Https: True
Relative URL: 7/os/x86_64/
Distributor Type Id: export_distributor
Id: export_distributor
Last Override Config:
Last Publish: None
Last Updated: 2019-03-13T19:54:40Z
Repo Id: centos-7-base
Scratchpad:
</code></pre>
<p>Once the repository is published, it ended up having duplicated RPMs</p>
<pre><code>pulp-admin rpm repo publish run --repo-id df05f52b-431e-483d-9878-9ffef41d70c8 --force-full
○ → tree /var/lib/pulp/published/yum/https/repos/7/os/x86_64/Packages/ | grep 389-ds-base
│ ├── 389-ds-base-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/07/cb4bf5d6199cdc7192cbdbbf283cccdef38f18751c53660107b14232d2d349/389-ds-base-1.3.8.4-15.el7.x86_64.rpm
│ ├── 389-ds-base-devel-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/ab/e6f96db9ad2c0f7f097f71b506b25a0d83a4a9926e904100f006361ef66d1f/389-ds-base-devel-1.3.8.4-15.el7.x86_64.rpm
│ ├── 389-ds-base-libs-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/94/5011e802dbf9d470522eac943442a8718e08ba321dc0649a55a0d7f130e363/389-ds-base-libs-1.3.8.4-15.el7.x86_64.rpm
│ └── 389-ds-base-snmp-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/e9/b6b32a0421e87ab83a6ea7346f51e3929746e10eeb2c9b5f05e8278c9e7b61/389-ds-base-snmp-1.3.8.4-15.el7.x86_64.rpm
├── 389-ds-base-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/07/cb4bf5d6199cdc7192cbdbbf283cccdef38f18751c53660107b14232d2d349/389-ds-base-1.3.8.4-15.el7.x86_64.rpm
├── 389-ds-base-devel-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/ab/e6f96db9ad2c0f7f097f71b506b25a0d83a4a9926e904100f006361ef66d1f/389-ds-base-devel-1.3.8.4-15.el7.x86_64.rpm
├── 389-ds-base-libs-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/94/5011e802dbf9d470522eac943442a8718e08ba321dc0649a55a0d7f130e363/389-ds-base-libs-1.3.8.4-15.el7.x86_64.rpm
├── 389-ds-base-snmp-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/e9/b6b32a0421e87ab83a6ea7346f51e3929746e10eeb2c9b5f05e8278c9e7b61/389-ds-base-snmp-1.3.8.4-15.el7.x86_64.rpm
</code></pre>
<p>That creates a big problem whenever exporting the content via ISO , as the size of the repo will be twice big.</p>
<p>The repodata is being created correctly as it only references the files under the new strucuture Packages/<first_letter>/</p>
<pre><code>○ → zcat /var/lib/pulp/published/yum/https/repos/7/os/x86_64/Packages/repodata/a3f98f3b850725e0ac52472653efd309be6ba436296e66b1b997ca92c95fcdf1-primary.xml.gz | grep 'href' | grep 389-ds-base
<location href="Packages/3/389-ds-base-libs-1.3.8.4-15.el7.x86_64.rpm"/>
<location href="Packages/3/389-ds-base-1.3.8.4-15.el7.x86_64.rpm"/>
<location href="Packages/3/389-ds-base-snmp-1.3.8.4-15.el7.x86_64.rpm"/>
<location href="Packages/3/389-ds-base-devel-1.3.8.4-15.el7.x86_64.rpm"/>
</code></pre> RPM Support - Story #4527 (CLOSED - CURRENTRELEASE): Improve performance of rpm duplicate nevra c...https://pulp.plan.io/issues/45272019-03-11T04:09:08Zrmcgoverrmcgover@redhat.com
<p>In current versions of Pulp 2.x, uploading an RPM to a repo will remove other RPMs with the same NEVRA.</p>
<p>Currently, we are upgrading from an old version of Pulp 2.7, and I've found that performance of import_uploaded_unit tasks for RPMs has regressed significantly. In Pulp 2.7, imports would usually take around 0.5s. In Pulp 2-master, imports to the same repos have taken from 8 to 130 seconds, depending on the size of the repo.</p>
<p>By debugging I've found most of the time is spent in this duplicate check (remove_unit_duplicate_nevra).</p>
<p>This issue is for improving the performance of remove_unit_duplicate_nevra to reduce the severity of the performance regression.</p> RPM Support - Issue #4518 (CLOSED - CURRENTRELEASE): Modules are not copied during recursive copy...https://pulp.plan.io/issues/45182019-03-07T19:39:41Zbherring
<a name="Concern"></a>
<h2 >Concern<a href="#Concern" class="wiki-anchor">¶</a></h2>
<p>The behavior in 2.18.1 is incorrect for modular errata copy which could affect Katello.</p>
<p>The correct behavior is now seen, recently, in 2-master.</p>
<p>Verified incorrect behavior with Tanya.</p>
<a name="Chronology"></a>
<h2 >Chronology<a href="#Chronology" class="wiki-anchor">¶</a></h2>
<p>A regression in test_modular_copy.py [0] was noted in nightly [1] once libsolv [2] was resolved by pcreech (thanks!) on 3/3/2019 in <a class="issue tracker-1 status-11 priority-6 priority-default closed" title="Issue: Pulp 2.19master nightly ci regression at test_modularity.py (CLOSED - CURRENTRELEASE)" href="https://pulp.plan.io/issues/4405">#4405</a>.</p>
<p>Upon looking through the newly delivered modular documentation [3], it appears that that current behavior in 2-master is correct and the behavior test_modular_copy.py and resulting behavior in 2.18.1-stable is incorrect (and has been since Sept 2018 [0]).</p>
<p>The test was originally written and approved without the understanding of the <strong>modulemd</strong> copies needed as part of the errata.</p>
<p>Please note the CLI examples below for comparisons between 2.18.1 and latest 2-master for all permutations of <strong>--recursive</strong> and <strong>--recursive-conservative</strong> through the CLI.</p>
<a name="Test-Update"></a>
<h3 >Test Update<a href="#Test-Update" class="wiki-anchor">¶</a></h3>
<p>The test with the correct modular errata copy behavior check [4] is already submitted and awaiting development designation of problem scope before implementing into test repositories.</p>
<p>Until then, we have a known, failing test in nightly.</p>
<a name="2181-Behaviors-since-Sept-21-2018-2181-GA-incorrect"></a>
<h2 >2.18.1 Behaviors (since Sept 21 2018 - 2.18.1-GA) [incorrect]<a href="#2181-Behaviors-since-Sept-21-2018-2181-GA-incorrect" class="wiki-anchor">¶</a></h2>
<a name="--recursive"></a>
<h3 >--recursive<a href="#--recursive" class="wiki-anchor">¶</a></h3>
<pre><code>[root@rhel76-218 ~]# pulp-admin rpm repo copy errata --str-eq="errata_id=RHEA-2012:0059" --from-repo-id=test --to-repo-id=new --recursive
This command may be exited via ctrl+c without affecting the request.
[\]
Running...
Copied:
erratum:
RHEA-2012:0059
rpm:
duck-0.7-1-noarch
kangaroo-0.3-1-noarch
</code></pre>
<a name="--recursive-conservative"></a>
<h3 >--recursive-conservative<a href="#--recursive-conservative" class="wiki-anchor">¶</a></h3>
<pre><code>[root@rhel76-218 ~]# pulp-admin rpm repo copy errata --str-eq="errata_id=RHEA-2012:0059" --from-repo-id=test --to-repo-id=new --recursive-conservative
This command may be exited via ctrl+c without affecting the request.
[\]
Running...
Copied:
erratum:
RHEA-2012:0059
rpm:
duck-0.7-1-noarch
kangaroo-0.3-1-noarch
</code></pre>
<a name="Both"></a>
<h3 >Both<a href="#Both" class="wiki-anchor">¶</a></h3>
<pre><code>[root@rhel76-218 ~]# pulp-admin rpm repo copy errata --str-eq="errata_id=RHEA-2012:0059" --from-repo-id=test --to-repo-id=new --recursive-conservative --recursive
This command may be exited via ctrl+c without affecting the request.
[\]
Running...
Copied:
erratum:
RHEA-2012:0059
rpm:
duck-0.7-1-noarch
kangaroo-0.3-1-noarch
</code></pre>
<a name="219-Behaviors-testable-since-libsolv-issues-were-resolved-on-31-correct"></a>
<h2 >2.19 Behaviors (testable since libsolv issues were resolved on ~3/1) [correct]<a href="#219-Behaviors-testable-since-libsolv-issues-were-resolved-on-31-correct" class="wiki-anchor">¶</a></h2>
<a name="--recursive-2"></a>
<h3 >--recursive<a href="#--recursive-2" class="wiki-anchor">¶</a></h3>
<pre><code>[root@rhel76-219 ~]# pulp-admin rpm repo copy errata --str-eq="errata_id=RHEA-2012:0059" --from-repo-id=test --to-repo-id=new --recursive
This command may be exited via ctrl+c without affecting the request.
[\]
Running...
Copied:
erratum:
RHEA-2012:0059
modulemd:
duck-0-20180730233102-deadbeef-noarch
kangaroo-0-20180730223407-deadbeef-noarch
rpm:
duck-0.7-1-noarch
kangaroo-0.3-1-noarch
</code></pre>
<a name="--recursive-conservative-2"></a>
<h3 >--recursive-conservative<a href="#--recursive-conservative-2" class="wiki-anchor">¶</a></h3>
<pre><code>[root@rhel76-219 ~]# pulp-admin rpm repo copy errata --str-eq="errata_id=RHEA-2012:0059" --from-repo-id=test --to-repo-id=new --recursive-conservative
This command may be exited via ctrl+c without affecting the request.
[\]
Running...
Copied:
erratum:
RHEA-2012:0059
modulemd:
duck-0-20180730233102-deadbeef-noarch
kangaroo-0-20180730223407-deadbeef-noarch
rpm:
duck-0.7-1-noarch
kangaroo-0.3-1-noarch
</code></pre>
<a name="Both-2"></a>
<h3 >Both<a href="#Both-2" class="wiki-anchor">¶</a></h3>
<pre><code>[root@rhel76-219 ~]# pulp-admin rpm repo copy errata --str-eq="errata_id=RHEA-2012:0059" --from-repo-id=test --to-repo-id=new --recursive-conservative --recursive
This command may be exited via ctrl+c without affecting the request.
[\]
Running...
Copied:
erratum:
RHEA-2012:0059
modulemd:
duck-0-20180730233102-deadbeef-noarch
kangaroo-0-20180730223407-deadbeef-noarch
rpm:
duck-0.7-1-noarch
kangaroo-0.3-1-noarch
</code></pre> RPM Support - Issue #4477 (CLOSED - CURRENTRELEASE): Duplicated empty collections are published w...https://pulp.plan.io/issues/44772019-02-26T18:14:48Zragbalak
<p><strong>Problem</strong></p>
<p>When a repository with feed "<a href="https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-modules/" class="external">https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-modules/</a>" is created, synced and published, the updateinfo.xml of the published repo contains an inconsistent number of modules.</p>
<p>Steps to reproduce :</p>
<p>Create and sync a repo pointing to the rpm-with-modules fixture.</p>
<pre><code>pulp-admin rpm repo create --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-modules/ --repo-id test-repo
pulp-admin rpm repo sync run --repo-id test-repo
</code></pre>
<p>After the repo is synced, get the number of module tags present in the repo</p>
<pre><code># note the repo id and the timestamp directory after that can change
zcat 8147252315a121c89a1be2058fe3367a52bf81dac61788e53358c271421cd9d6-updateinfo.xml.gz |grep "module" | wc -l
33 # current output of the number of modules in the update_info.xml
</code></pre>
<p>If the above repository is deleted, and then again the same steps are followed for a new repo, the total number of modules in the new repo are seen to be increased</p>
<pre><code>pulp-admin rpm repo delete --repo-id test-repo
pulp-admin rpm repo create --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-modules/ --repo-id test-repo-new
pulp-admin rpm repo sync run --repo-id test-repo-new
zcat ef13ff9115725082dd55fef6251880d28d66c1b0acf28abf89ba7fd2b1ade96f-updateinfo.xml.gz |grep module |wc -l
36 # current output of the number of modules in the update_info.xml
</code></pre>
<p>As seen the number of modules have increased for the same feed url with different repo names.</p>
<p>Also the update-info.xml of the published repo is different comparing with the actual update-info.xml of the fixtures repo.</p>
<p>Have attached the update-info.xml of both the syncs and the fixtures repo.</p>
<p>Environment details :</p>
<pre><code># cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.6 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.6"
PRETTY_NAME="Employee SKU"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.6:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.6
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.6"
# rpm -qa|grep pulp|sort
libsolv-0.6.34-2.pulp.el7.x86_64
pulp-admin-client-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
pulp-deb-admin-extensions-1.9.0-0.1.alpha.201812120524giteefc050.el7.noarch
pulp-deb-plugins-1.9.0-0.1.alpha.201812120524giteefc050.el7.noarch
pulp-docker-admin-extensions-3.4.0-0.1.alpha.201812120503gitaee36e5.el7.noarch
pulp-docker-plugins-3.4.0-0.1.alpha.201812120503gitaee36e5.el7.noarch
pulp-ostree-admin-extensions-1.4.0-0.1.alpha.201812120505git6c63638.el7.noarch
pulp-ostree-plugins-1.4.0-0.1.alpha.201812120505git6c63638.el7.noarch
pulp-puppet-admin-extensions-2.19.0-0.1.alpha.201812120507gitc8c9a9a.el7.noarch
pulp-puppet-plugins-2.19.0-0.1.alpha.201812120507gitc8c9a9a.el7.noarch
pulp-puppet-tools-2.19.0-0.1.alpha.201812120507gitc8c9a9a.el7.noarch
pulp-python-admin-extensions-2.1.0-0.1.alpha.201812120501git5e2aa35.el7.noarch
pulp-python-plugins-2.1.0-0.1.alpha.201812120501git5e2aa35.el7.noarch
pulp-rpm-admin-extensions-2.19.0-0.1.alpha.201812120511git9bc9e73.el7.noarch
pulp-rpm-plugins-2.19.0-0.1.alpha.201812120511git9bc9e73.el7.noarch
pulp-selinux-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
pulp-server-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python2-solv-0.6.34-2.pulp.el7.x86_64
python-isodate-0.5.0-4.pulp.el7.noarch
python-pulp-bindings-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-client-lib-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-common-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-deb-common-1.9.0-0.1.alpha.201812120524giteefc050.el7.noarch
python-pulp-docker-common-3.4.0-0.1.alpha.201812120503gitaee36e5.el7.noarch
python-pulp-oid_validation-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-ostree-common-1.4.0-0.1.alpha.201812120505git6c63638.el7.noarch
python-pulp-puppet-common-2.19.0-0.1.alpha.201812120507gitc8c9a9a.el7.noarch
python-pulp-python-common-2.1.0-0.1.alpha.201812120501git5e2aa35.el7.noarch
python-pulp-repoauth-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
python-pulp-rpm-common-2.19.0-0.1.alpha.201812120511git9bc9e73.el7.noarch
python-pulp-streamer-2.19.0-0.1.alpha.201812120516git129f7b2.el7.noarch
</code></pre> Pulp - Issue #4425 (CLOSED - CURRENTRELEASE): It's possible to create a duplicate login if name i...https://pulp.plan.io/issues/44252019-02-15T06:58:19Zflamarion
<p>Hi,</p>
<p>By mistake I've added two users with the same login (oddly Pulp accepted this)</p>
<pre><code>pulp-admin auth user create --login fjorg1 --name anyname1
pulp-admin auth user create --login fjorg1 --name anyname2
</code></pre>
<p>and now when I'm trying to remove one them I'm getting the following error message:</p>
<pre><code class="python syntaxhl" data-language="python"><span class="n">pulp</span><span class="o">-</span><span class="n">admin</span> <span class="n">auth</span> <span class="n">user</span> <span class="n">delete</span> <span class="o">--</span><span class="n">login</span> <span class="n">fjorg1</span>
<span class="n">An</span> <span class="n">internal</span> <span class="n">error</span> <span class="n">occurred</span> <span class="n">on</span> <span class="n">the</span> <span class="n">Pulp</span> <span class="n">server</span><span class="p">:</span>
<span class="n">RequestException</span><span class="p">:</span> <span class="n">DELETE</span> <span class="n">request</span>
<span class="n">on</span> <span class="o">/</span><span class="n">pulp</span><span class="o">/</span><span class="n">api</span><span class="o">/</span><span class="n">v2</span><span class="o">/</span><span class="n">users</span><span class="o">/</span><span class="n">fjorg1</span><span class="o">/</span> <span class="n">failed</span> <span class="k">with</span> <span class="mi">500</span> <span class="o">-</span> <span class="mi">2</span> <span class="n">items</span> <span class="n">returned</span><span class="p">,</span> <span class="n">instead</span> <span class="n">of</span> <span class="mi">1</span>
</code></pre>
<p>I have a monolithic installation with the following packages installed:</p>
<p>python-pulp-repoauth-2.18.0-2.el7.noarch<br>
python-pulp-rpm-common-2.18.0-1.el7.noarch<br>
pulp-rpm-plugins-2.18.0-1.el7.noarch<br>
pulp-puppet-tools-2.18.0-1.el7.noarch<br>
pulp-ostree-admin-extensions-1.3.1-1.el7.noarch<br>
python-pulp-deb-common-1.8.0-1.el7.noarch<br>
pulp-docker-plugins-3.2.2-1.el7.noarch<br>
pulp-rpm-admin-extensions-2.18.0-1.el7.noarch<br>
python-pulp-common-2.18.0-2.el7.noarch<br>
python-pulp-oid_validation-2.18.0-2.el7.noarch<br>
python-pulp-python-common-2.0.3-1.el7.noarch<br>
pulp-selinux-2.18.0-2.el7.noarch<br>
pulp-server-2.18.0-2.el7.noarch<br>
pulp-python-plugins-2.0.3-1.el7.noarch<br>
pulp-puppet-admin-extensions-2.18.0-1.el7.noarch<br>
pulp-python-admin-extensions-2.0.3-1.el7.noarch<br>
python-pulp-puppet-common-2.18.0-1.el7.noarch<br>
python-pulp-docker-common-3.2.2-1.el7.noarch<br>
libsolv-0.6.34-2.pulp.el7.x86_64<br>
pulp-deb-plugins-1.8.0-1.el7.noarch<br>
python-pulp-client-lib-2.18.0-2.el7.noarch<br>
pulp-deb-admin-extensions-1.8.0-1.el7.noarch<br>
python-pulp-ostree-common-1.3.1-1.el7.noarch<br>
python2-solv-0.6.34-2.pulp.el7.x86_64<br>
pulp-ostree-plugins-1.3.1-1.el7.noarch<br>
pulp-admin-client-2.18.0-2.el7.noarch<br>
pulp-puppet-plugins-2.18.0-1.el7.noarch<br>
python-pulp-bindings-2.18.0-2.el7.noarch<br>
pulp-docker-admin-extensions-3.2.2-1.el7.noarch</p>
<p>Please, can you help to sort this small issue, which is very important for me because it blocks the user to use the pulp-admin. The user can't login.</p> Pulp - Issue #4405 (CLOSED - CURRENTRELEASE): Pulp 2.19master nightly ci regression at test_modul...https://pulp.plan.io/issues/44052019-02-11T19:31:21Zbherring
<a name="Failure-Scenario"></a>
<h2 >Failure Scenario<a href="#Failure-Scenario" class="wiki-anchor">¶</a></h2>
<p>On nightly ci, test_modularity.py runs on 2.19master and fails in such a way that the box is no longer responsive to API testing.</p>
<p>Note this test works and was re-tested after the above error on 2.19 without issue on 2.18.1b1.</p>
<a name="Information"></a>
<h2 >Information:<a href="#Information" class="wiki-anchor">¶</a></h2>
<ul>
<li>Test: <a href="https://github.com/PulpQE/Pulp-2-Tests/blob/master/pulp_2_tests/tests/rpm/api_v2/test_modularity.py" class="external">https://github.com/PulpQE/Pulp-2-Tests/blob/master/pulp_2_tests/tests/rpm/api_v2/test_modularity.py</a>
</li>
<li>Repeatable: Every time</li>
<li>Build: 2.19
<ul>
<li>Branch: 2-master</li>
</ul>
</li>
</ul>
<a name="Failure-Seen"></a>
<h2 >Failure Seen<a href="#Failure-Seen" class="wiki-anchor">¶</a></h2>
<ul>
<li>Test Harness locally (RHEL76 base + Pulp 2 installed, non-FIPS) fails and dies at test_modularity</li>
<li>Jenkins also fails at the same location killing the test harness</li>
</ul>
<a name="journalctl-output-at-time-of-failure"></a>
<h2 >journalctl output at time of failure<a href="#journalctl-output-at-time-of-failure" class="wiki-anchor">¶</a></h2>
<pre><code>Feb 11 01:42:00 host-172-16-46-47.openstacklocal kernel: celery[12053]: segfault at 0 ip 00007f9b1e4a8b99 sp 00007ffe64b687e0 error 4 in libsolv.so.0[7f9b1e43f000+89000]
Feb 11 01:42:00 host-172-16-46-47.openstacklocal pulp[30621]: pulp.server.db.connection:INFO: Attempting to connect to localhost:27017
Feb 11 01:42:00 host-172-16-46-47.openstacklocal pulp[30621]: pulp.server.db.connection:INFO: Attempting to connect to localhost:27017
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: pulp.server.db.connection:INFO: Write concern for Mongo connection: {}
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: py.warnings:WARNING: [9cbb7189] (30621-64032) /usr/lib64/python2.7/site-packages/pymongo/topology.py:74: UserWarning: MongoClient opened before fork.
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: py.warnings:WARNING: [9cbb7189] (30621-64032) "MongoClient opened before fork. Create MongoClient "
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: py.warnings:WARNING: [9cbb7189] (30621-64032)
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: pulp.server.async.tasks:INFO: [9cbb7189] Task failed : [0349bb3b-ebb7-4585-810d-8b46ed94cb8f] : Worker terminated abnormally while processing task 03
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) Task pulp.server.async.tasks._release_resource[9cbb7189-331f-42f5-b957-0d05211d5c62] raised unexpect
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) Traceback (most recent call last):
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) R = retval = fun(*args, **kwargs)
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) return super(PulpTask, self).__call__(*args, **kwargs)
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) return self.run(*args, **kwargs)
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 296, in _release_resource
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) new_task.on_failure(exception, task_id, (), {}, MyEinfo)
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 602, in on_failure
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) if not self.request.called_directly:
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) File "/usr/lib/python2.7/site-packages/celery/app/task.py", line 978, in _get_request
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) req = self.request_stack.top
Feb 11 01:42:01 host-172-16-46-47.openstacklocal pulp[30621]: celery.app.trace:ERROR: [9cbb7189] (30621-64032) AttributeError: 'NoneType' object has no attribute 'top'
Feb 11 01:42:11 host-172-16-46-47.openstacklocal pulp[11083]: celery.worker.request:ERROR: (11083-64032) Task handler raised error: WorkerLostError('Worker exited prematurely: signal 11 (SIGSEGV).',)
Feb 11 01:42:11 host-172-16-46-47.openstacklocal pulp[11083]: celery.worker.request:ERROR: (11083-64032) Traceback (most recent call last):
Feb 11 01:42:11 host-172-16-46-47.openstacklocal pulp[11083]: celery.worker.request:ERROR: (11083-64032) File "/usr/lib64/python2.7/site-packages/billiard/pool.py", line 1223, in mark_as_worker_lost
Feb 11 01:42:11 host-172-16-46-47.openstacklocal pulp[11083]: celery.worker.request:ERROR: (11083-64032) human_status(exitcode)),
Feb 11 01:42:11 host-172-16-46-47.openstacklocal pulp[11083]: celery.worker.request:ERROR: (11083-64032) WorkerLostError: Worker exited prematurely: signal 11 (SIGSEGV).
</code></pre> Docker Support - Issue #4384 (CLOSED - CURRENTRELEASE): Pulp Crane not returning correct headershttps://pulp.plan.io/issues/43842019-02-05T23:10:38Zpaji@redhat.compaji@redhat.com
<p>Sync the following docker repo<br>
feed: <a href="https://quay.io/" class="external">https://quay.io/</a><br>
upstream_name: openshift-release-dev/ocp-release</p>
<p>Pulp Crane</p>
<p>Check out the following output when syncing v1 content</p>
<pre><code>$ curl --head -L https://<fqdn>:5000/v2/default_organization-custom-ocp/manifests/4.0.0-0
HTTP/1.1 302 FOUND
Date: Mon, 04 Feb 2019 16:29:14 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux)
Content-Length: 437
Location: https://<fqdn>/pulp/docker/v2/fe117c3c-5820-4406-ad34-8404062b8458/manifests/1/4.0.0-0
Content-Type: text/html; charset=utf-8
HTTP/1.1 200 OK
Date: Mon, 04 Feb 2019 16:29:15 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux)
Last-Modified: Wed, 30 Jan 2019 16:28:40 GMT
ETag: "e5e-580af6711a0c7"
Accept-Ranges: bytes
Content-Length: 3678
Docker-Distribution-API-Version: registry/2.0
</code></pre>
<p>Quay:<br>
Now compare it to the same in quay</p>
<pre><code>$ curl --head -L https://quay.io/v2/coreos/etcd/manifests/latest
HTTP/2 200
server: nginx/1.14.2
date: Mon, 04 Feb 2019 16:29:41 GMT
content-type: application/vnd.docker.distribution.manifest.v1+prettyjws
content-length: 6655
docker-content-digest: sha256:5b6691b7225a3f77a5a919a81261bbfb31283804418e187f7116a0a9ef65d21d
x-frame-options: DENY
strict-transport-security: max-age=63072000; preload
</code></pre>
<p>Notice how content-type is set to application/vnd.docker.distribution.manifest.v1+prettyjws in the case of quay, while its blank for pulp-crane.</p>
<p>This causes issues in clients pulling this information.</p> Docker Support - Issue #4328 (CLOSED - CURRENTRELEASE): fsLayers error during synchttps://pulp.plan.io/issues/43282019-01-15T21:04:00ZEppO
<p>Hello,</p>
<p>trying to mirror gcr.io/google-containers/etcd, I encountered that issue during a sync:</p>
<pre><code class="python syntaxhl" data-language="python"><span class="mi">2019</span><span class="o">-</span><span class="mi">01</span><span class="o">-</span><span class="mi">15</span> <span class="mi">12</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mi">13</span><span class="p">,</span><span class="mi">606</span> <span class="o">-</span> <span class="n">INFO</span> <span class="o">-</span> <span class="n">Response</span> <span class="n">body</span> <span class="p">:</span>
<span class="p">{</span>
<span class="s">"exception"</span><span class="p">:</span> <span class="n">null</span><span class="p">,</span>
<span class="s">"task_type"</span><span class="p">:</span> <span class="s">"pulp.server.managers.repo.sync.sync"</span><span class="p">,</span>
<span class="s">"_href"</span><span class="p">:</span> <span class="s">"/pulp/api/v2/tasks/291a8d9d-4b84-4aeb-af0f-2772e8ade987/"</span><span class="p">,</span>
<span class="s">"task_id"</span><span class="p">:</span> <span class="s">"291a8d9d-4b84-4aeb-af0f-2772e8ade987"</span><span class="p">,</span>
<span class="s">"tags"</span><span class="p">:</span> <span class="p">[</span>
<span class="s">"pulp:repository:etcd-gcr2"</span><span class="p">,</span>
<span class="s">"pulp:action:sync"</span>
<span class="p">],</span>
<span class="s">"finish_time"</span><span class="p">:</span> <span class="s">"2019-01-15T20:54:12Z"</span><span class="p">,</span>
<span class="s">"_ns"</span><span class="p">:</span> <span class="s">"task_status"</span><span class="p">,</span>
<span class="s">"start_time"</span><span class="p">:</span> <span class="s">"2019-01-15T20:54:09Z"</span><span class="p">,</span>
<span class="s">"traceback"</span><span class="p">:</span> <span class="s">"Traceback (most recent call last):</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/celery/app/trace.py</span><span class="se">\"</span><span class="s">, line 367, in trace_task</span><span class="se">\n</span><span class="s"> R = retval = fun(*args, **kwargs)</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py</span><span class="se">\"</span><span class="s">, line 529, in __call__</span><span class="se">\n</span><span class="s"> return super(Task, self).__call__(*args, **kwargs)</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py</span><span class="se">\"</span><span class="s">, line 107, in __call__</span><span class="se">\n</span><span class="s"> return super(PulpTask, self).__call__(*args, **kwargs)</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/celery/app/trace.py</span><span class="se">\"</span><span class="s">, line 622, in __protected_call__</span><span class="se">\n</span><span class="s"> return self.run(*args, **kwargs)</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py</span><span class="se">\"</span><span class="s">, line 770, in sync</span><span class="se">\n</span><span class="s"> sync_report = sync_repo(transfer_repo, conduit, call_config)</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py</span><span class="se">\"</span><span class="s">, line 737, in wrap_f</span><span class="se">\n</span><span class="s"> return f(*args, **kwargs)</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py</span><span class="se">\"</span><span class="s">, line 85, in sync_repo</span><span class="se">\n</span><span class="s"> return self.sync_step.process_lifecycle()</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py</span><span class="se">\"</span><span class="s">, line 572, in process_lifecycle</span><span class="se">\n</span><span class="s"> super(PluginStep, self).process_lifecycle()</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py</span><span class="se">\"</span><span class="s">, line 163, in process_lifecycle</span><span class="se">\n</span><span class="s"> step.process()</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py</span><span class="se">\"</span><span class="s">, line 256, in process</span><span class="se">\n</span><span class="s"> self._process_block()</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py</span><span class="se">\"</span><span class="s">, line 303, in _process_block</span><span class="se">\n</span><span class="s"> self.process_main()</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py</span><span class="se">\"</span><span class="s">, line 237, in process_main</span><span class="se">\n</span><span class="s"> self._process_manifest_list(manifest, digest, available_blobs, tag)</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py</span><span class="se">\"</span><span class="s">, line 283, in _process_manifest_list</span><span class="se">\n</span><span class="s"> self._process_manifest(manifest, digest, available_blobs, tag=tag)</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py</span><span class="se">\"</span><span class="s">, line 313, in _process_manifest</span><span class="se">\n</span><span class="s"> manifest = models.Manifest.from_json(manifest, digest)</span><span class="se">\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp_docker/plugins/models.py</span><span class="se">\"</span><span class="s">, line 219, in from_json</span><span class="se">\n</span><span class="s"> fs_layers = [FSLayer(blob_sum=layer['blobSum']) for layer in manifest['fsLayers']]</span><span class="se">\n</span><span class="s">KeyError: 'fsLayers'</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span>
<span class="s">"spawned_tasks"</span><span class="p">:</span> <span class="p">[],</span>
<span class="s">"progress_report"</span><span class="p">:</span> <span class="p">{</span>
<span class="s">"docker_importer"</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="s">"num_success"</span><span class="p">:</span> <span class="mi">5</span><span class="p">,</span>
<span class="s">"description"</span><span class="p">:</span> <span class="s">"Downloading manifests"</span><span class="p">,</span>
<span class="s">"step_type"</span><span class="p">:</span> <span class="s">"sync_step_metadata"</span><span class="p">,</span>
<span class="s">"items_total"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s">"state"</span><span class="p">:</span> <span class="s">"FAILED"</span><span class="p">,</span>
<span class="s">"error_details"</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="s">"traceback"</span><span class="p">:</span> <span class="s">" File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py</span><span class="se">\"</span><span class="s">, line 256, in process</span><span class="se">\n</span><span class="s"> self._process_block()</span><span class="se">\n\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py</span><span class="se">\"</span><span class="s">, line 303, in _process_block</span><span class="se">\n</span><span class="s"> self.process_main()</span><span class="se">\n\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py</span><span class="se">\"</span><span class="s">, line 237, in process_main</span><span class="se">\n</span><span class="s"> self._process_manifest_list(manifest, digest, available_blobs, tag)</span><span class="se">\n\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py</span><span class="se">\"</span><span class="s">, line 283, in _process_manifest_list</span><span class="se">\n</span><span class="s"> self._process_manifest(manifest, digest, available_blobs, tag=tag)</span><span class="se">\n\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py</span><span class="se">\"</span><span class="s">, line 313, in _process_manifest</span><span class="se">\n</span><span class="s"> manifest = models.Manifest.from_json(manifest, digest)</span><span class="se">\n\n</span><span class="s"> File </span><span class="se">\"</span><span class="s">/usr/lib/python2.7/site-packages/pulp_docker/plugins/models.py</span><span class="se">\"</span><span class="s">, line 219, in from_json</span><span class="se">\n</span><span class="s"> fs_layers = [FSLayer(blob_sum=layer['blobSum']) for layer in manifest['fsLayers']]</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span>
<span class="s">"error"</span><span class="p">:</span> <span class="s">"'fsLayers'"</span>
<span class="p">}</span>
<span class="p">],</span>
<span class="s">"details"</span><span class="p">:</span> <span class="s">""</span><span class="p">,</span>
<span class="s">"num_failures"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s">"step_id"</span><span class="p">:</span> <span class="s">"77747e24-bc58-46ed-b5a2-60c21ba5c5ad"</span><span class="p">,</span>
<span class="s">"num_processed"</span><span class="p">:</span> <span class="mi">6</span>
<span class="p">},</span>
<span class="p">{</span>
<span class="s">"num_success"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="s">"description"</span><span class="p">:</span> <span class="s">"Copying units already in pulp"</span><span class="p">,</span>
<span class="s">"step_type"</span><span class="p">:</span> <span class="s">"get_local"</span><span class="p">,</span>
<span class="s">"items_total"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s">"state"</span><span class="p">:</span> <span class="s">"NOT_STARTED"</span><span class="p">,</span>
<span class="s">"error_details"</span><span class="p">:</span> <span class="p">[],</span>
<span class="s">"details"</span><span class="p">:</span> <span class="s">""</span><span class="p">,</span>
<span class="s">"num_failures"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="s">"step_id"</span><span class="p">:</span> <span class="s">"695588b1-9004-4bce-ac6a-4f20bb1209c8"</span><span class="p">,</span>
<span class="s">"num_processed"</span><span class="p">:</span> <span class="mi">0</span>
<span class="p">},</span>
<span class="p">{</span>
<span class="s">"num_success"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="s">"description"</span><span class="p">:</span> <span class="s">"Copying units already in pulp"</span><span class="p">,</span>
<span class="s">"step_type"</span><span class="p">:</span> <span class="s">"get_local"</span><span class="p">,</span>
<span class="s">"items_total"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s">"state"</span><span class="p">:</span> <span class="s">"NOT_STARTED"</span><span class="p">,</span>
<span class="s">"error_details"</span><span class="p">:</span> <span class="p">[],</span>
<span class="s">"details"</span><span class="p">:</span> <span class="s">""</span><span class="p">,</span>
<span class="s">"num_failures"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="s">"step_id"</span><span class="p">:</span> <span class="s">"d211ca5e-bcf5-49df-a390-0e0016b4246f"</span><span class="p">,</span>
<span class="s">"num_processed"</span><span class="p">:</span> <span class="mi">0</span>
<span class="p">},</span>
<span class="p">{</span>
<span class="s">"num_success"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="s">"description"</span><span class="p">:</span> <span class="s">"Downloading remote files"</span><span class="p">,</span>
<span class="s">"step_type"</span><span class="p">:</span> <span class="s">"sync_step_download"</span><span class="p">,</span>
<span class="s">"items_total"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s">"state"</span><span class="p">:</span> <span class="s">"NOT_STARTED"</span><span class="p">,</span>
<span class="s">"error_details"</span><span class="p">:</span> <span class="p">[],</span>
<span class="s">"details"</span><span class="p">:</span> <span class="s">""</span><span class="p">,</span>
<span class="s">"num_failures"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="s">"step_id"</span><span class="p">:</span> <span class="s">"d627e410-d608-4a89-bec1-9709a1172821"</span><span class="p">,</span>
<span class="s">"num_processed"</span><span class="p">:</span> <span class="mi">0</span>
<span class="p">},</span>
<span class="p">{</span>
<span class="s">"num_success"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="s">"description"</span><span class="p">:</span> <span class="s">"Saving Manifests and Blobs"</span><span class="p">,</span>
<span class="s">"step_type"</span><span class="p">:</span> <span class="s">"sync_step_save"</span><span class="p">,</span>
<span class="s">"items_total"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s">"state"</span><span class="p">:</span> <span class="s">"NOT_STARTED"</span><span class="p">,</span>
<span class="s">"error_details"</span><span class="p">:</span> <span class="p">[],</span>
<span class="s">"details"</span><span class="p">:</span> <span class="s">""</span><span class="p">,</span>
<span class="s">"num_failures"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="s">"step_id"</span><span class="p">:</span> <span class="s">"23d120bd-01d3-40b8-ac6c-2cd434aed12f"</span><span class="p">,</span>
<span class="s">"num_processed"</span><span class="p">:</span> <span class="mi">0</span>
<span class="p">},</span>
<span class="p">{</span>
<span class="s">"num_success"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="s">"description"</span><span class="p">:</span> <span class="s">"Saving Tags"</span><span class="p">,</span>
<span class="s">"step_type"</span><span class="p">:</span> <span class="s">"sync_step_save"</span><span class="p">,</span>
<span class="s">"items_total"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s">"state"</span><span class="p">:</span> <span class="s">"NOT_STARTED"</span><span class="p">,</span>
<span class="s">"error_details"</span><span class="p">:</span> <span class="p">[],</span>
<span class="s">"details"</span><span class="p">:</span> <span class="s">""</span><span class="p">,</span>
<span class="s">"num_failures"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="s">"step_id"</span><span class="p">:</span> <span class="s">"97e64823-ef75-4406-ba71-fe7b8a081e61"</span><span class="p">,</span>
<span class="s">"num_processed"</span><span class="p">:</span> <span class="mi">0</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">},</span>
</code></pre>
<p>Changing the source repository to quay.io/etcd works though.<br>
I tried deleting the repo in pulp and re-create it or whitelisting a single tag, no luck.</p>
<p>Here are the repository details:</p>
<pre><code>Importers:
Config:
Feed: https://gcr.io
Tags: 3.2.24
Upstream Name: google-containers/etcd
Id: docker_importer
Importer Type Id: docker_importer
Last Override Config:
Last Sync: 2019-01-15T20:54:12Z
Last Updated: 2019-01-15T20:53:58Z
Repo Id: etcd-gcr2
Scratchpad: None
Distributors:
Auto Publish: True
Config:
Repo-registry-id: google-containers/etcd2
Distributor Type Id: docker_distributor_web
Id: docker_web_distributor_name_cli
Last Override Config:
Last Publish: None
Last Updated: 2019-01-15T20:53:58Z
Repo Id: etcd-gcr2
Scratchpad:
Auto Publish: False
Config:
Repo-registry-id: google-containers/etcd2
Distributor Type Id: docker_distributor_export
Id: docker_export_distributor_name_cli
Last Override Config:
Last Publish: None
Last Updated: 2019-01-15T20:53:58Z
Repo Id: etcd-gcr2
</code></pre> Pulp - Issue #4228 (CLOSED - CURRENTRELEASE): Update docs with instructions to install Mongo DB 3.xhttps://pulp.plan.io/issues/42282018-12-05T09:03:02Zttereshcttereshc@redhat.com
<p>Installation docs suggest to configure EPEL7 which has only MongoDB 2.6.<br>
In many cases users will benefit using newer MongoDB.</p> Crane - Issue #4227 (CLOSED - CURRENTRELEASE): Crane 3.2 to 3.3 - Javascript errorhttps://pulp.plan.io/issues/42272018-12-05T07:55:38ZPoil
<p>After upgrading from Crane 3.2 to 3.3, the webpage stays white and I have a javascript error</p>
<pre><code>TypeError: $(...).dataTable is not a function repositories:62:25
<anonyme>
https://repos.xxxx:8443/crane/repositories:62:25
c
jquery-1.10.2.js:3048:9
fireWith
jquery-1.10.2.js:3160:6
ready
jquery-1.10.2.js:433:2
q
jquery-1.10.2.js:104:3
</code></pre> Debian Support - Issue #4176 (CLOSED - CURRENTRELEASE): from_file() and from_metadata() create di...https://pulp.plan.io/issues/41762018-11-21T14:21:58Zquba42
<p>When a debian package is added to mongodb using the from_file() function, then the REL_FIELDS will contain lists of dicts.</p>
<p>If they are added using the from_metadata() function instead, then the same information is stored in a single string as found in a Packages file instead.</p>
<p>Both functions are located in:</p>
<pre><code class="text syntaxhl" data-language="text">plugins/pulp_deb/plugins/db/models.py
</code></pre>
<p>Currently sync.py uses from_metadata, while importer.py uses from_file.<br>
This can lead to structurally inconsistent databases.<br>
While this is surely undesirable I am not aware of any known issues being caused by it.</p>
<p>I only noticed this issue because it causes a bug in a PR of mine: <a href="https://github.com/pulp/pulp_deb/pull/57" class="external">https://github.com/pulp/pulp_deb/pull/57</a><br>
I will amend the PR to be able to handle both types of db entries.</p>
<p>In my estimation, to fully fix the problem such that all future db entries will be consistent, would require some significant work.<br>
I am unsure if it is worth fixing at all, given that we all hope to switch to pulp3 in the not to distant future.</p>
<p>That being said, any code that accesses any REL_FIELDS in the db should be aware of this issue.<br>
A test to explicitly test for compatibility with the two cases is also highly desirable, since the current test suite did not catch this issue in my PR.</p>
<p>Once I get around to it, I may add a complete example how some example package would end up in the db for each of the two functions.</p> Debian Support - Issue #4148 (CLOSED - CURRENTRELEASE): Fix flake8 violationshttps://pulp.plan.io/issues/41482018-11-12T20:37:59Zmihai.ibanescu@gmail.commihai.ibanescu@gmail.comDebian Support - Issue #4008 (CLOSED - CURRENTRELEASE): python-debpkgr does not structure Reposit...https://pulp.plan.io/issues/40082018-09-13T16:16:59Zquba42
<p>python-debpkgr (a dependency of Pulp Debian Support) does not structure Debian repositories quite right:</p>
<p>The Debian repository structure (<a href="https://wiki.debian.org/DebianRepository/Format" class="external">https://wiki.debian.org/DebianRepository/Format</a>) is built from the following schema:</p>
<p><repo_base_url>/dists/<distribution>/<plain_component>/Packages<br>
<repo_base_url>/dists/<distribution>/Release</p>
<p>Here, <distribution> may contain additional "/" (like "stretch/updates").<br>
Even were components are listed in the Release file as "updates/main", only the "main" part is the "<plain_component>". (slightly simplified.)</p>
<p>However, the debpkgr data structure of the relevant object is built entirely from a <repo_base_url> and a Release file.<br>
Therfore, debpkgr has no way of knowing what the <distribution> part of the paths should be.<br>
It builds the following paths instead:</p>
<p><repo_base_url>/dists/<Releasefile.codename>/<Releasefile.component>/Packages<br>
<repo_base_url>/dists/<Releasefile.codename>/Release</p>
<p>For most official debian repos this happens to work since <Releasefile.codename>=<distributrion>.<br>
However for debian security the <Releasefile.codename> is for example "stretch" and the <distribution> is "stretch/updates".<br>
The reason we can still synchronize debian security is because <Releasefile.component> happens to be "update/stretch" while <plain_component> is "stetch". (Two wrongs do make a right...)</p>
<p>However the downstream repository in pulp will have a different structure than the upstream Debian repository, making it impossible to reuse this downstream repository as a new upstream repository. In other words, pulp to pulp synchronization is broken.</p> Pulp - Issue #3896 (CLOSED - CURRENTRELEASE): pulp_docker importer discards relevant exception de...https://pulp.plan.io/issues/38962018-08-02T01:01:49Zrmcgoverrmcgover@redhat.com
<p>We recently hit an exception on our Pulp installation with pulp_docker:</p>
<pre><code>{
"exception": null,
"task_type": "pulp.server.managers.content.upload.import_uploaded_unit",
"_href": "/pulp/api/v2/tasks/7f3e683b-0e29-4c6e-a600-b962932a0bb0/",
"task_id": "7f3e683b-0e29-4c6e-a600-b962932a0bb0",
"tags": [
"pulp:repository:redhat-rhel",
"pulp:action:import_upload"
],
"finish_time": "2018-08-01T17:44:10Z",
"_ns": "task_status",
"start_time": "2018-08-01T17:44:10Z",
"traceback": "Traceback (most recent call last):\n File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in trace_task\n R = retval = fun(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 529, in __call__\n return super(Task, self).__call__(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 107, in __call__\n return super(PulpTask, self).__call__(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 438, in __protected_call__\n return self.run(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/managers/content/upload.py\", line 223, in import_uploaded_unit\n unit_type=unit_type_id, summary=result['summary'], details=result['details']\nPulpCodedException: The importer docker_importer indicated a failed response when uploading docker_manifest_list unit to repository redhat-rhel.\n",
"spawned_tasks": [],
"progress_report": {},
"state": "error",
"worker_name": "reserved_resource_worker-3@brew-pulp-docker02.web.prod.ext.phx2.redhat.com",
"result": null,
"error": {
"code": "PLP0047",
"data": {
"unit_type": "docker_manifest_list",
"importer_id": "docker_importer",
"repo_id": "redhat-rhel",
"details": {},
"summary": ""
},
"description": "The importer docker_importer indicated a failed response when uploading docker_manifest_list unit to repository redhat-rhel.",
"sub_errors": []
},
"_id": {
"$oid": "5b61f151e95a3bea68e6ef47"
},
"id": "5b61f151e95a3bea68e6ef47"
}
</code></pre>
<p>With formatted traceback:</p>
<pre><code> Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 529, in __call__
return super(Task, self).__call__(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
return super(PulpTask, self).__call__(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
return self.run(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/pulp/server/managers/content/upload.py", line 223, in import_uploaded_unit
unit_type=unit_type_id, summary=result['summary'], details=result['details']
PulpCodedException: The importer docker_importer indicated a failed response when uploading docker_manifest_list unit to repository redhat-rhel.
</code></pre>
<p>Note that the error reported via the API has empty summary & details, the traceback doesn't point at the source of the error, and the system logs contain no useful information.</p>
<p>Looking through the code, it seems this unhelpful error handling is introduced here in plugins/pulp_docker/plugins/importers/importer.py, upload_unit method:</p>
<pre><code> try:
upload_step = upload.UploadStep(repo=repo, file_path=file_path, config=config,
metadata=metadata, type_id=type_id)
upload_step.process_lifecycle()
except PulpCodedValidationException:
raise
except Exception as e:
return {'success_flag': False, 'summary': e.message, 'details': {}}
</code></pre>
<p>Consider if an exception is raised during UploadStep constructor or process_lifecycle. Then Pulp discards all information about the exception except for the "message" (which can be empty as not all types of exceptions use a message). Thus it's virtually impossible to figure out what failed. Can it please be updated to not discard critical error information?</p>
<p><strong>Actual behavior:</strong> when an exception other than PulpCodedValidationException occurs during UploadStep on docker imports, all information about the exception is discarded other than the message.</p>
<p><strong>Expected behavior:</strong> when an exception occurs during UploadStep, the exception details are available either through logs or through API, including at least the type of the exception and the traceback.</p>
<p>Checked as of pulp_docker commit fd8d4e767ba9ca0.</p>