Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-08-11T22:27:01ZPulp
Planio RPM Support - Issue #9233 (CLOSED - DUPLICATE): Downloaded content seems to be removed when a tas...https://pulp.plan.io/issues/92332021-08-11T22:27:01Zlmjachky
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2294":<a href="https://github.com/pulp/pulp_rpm/issues/2294" class="external">https://github.com/pulp/pulp_rpm/issues/2294</a></p>
<hr>
<p>For large repositories, this means that when performing <strong>immediate</strong> synchronization, all the downloaded content is lost after the task's failure. Pulp is then trying to download the content (artifacts) once again from scratch when issuing re-syncing, rather than associating the orphaned artifacts with corresponding content units.</p>
<p>More info can be found here: <a href="https://community.theforeman.org/t/oracle-linux-8-appstream-sync-fails/24676/9" class="external">https://community.theforeman.org/t/oracle-linux-8-appstream-sync-fails/24676/9</a>. The following part is the most important to us:</p>
<blockquote>
<p>If you are performing one of these large syncs, and get part way through before the sync fails with the timeout, it seems everything which was downloaded into the tmp directory is thrown away, and not kept. So, the next time the sync is tried, it has to download everything again.</p>
</blockquote>
<p>Used versions: pulpcore 3.14.3 and pulp_rpm 3.14.0</p>
<p>This behaviour could not be reproduced for pulp_file (<a href="https://fixtures.pulpproject.org/file/" class="external">https://fixtures.pulpproject.org/file/</a>; pulpcore 3.15.0.dev and pulp_file 1.9.0.dev); therefore, we should focus on the pulp_rpm plugin.</p> RPM Support - Issue #9208 (CLOSED - DUPLICATE): Published .treeinfo metadata not matching expecta...https://pulp.plan.io/issues/92082021-08-04T04:48:45Zdalleydalley@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2292":<a href="https://github.com/pulp/pulp_rpm/issues/2292" class="external">https://github.com/pulp/pulp_rpm/issues/2292</a></p>
<hr>
<ol>
<li>"packages" field on variants may not match with reality in the mirror case. You can see that the "repository" and "packages" fields for the "variant-External" variant which previously had both the repository and packages listed as "../rpm-signed/" now maps to "External" and "External/Packages", respectively. But in the event of mirror sync, this is incorrect.</li>
</ol>
<p>Likewise, the Land variant is wrong - but this may be a fixture issue. There is no such "Packages" directory. <a href="https://fixtures.pulpproject.org/rpm-distribution-tree/variants/land/" class="external">https://fixtures.pulpproject.org/rpm-distribution-tree/variants/land/</a></p>
<pre><code> ('change', 'variant-Land.packages', ('Packages', 'Land/Packages')),
('change', 'variant-Land.repository', ('variants/land', 'Land')),
('change',
'variant-External.packages',
('../rpm-signed/', 'External/Packages')),
('change', 'variant-External.repository', ('../rpm-signed/', 'External'))]
</code></pre>
<ol start="2">
<li>The order of the "variants" is different from the original - since productmd sometimes places significance on the first variant in the list, this could possibly have an impact in some cases.</li>
</ol>
<pre><code>[('change', 'general.variants', ('Land,Sea,External', 'External,Land,Sea')),
('change', 'tree.variants', ('Land,Sea,External', 'External,Land,Sea')),
</code></pre>
<p><code>test_publish.py::DistributionTreeMetadataTestCase</code> needs to be updated for both of these</p> Migration Plugin - Issue #9191 (CLOSED - DUPLICATE): Migrated advisories with dates-as-timestamps...https://pulp.plan.io/issues/91912021-08-02T19:37:16Zggainey
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp-2to3-migration/issues/483": <a href="https://github.com/pulp/pulp-2to3-migration/issues/483" class="external">https://github.com/pulp/pulp-2to3-migration/issues/483</a></p> Pulp - Issue #9009 (CLOSED - DUPLICATE): CacheBasicOperationsTestCase::test_04_basic_expires unit...https://pulp.plan.io/issues/90092021-07-03T14:52:45Zdkliban@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulpcore/2023":<a href="https://github.com/pulp/pulpcore/issues/2023" class="external">https://github.com/pulp/pulpcore/issues/2023</a></p>
<hr>
<p>I am seeing the following in our CI from time to time:</p>
<pre><code># Run unit tests.
cmd_prefix bash -c "PULP_DATABASES__default__USER=postgres django-admin test --noinput /usr/local/lib/python3.6/site-packages/pulpcore/tests/unit/"
Creating test database for alias 'default'...
Access policy for groups/users created.
Access policy for groups created.
Access policy for tasks updated.
Deleting Guardians' AnonymousUser
System check identified no issues (0 silenced).
.............................F.......s.............................................pulp [None]: asyncio:ERROR: Task was destroyed but it is pending!
task: <Task pending coro=<ArtifactDownloader._handle_content_unit() running at /usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py:207> wait_for=<_GatheringFuture pending cb=[<TaskWakeupMethWrapper object at 0x7f8fd9a270d8>()]>>
pulp [None]: asyncio:ERROR: Task was destroyed but it is pending!
task: <Task pending coro=<ArtifactDownloader._handle_content_unit() running at /usr/local/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py:207> wait_for=<_GatheringFuture pending cb=[<TaskWakeupMethWrapper object at 0x7f8fd9a27468>()]>>
.............
======================================================================
FAIL: test_04_basic_expires (pulpcore.tests.unit.test_cache.CacheBasicOperationsTestCase)
Tests setting values with expiration times
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/pulpcore/tests/unit/test_cache.py", line 42, in test_04_basic_expires
self.assertIsNone(ret)
AssertionError: b'hi' is not None
----------------------------------------------------------------------
Ran 96 tests in 6.627s
</code></pre> RPM Support - Issue #8992 (CLOSED - DUPLICATE): Memory usage when destroy repository; The memory ...https://pulp.plan.io/issues/89922021-06-30T07:11:25Zwilful
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2280":<a href="https://github.com/pulp/pulp_rpm/issues/2280" class="external">https://github.com/pulp/pulp_rpm/issues/2280</a></p>
<hr>
<p>When i try to destroy one of repositories, pulp (RQ basically) are using more 3G RAM and crashed by OOM-Killer</p>
<pre><code class="c syntaxhl" data-language="c"><span class="n">Jun</span> <span class="mi">30</span> <span class="mo">06</span><span class="o">:</span><span class="mi">59</span><span class="o">:</span><span class="mi">16</span> <span class="o">***</span> <span class="n">kernel</span><span class="o">:</span> <span class="n">Out</span> <span class="n">of</span> <span class="n">memory</span><span class="o">:</span> <span class="n">Kill</span> <span class="n">process</span> <span class="mi">81458</span> <span class="p">(</span><span class="n">rq</span><span class="p">)</span> <span class="n">score</span> <span class="mi">493</span> <span class="n">or</span> <span class="n">sacrifice</span> <span class="n">child</span>
<span class="n">Jun</span> <span class="mi">30</span> <span class="mo">06</span><span class="o">:</span><span class="mi">59</span><span class="o">:</span><span class="mi">16</span> <span class="o">***</span> <span class="n">kernel</span><span class="o">:</span> <span class="n">Killed</span> <span class="n">process</span> <span class="mi">81458</span> <span class="p">(</span><span class="n">rq</span><span class="p">),</span> <span class="n">UID</span> <span class="mi">1002</span><span class="p">,</span> <span class="n">total</span><span class="o">-</span><span class="n">vm</span><span class="o">:</span><span class="mi">4434448</span><span class="n">kB</span><span class="p">,</span> <span class="n">anon</span><span class="o">-</span><span class="n">rss</span><span class="o">:</span><span class="mi">3936104</span><span class="n">kB</span><span class="p">,</span> <span class="n">file</span><span class="o">-</span><span class="n">rss</span><span class="o">:</span><span class="mi">0</span><span class="n">kB</span><span class="p">,</span> <span class="n">shmem</span><span class="o">-</span><span class="n">rss</span><span class="o">:</span><span class="mi">0</span><span class="n">kB</span>
</code></pre>
<p>State before on my node:</p>
<pre><code class="c syntaxhl" data-language="c"><span class="cp"># free -h
</span> <span class="n">total</span> <span class="n">used</span> <span class="n">free</span> <span class="n">shared</span> <span class="n">buff</span><span class="o">/</span><span class="n">cache</span> <span class="n">available</span>
<span class="n">Mem</span><span class="o">:</span> <span class="mi">7</span><span class="p">.</span><span class="mi">6</span><span class="n">G</span> <span class="mi">1</span><span class="p">.</span><span class="mi">5</span><span class="n">G</span> <span class="mi">3</span><span class="p">.</span><span class="mi">6</span><span class="n">G</span> <span class="mi">2</span><span class="p">.</span><span class="mi">2</span><span class="n">G</span> <span class="mi">2</span><span class="p">.</span><span class="mi">6</span><span class="n">G</span> <span class="mi">3</span><span class="p">.</span><span class="mi">7</span><span class="n">G</span>
<span class="n">Swap</span><span class="o">:</span> <span class="mi">0</span><span class="n">B</span> <span class="mi">0</span><span class="n">B</span> <span class="mi">0</span><span class="n">B</span>
</code></pre>
<pre><code class="c syntaxhl" data-language="c"> <span class="s">"versions"</span><span class="o">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="s">"component"</span><span class="o">:</span> <span class="s">"core"</span><span class="p">,</span>
<span class="s">"version"</span><span class="o">:</span> <span class="s">"3.13.0"</span>
<span class="p">},</span>
<span class="p">{</span>
<span class="s">"component"</span><span class="o">:</span> <span class="s">"rpm"</span><span class="p">,</span>
<span class="s">"version"</span><span class="o">:</span> <span class="s">"3.13.0"</span>
<span class="p">},</span>
<span class="p">{</span>
<span class="s">"component"</span><span class="o">:</span> <span class="s">"python"</span><span class="p">,</span>
<span class="s">"version"</span><span class="o">:</span> <span class="s">"3.4.0"</span>
<span class="p">},</span>
<span class="p">{</span>
<span class="s">"component"</span><span class="o">:</span> <span class="s">"file"</span><span class="p">,</span>
<span class="s">"version"</span><span class="o">:</span> <span class="s">"1.8.0"</span>
<span class="p">},</span>
<span class="p">{</span>
<span class="s">"component"</span><span class="o">:</span> <span class="s">"deb"</span><span class="p">,</span>
<span class="s">"version"</span><span class="o">:</span> <span class="s">"2.13.0"</span>
<span class="p">},</span>
<span class="p">{</span>
<span class="s">"component"</span><span class="o">:</span> <span class="s">"container"</span><span class="p">,</span>
<span class="s">"version"</span><span class="o">:</span> <span class="s">"2.6.0"</span>
<span class="p">}</span>
<span class="p">],</span>
</code></pre> Pulp - Issue #8988 (CLOSED - CURRENTRELEASE): `pulpcore-worker` startup should remove old worker ...https://pulp.plan.io/issues/89882021-06-29T19:37:18Zbmbouterbmbouter@redhat.com
<a name="To-reproduce"></a>
<h2 >To reproduce<a href="#To-reproduce" class="wiki-anchor">¶</a></h2>
<ol>
<li>Start a pulpcore-worker against an empty database</li>
<li>Observe that the status API shows that worker</li>
<li>kill -9 your worker</li>
<li>Observe the status API after 30 seconds no longer shows your worker</li>
<li>Start your pulpcore-worker again</li>
<li>Go into shell_plus and observe this query shows 2 workers present: <code>Workers.objects.count()</code>
</li>
</ol>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<pre><code>In [1]: Worker.objects.count()
Out[1]: 1
</code></pre>
<a name="Solution"></a>
<h2 >Solution<a href="#Solution" class="wiki-anchor">¶</a></h2>
<p>Have workers that startup run a query that delete any workers that haven't issued a heartbeat in say 7 days. This does not need to be configurable. The 7 day idea is to make something that won't let records accumulate on the long term, but leave them in place for someone to look at the db post-mortem and still see them for investigation.</p> RPM Support - Issue #8720 (CLOSED - DUPLICATE): Published RPM metadata isn't sorted properlyhttps://pulp.plan.io/issues/87202021-05-07T22:09:20Zdalleydalley@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2274":<a href="https://github.com/pulp/pulp_rpm/issues/2274" class="external">https://github.com/pulp/pulp_rpm/issues/2274</a></p>
<hr>
<p>RPM metadata should be published in-order, which helps with compression efficency (see associated BZ). createrepo_c does this, but not via the library itself, so Pulp is still publishing unordered metadata.</p>
<p>Note that the metadata is "fine", it works, it's just inefficient to compress.</p>
<p>createrepo_c uses location_href as the sort key.</p>
<p>Problem: Pulp mixes location_href's together from many different repositories, and because they are meaningless, it basically ignores them. So we store useless data in the database.</p>
<p>We should remove the location_href and location_base fields (the latter is entirely unused), and replace them with just a filename, which we can possibly use to reconstruct a location_href if we need to keep it for backwards compatibility. Then we can properly sort by it, and we can use it directly in various places without needing to rewrite the value constantly.</p>
<p>It is not a "real" part of the RPM package metadata, only a value which createrepo_c happens to provide on the objects which we copied over. This probably shouldn't have been done.</p>
<p>We should therefore sort by the filename, which is basically equivalent to sorting by location_href since within a repository all the packages should have the same directory.</p> Debian Support - Issue #8544 (CLOSED - DUPLICATE): Add some translation files to at least one tes...https://pulp.plan.io/issues/85442021-04-12T15:36:57Zquba42
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_deb/401":<a href="https://github.com/pulp/pulp_deb/issues/401" class="external">https://github.com/pulp/pulp_deb/issues/401</a></p>
<hr>
<p>Should provide test coverage for the following:</p>
<ul>
<li><a href="https://pulp.plan.io/issues/8410" class="external">https://pulp.plan.io/issues/8410</a></li>
<li><a href="https://pulp.plan.io/issues/8096" class="external">https://pulp.plan.io/issues/8096</a></li>
</ul> RPM Support - Issue #8229 (CLOSED - NOTABUG): Do not replace advisory if only 'updated_date' has ...https://pulp.plan.io/issues/82292021-02-09T07:18:23Zadam.winberg@smhi.se
<p>It seems that EPEL sets a new 'updated_date' every day, so when I run my daily sync I get:</p>
<pre><code> "content_summary": {
"added": {
"rpm.advisory": {
"count": 2540,
"href": "/pulp/api/v3/content/rpm/advisories/?repository_version_added=/pulp/api/v3/repositories/rpm/rpm/10e51ae6-65c7-42aa-8ab1-ffebdf752500/versions/19/"
},
"rpm.package": {
"count": 6,
"href": "/pulp/api/v3/content/rpm/packages/?repository_version_added=/pulp/api/v3/repositories/rpm/rpm/10e51ae6-65c7-42aa-8ab1-ffebdf752500/versions/19/"
}
},
"removed": {
"rpm.advisory": {
"count": 2536,
"href": "/pulp/api/v3/content/rpm/advisories/I've tested this and it works?repository_version_removed=/pulp/api/v3/repositories/rpm/rpm/10e51ae6-65c7-42aa-8ab1-ffebdf752500/versions/19/"
}
}
},
</code></pre>
<p>Inspecting the advisories, the only thing changed is the 'updated_date' metadata, version and package lists are identical. EPEL seems to set a new 'updated_date' each day on existing advisories, and in such a case there is not much sense in replacing the existing advisory with the 'new' one. I run weekly scripts to copy new advisories to a frozen repo which is a lot harder to do when thousands of new advisories are coming in every day.</p>
<p>I looked at the code ("pulp_rpm/app/advisory.py") and propose that an additional test is made in the "resolve_advisory_conflict" function, a crude example:</p>
<pre><code> elif not same_dates and pkgs_intersection:
if same_version and previous_pkglist == added_pkglist:
# Ignore new advisory if only 'updated_date' has changed
to_exclude.append(added_advisory.pk)
</code></pre> Pulp - Issue #8187 (CLOSED - DUPLICATE): Can't use global checks in boolean expressions in policyhttps://pulp.plan.io/issues/81872021-02-01T20:17:45Zttereshcttereshc@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulpcore/1964":<a href="https://github.com/pulp/pulpcore/issues/1964" class="external">https://github.com/pulp/pulpcore/issues/1964</a></p>
<hr>
<p>If the condition contains a boolean expression like below, and even though <code>has_repo_attr_model_or_obj_perms</code> is defined in global checks, it fails.</p>
<pre><code> {
"action": ["destroy"],
"principal": "authenticated",
"effect": "allow",
"condition": [
"(has_repo_attr_model_or_obj_perms:container.delete_containerrepository_versions or"
" has_repo_attr_model_or_obj_perms:container.view_containerrepository)",
]
}
</code></pre>
<pre><code>Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: pulp [8f28125057bf41f1b157a7cb47848fca]: django.request:ERROR: Internal Server Error: /pulp/api/v3/repositories/container/container/46e44a17-dd48-4918-aa99-e54b6ea56b8d/versions/2/
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: Traceback (most recent call last):
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: response = get_response(request)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: response = self.process_exception_by_middleware(e, request)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: response = wrapped_callback(request, *callback_args, **callback_kwargs)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: return view_func(*args, **kwargs)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rest_framework/viewsets.py", line 125, in view
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: return self.dispatch(request, *args, **kwargs)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: response = self.handle_exception(exc)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: self.raise_uncaught_exception(exc)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: raise exc
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rest_framework/views.py", line 497, in dispatch
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: self.initial(request, *args, **kwargs)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/home/vagrant/devel/pulpcore/pulpcore/app/viewsets/base.py", line 301, in initial
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: super().initial(request, *args, **kwargs)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rest_framework/views.py", line 415, in initial
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: self.check_permissions(request)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rest_framework/views.py", line 332, in check_permissions
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: if not permission.has_permission(request, self):
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rest_access_policy/access_policy.py", line 23, in has_permission
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: return self._evaluate_statements(statements, request, view, action)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rest_access_policy/access_policy.py", line 59, in _evaluate_statements
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: matched = self._get_statements_matching_context_conditions(
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rest_access_policy/access_policy.py", line 160, in _get_statements_matching_context_conditions
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: passed = self._check_condition(condition, request, view, action)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rest_access_policy/access_policy.py", line 181, in _check_condition
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: method = self._get_condition_method(method_name)
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: File "/usr/local/lib/pulp/lib64/python3.8/site-packages/rest_access_policy/access_policy.py", line 209, in _get_condition_method
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: raise AccessPolicyException(
Feb 01 19:17:34 pulp3-source-fedora32.rhgoose.example.com gunicorn[132380]: rest_access_policy.exceptions.AccessPolicyException: condition '(has_repo_attr_model_or_obj_perms' must be a method on the access policy or be defined in the 'reusable_conditions' module
</code></pre>
<p>The workaround is to split it into 2 policy statements:</p>
<pre><code> {
"action": ["destroy"],
"principal": "authenticated",
"effect": "allow",
"condition": [
"has_repo_attr_model_or_obj_perms:container.delete_containerrepository_versions",
]
},
{
"action": ["destroy"],
"principal": "authenticated",
"effect": "allow",
"condition": [
"has_repo_attr_model_or_obj_perms:container.delete_containerrepository",
]
},
</code></pre> Debian Support - Issue #8028 (CLOSED - WORKSFORME): Cannot sync Ubuntu repositorieshttps://pulp.plan.io/issues/80282020-12-29T03:40:11Znhavens
<p>With the following remote configuration, I'm unable to sync to a pulp repository:</p>
<pre><code class="json syntaxhl" data-language="json"><span class="p">{</span><span class="w">
</span><span class="nl">"pulp_href"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/remotes/deb/apt/d5143b62-3f97-4f10-a466-734dcd4d5146/"</span><span class="p">,</span><span class="w">
</span><span class="nl">"pulp_created"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2020-12-29T02:49:24.689377Z"</span><span class="p">,</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"ubuntu"</span><span class="p">,</span><span class="w">
</span><span class="nl">"url"</span><span class="p">:</span><span class="w"> </span><span class="s2">"http://mirror.us.leaseweb.net/ubuntu"</span><span class="p">,</span><span class="w">
</span><span class="nl">"ca_cert"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"client_cert"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"client_key"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"tls_validation"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w">
</span><span class="nl">"proxy_url"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"username"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"password"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"pulp_last_updated"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2020-12-29T03:06:33.546708Z"</span><span class="p">,</span><span class="w">
</span><span class="nl">"download_concurrency"</span><span class="p">:</span><span class="w"> </span><span class="mi">10</span><span class="p">,</span><span class="w">
</span><span class="nl">"policy"</span><span class="p">:</span><span class="w"> </span><span class="s2">"immediate"</span><span class="p">,</span><span class="w">
</span><span class="nl">"total_timeout"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"connect_timeout"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"sock_connect_timeout"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"sock_read_timeout"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"distributions"</span><span class="p">:</span><span class="w"> </span><span class="s2">"bionic bionic-backports bionic-proposed bionic-security bionic-updates"</span><span class="p">,</span><span class="w">
</span><span class="nl">"components"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"architectures"</span><span class="p">:</span><span class="w"> </span><span class="s2">"amd64 i386"</span><span class="p">,</span><span class="w">
</span><span class="nl">"sync_sources"</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="p">,</span><span class="w">
</span><span class="nl">"sync_udebs"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w">
</span><span class="nl">"sync_installer"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w">
</span><span class="nl">"gpgkey"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"ignore_missing_package_indices"</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="w">
</span></code></pre>
<p>I get the following error message:</p>
<pre><code>Dec 28 21:32:42 pulp-server rq[48697]: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "core_artifact_sha256_key"
Dec 28 21:32:42 pulp-server rq[48697]: DETAIL: Key (sha256)=(e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855) already exists.
</code></pre>
<p>If I subsequently patch the remote to set <code>ignore_missing_package_indices=true</code>, the sync also fails, but with a different error message:</p>
<pre><code>Dec 28 21:36:19 pulp-server rq[48741]: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "core_contentartifact_content_id_relative_path_d2ef8231_uniq"
Dec 28 21:36:19 pulp-server rq[48741]: DETAIL: Key (content_id, relative_path)=(6bfcee15-6b12-4c36-831d-9b34d1dc9cf8, dists/bionic-backports) already exists.
</code></pre>
<p>I'd be happy to provide full stack traces if that would be helpful.</p> Debian Support - Issue #7756 (CLOSED - WONTFIX): Content upload has different method-definintion ...https://pulp.plan.io/issues/77562020-10-27T15:58:32Zmbucher
<p>Looking at the generated API-Bindings for ruby, the method for creating/uploading a new deb-package differs from the one used for e.g. rpm.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">module</span> <span class="nn">PulpDebClient</span>
<span class="k">class</span> <span class="nc">ContentPackagesApi</span>
<span class="o">...</span>
<span class="k">def</span> <span class="nf">create</span><span class="p">(</span><span class="n">opts</span> <span class="o">=</span> <span class="p">{})</span>
</code></pre>
<p>While in the RPM-bindings it looks like this:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">module</span> <span class="nn">PulpRpmClient</span>
<span class="k">class</span> <span class="nc">ContentPackagesApi</span>
<span class="o">...</span>
<span class="k">def</span> <span class="nf">create</span><span class="p">(</span><span class="n">relative_path</span><span class="p">,</span> <span class="n">opts</span> <span class="o">=</span> <span class="p">{})</span>
</code></pre>
<p>The fact that <code>relative_path</code> is specified as parameter in the latter case seems to be due to the fact that in the API, <code>relative_path</code> is marked as required in pulp_rpm, but not in pulp_deb.</p>
<p><a href="https://pulp-deb.readthedocs.io/en/latest/restapi.html#operation/content_deb_packages_create" class="external">https://pulp-deb.readthedocs.io/en/latest/restapi.html#operation/content_deb_packages_create</a></p>
<p><a href="https://pulp-rpm.readthedocs.io/en/latest/restapi.html#operation/content_rpm_packages_create" class="external">https://pulp-rpm.readthedocs.io/en/latest/restapi.html#operation/content_rpm_packages_create</a></p>
<p>THis came up during the <a href="https://github.com/Katello/katello/pull/8632#issuecomment-709563551" class="external">katello integration of pulp_deb for pulpcore</a>.</p> Pulp - Issue #7715 (CLOSED - CURRENTRELEASE): Spelling error in import-export workflow dochttps://pulp.plan.io/issues/77152020-10-15T15:35:04Zggainey
<p>From <a href="https://docs.pulpproject.org/pulpcore/workflows/import-export.html" class="external">https://docs.pulpproject.org/pulpcore/workflows/import-export.html</a>:</p>
<p>"Finally, if we need complete comtrol over incremental exporting, " - 'comtrol' is not a word :)</p> RPM Support - Issue #7283 (CLOSED - NOTABUG): new ruby bindings send default checksum typeshttps://pulp.plan.io/issues/72832020-08-05T17:53:53Zjsherril@redhat.comjsherril@redhat.com
<p>new bindings send a default checksum type when creating a publication.</p>
<p>Previously the 3.4 ruby bindings when creating a publication resulted in a publication request body of:</p>
<p>{"repository_version":"/pulp/api/v3/repositories/rpm/rpm/03b20f89-a2a3-4a02-8ea5-9af641623d15/versions/0/"}</p>
<p>after updating to pulp_rpm_client (3.6.0.dev01596557138), we now get a request body of:</p>
<p>{"repository_version":"/pulp/api/v3/repositories/rpm/rpm/7fcae337-d94c-4422-bd35-295943f48533/versions/0/","metadata_checksum_type":"sha256","package_checksum_type":"sha256"}</p>
<p>This might be fine, but i wanted to make sure it was intentional.</p> Migration Plugin - Issue #6935 (CLOSED - WONTFIX): Validation failure reports incorrect resources...https://pulp.plan.io/issues/69352020-06-09T12:05:08Zttereshcttereshc@redhat.com
<p><code>rpm_unsigned</code> repo exists, <code>drpm</code> does not.</p>
<p>Migration Plan:</p>
<pre><code class="json syntaxhl" data-language="json"><span class="p">{</span><span class="w"> </span><span class="nl">"plan"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"plugins"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"rpm"</span><span class="p">,</span><span class="w">
</span><span class="nl">"repositories"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"rpm_unsigned"</span><span class="p">,</span><span class="w">
</span><span class="nl">"repository_versions"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"pulp2_repository_id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"rpm_unsigned"</span><span class="p">,</span><span class="w">
</span><span class="nl">"pulp2_distributor_repository_ids"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w"> </span><span class="s2">"rpm_unsigned"</span><span class="p">,</span><span class="w"> </span><span class="s2">"drpm"</span><span class="w"> </span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}}</span><span class="w">
</span></code></pre>
<p>Result (see the description, it should have onlt <code>drpm</code> there):</p>
<pre><code>$ http :/pulp/api/v3/tasks/7bf152e9-971e-48c4-82a7-9bd8a0ce80fe/
HTTP/1.1 200 OK
Allow: GET, PATCH, DELETE, HEAD, OPTIONS
Connection: Keep-Alive
Content-Length: 1250
Content-Type: application/json
Date: Tue, 09 Jun 2020 10:48:03 GMT
Keep-Alive: timeout=5, max=10000
Server: gunicorn/20.0.4
Vary: Accept,Cookie
X-Frame-Options: SAMEORIGIN
{
"child_tasks": [],
"created_resources": [],
"error": {
"description": "Validation failed: resources missing {'repositories_missing_distributors': ['drpm', 'rpm_unsigned']}",
"traceback": " File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 886, in perform_job\n rv = job.perform()\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 664, in perform\n self._result = self._execute()\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 670, in _execute\n return self.func(*self.args, **self.kwargs)\n File \"/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/tasks/migrate.py\", line 102, in migrate_from_pulp2\n \"Validation failed: resources missing {}\".format(missing_resources)\n"
},
"finished_at": "2020-06-09T10:47:56.137359Z",
"name": "pulp_2to3_migration.app.tasks.migrate.migrate_from_pulp2",
"parent_task": null,
"progress_reports": [],
"pulp_created": "2020-06-09T10:47:55.933361Z",
"pulp_href": "/pulp/api/v3/tasks/7bf152e9-971e-48c4-82a7-9bd8a0ce80fe/",
"reserved_resources_record": [
"pulp_2to3_migration"
],
"started_at": "2020-06-09T10:47:56.090277Z",
"state": "failed",
"task_group": null,
"worker": "/pulp/api/v3/workers/1b7a3c5f-86f6-4da7-922f-2a61cf6eac56/"
}
</code></pre>