Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-12-08T19:00:00ZPulp
Planio RPM Support - Test #9622 (MODIFIED): Add a repo signed using 'sha' as alias for 'sha1'https://pulp.plan.io/issues/96222021-12-08T19:00:00Zggainey
<p>'sha' support exists in the wild, is the same as 'sha1', and has broken us several times now, Let's make it possible to write tests for it.</p> RPM Support - Test #7350 (MODIFIED): Test syncing from a repository located on a local diskhttps://pulp.plan.io/issues/73502020-08-19T15:40:07Zdalleydalley@redhat.com
<p>e.g. a repository with a file:// url such as file:///var/lib/pulp/sync_imports/test_repos/zoo/</p> Debian Support - Test #7344 (CLOSED - COMPLETE): Add test to publish empty repositoryhttps://pulp.plan.io/issues/73442020-08-19T08:10:24Zmsinghal
<p>Empty repository consists of release files, package index files but no packages.</p> Debian Support - Test #6051 (CLOSED - COMPLETE): Add tests for known cases where the distribution...https://pulp.plan.io/issues/60512020-01-24T10:38:34Zquba42
<p>In pulp_deb for pulp2 we had significant problems getting synchronization of remote repositories where the distribution is equal to neither the codename nor the suite to work.</p>
<p>The most well known examples of this are the Debian security repositories:<br>
<a href="http://security-cdn.debian.org/debian-security/" class="external">http://security-cdn.debian.org/debian-security/</a></p>
<p>As well as all official Ubuntu repositories with a "-" in their distribution path, see:<br>
<a href="http://archive.ubuntu.com/ubuntu/dists/" class="external">http://archive.ubuntu.com/ubuntu/dists/</a></p>
<p>This issue can be considered "done" when these two major cases are covered by the pulp_deb test suite.</p> Container Support - Test #5554 (CLOSED - COMPLETE): A test scenario for syncing repositories with...https://pulp.plan.io/issues/55542019-10-08T13:35:49Zlmjachky
<p>The PR <a href="https://github.com/pulp/pulp_docker/pull/429/files" class="external">https://github.com/pulp/pulp_docker/pull/429/files</a> introduces new changes to the model and the representation of the field whitelist_tags.</p>
<p>The following test scenario to test the changes is proposed:<br>
1. Create a new repository:</p>
<pre><code class="text syntaxhl" data-language="text">http POST http://localhost:24817/pulp/api/v3/repositories/ name=foo
</code></pre>
<p>2. Create a new docker remote with a sample set of whitelisted tags:</p>
<pre><code class="text syntaxhl" data-language="text">http POST http://localhost:24817/pulp/api/v3/remotes/docker/docker/ name='library/busybox' upstream_name='busybox' url='https://registry-1.docker.io' policy='on_demand' whitelist_tags:="[\"latest\", \"1.31\"]"
</code></pre>
<p>3. Execute a sync and check if only whitelisted tags were correctly synced. In addition, check if whitelisted tags were correctly serialized as a list.</p>
<pre><code class="text syntaxhl" data-language="text">http :24817/pulp/api/v3/content/docker/tags/?repository_version_added=/pulp/api/v3/repositories/f2cdfe03-ad46-4760-875b-9b5228edc684/versions/1/
</code></pre>
<pre><code class="json syntaxhl" data-language="json"><span class="p">{</span><span class="w">
</span><span class="nl">"count"</span><span class="p">:</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w">
</span><span class="nl">"next"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"previous"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"results"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"_created"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2019-10-08T13:20:55.325935Z"</span><span class="p">,</span><span class="w">
</span><span class="nl">"_href"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/content/docker/tags/6fc08249-5994-47ee-9cbb-392332e22a14/"</span><span class="p">,</span><span class="w">
</span><span class="nl">"_type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"docker.tag"</span><span class="p">,</span><span class="w">
</span><span class="nl">"artifact"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/artifacts/b581ddbe-e8d0-4d05-bb2b-9afb39572451/"</span><span class="p">,</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1.31"</span><span class="p">,</span><span class="w">
</span><span class="nl">"tagged_manifest"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/content/docker/manifests/325b8e2b-b665-4185-8bbb-74ed7691378a/"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"_created"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2019-10-08T13:20:55.317976Z"</span><span class="p">,</span><span class="w">
</span><span class="nl">"_href"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/content/docker/tags/cebc99db-bd92-4327-a79c-dbbde38098f9/"</span><span class="p">,</span><span class="w">
</span><span class="nl">"_type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"docker.tag"</span><span class="p">,</span><span class="w">
</span><span class="nl">"artifact"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/artifacts/b581ddbe-e8d0-4d05-bb2b-9afb39572451/"</span><span class="p">,</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"latest"</span><span class="p">,</span><span class="w">
</span><span class="nl">"tagged_manifest"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/content/docker/manifests/325b8e2b-b665-4185-8bbb-74ed7691378a/"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre>
<pre><code class="text syntaxhl" data-language="text">http :24817/pulp/api/v3/remotes/docker/docker/9db3bb5d-3288-474a-b8b8-180784d6a8fa/
</code></pre>
<pre><code class="json syntaxhl" data-language="json"><span class="p">{</span><span class="w">
</span><span class="nl">"_created"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2019-10-08T13:20:50.593489Z"</span><span class="p">,</span><span class="w">
</span><span class="nl">"_href"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/pulp/api/v3/remotes/docker/docker/9db3bb5d-3288-474a-b8b8-180784d6a8fa/"</span><span class="p">,</span><span class="w">
</span><span class="nl">"_last_updated"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2019-10-08T13:20:50.593502Z"</span><span class="p">,</span><span class="w">
</span><span class="nl">"_type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"docker.docker"</span><span class="p">,</span><span class="w">
</span><span class="nl">"download_concurrency"</span><span class="p">:</span><span class="w"> </span><span class="mi">20</span><span class="p">,</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"library/busybox"</span><span class="p">,</span><span class="w">
</span><span class="nl">"policy"</span><span class="p">:</span><span class="w"> </span><span class="s2">"on_demand"</span><span class="p">,</span><span class="w">
</span><span class="nl">"proxy_url"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"ssl_ca_certificate"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"ssl_client_certificate"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"ssl_client_key"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"ssl_validation"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w">
</span><span class="nl">"upstream_name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"busybox"</span><span class="p">,</span><span class="w">
</span><span class="nl">"url"</span><span class="p">:</span><span class="w"> </span><span class="s2">"https://registry-1.docker.io"</span><span class="p">,</span><span class="w">
</span><span class="nl">"whitelist_tags"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="s2">"latest"</span><span class="p">,</span><span class="w">
</span><span class="s2">"1.31"</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre> Ansible Plugin - Test #5333 (CLOSED - COMPLETE): As a user, I can sync collections from another P...https://pulp.plan.io/issues/53332019-08-22T17:28:11Zdaviddavis
<p>In lieu of setting up two Pulp servers, I would just use two repositories.</p>
<p>Test steps:</p>
<p>1. Create a collections repo, remote, sync and publish<br>
2. Create a second repository and a remote which points to the distribution in step 1<br>
3. Sync the second repo and confirm it has the same contents as the first.</p> Ansible Plugin - Test #5262 (CLOSED - COMPLETE): Test - Uploading same collection twice causes 50...https://pulp.plan.io/issues/52622019-08-13T17:53:41Zkersom
<p>To reproduce:</p>
<p>1. build a collection to upload. I use the default one from mazer "here":<br>
2. Upload it once. I use: <code>http --form POST :24817/ansible/collections/ file@greetings_namespace-hello-11.11.11.tar.gz</code>. Observe that works.<br>
3. Upload it a second time and observe that you get a 500 error</p> Ansible Plugin - Test #5256 (CLOSED - COMPLETE): Test - Ansible collection remote doesn't handle ...https://pulp.plan.io/issues/52562019-08-13T14:42:32Zkersom
<p>Ex:<br>
With a url <a href="https://galaxy.ansible.com" class="external">https://galaxy.ansible.com</a> it works fine.<br>
With a url <a href="https://galaxy.ansible.com/" class="external">https://galaxy.ansible.com/</a> , throws the following error: <a href="https://gist.github.com/sjha4/0afdd783bac7cea56055b706f6b02155" class="external">https://gist.github.com/sjha4/0afdd783bac7cea56055b706f6b02155</a></p> Docker Support - Test #5181 (CLOSED - COMPLETE): Removing docker manifests from a docker reposito...https://pulp.plan.io/issues/51812019-07-24T12:58:57Zbherring
<a name="Original-Dev-Problem"></a>
<h3 >Original Dev Problem<a href="#Original-Dev-Problem" class="wiki-anchor">¶</a></h3>
<p>Removing all docker manifests from a large docker repo seems to take a long time:</p>
<p>~300 manifests takes ~2 minutes<br>
~2000 manifests takes ~30-40 minutes</p>
<a name="Reproducer"></a>
<h3 >Reproducer:<a href="#Reproducer" class="wiki-anchor">¶</a></h3>
<p>1. Create and sync a docker repo such as: <a href="https://quay.io" class="external">https://quay.io</a> datawire/ambassador<br>
2. Remove all docker manifests from the repo: pulp-admin docker repo remove manifest --repo-id=1-docker-dev-7915f7d0-7a98-4131-9c41-1be7b578d442 --not id=foo</p>
<a name="Discussion-with-asmacado"></a>
<h3 >Discussion with asmacado<a href="#Discussion-with-asmacado" class="wiki-anchor">¶</a></h3>
<ul>
<li>A lot of the delay were caused by the sequential removals</li>
<li>If there was an outage while the task was on-going, the removal was left in a non-deterministic state</li>
<li>The timing model for removal for general use cases grew incredibly fast</li>
</ul>
<a name="Solutions-with-the-new-refactor"></a>
<h3 >Solutions with the new refactor:<a href="#Solutions-with-the-new-refactor" class="wiki-anchor">¶</a></h3>
<ul>
<li>All BLOBS and MANIFESTS are all removed at once</li>
<li>Higher level units are all removed second</li>
<li>Timing shrank from 300 ~=1hr to 300 ~=a few seconds</li>
</ul>
<a name="Concerns"></a>
<h3 >Concerns:<a href="#Concerns" class="wiki-anchor">¶</a></h3>
<ul>
<li>This refactor would affect removals and syncs</li>
<li>The removal, if interrupted, would leave Orphans which could affect upstream verification</li>
</ul>
<a name="Actions"></a>
<h3 >Actions:<a href="#Actions" class="wiki-anchor">¶</a></h3>
<a name="Upstream"></a>
<h4 >Upstream<a href="#Upstream" class="wiki-anchor">¶</a></h4>
<ul>
<li>Need an upstream verification that ensures there is no change in functionality with remove units and sync units</li>
</ul>
<a name="Downstream"></a>
<h4 >Downstream<a href="#Downstream" class="wiki-anchor">¶</a></h4>
<ul>
<li>Need to verify downstream verification are not affected by this change/patch</li>
<li>BZ verification will just be the removal of a large number of repos takes a short amount of time</li>
</ul>
<a name="Test-Permutations"></a>
<h3 >Test Permutations<a href="#Test-Permutations" class="wiki-anchor">¶</a></h3>
<a name="Remove"></a>
<h4 >Remove<a href="#Remove" class="wiki-anchor">¶</a></h4>
<p>Variables:</p>
<ul>
<li>(Permutations) Shared or Not Shared [manifest_lists, manifests]</li>
<li>type_ids": ['docker_tag', 'docker_manifest_list', 'docker_manifest', 'docker_blob']</li>
</ul>
<p>Verification Scenarios:</p>
<ul>
<li>For Not Shared, remove $REMOVAL_TYPES and verify all units have been removed</li>
<li>For Shared, remove $REMOVAL_TYPES and verify only non-shared units have been removed. Shared units should remain.</li>
</ul>
<p>Note:</p>
<ul>
<li>Removal Criteria short-cut for relevant type_id being removed.</li>
</ul>
<pre><code> criteria = {
"type_ids": ['docker_tag', 'docker_manifest_list', 'docker_manifest', 'docker_blob'],
"filters": {"unit": {"_id": {"$in": all_units}}},
}
</code></pre>
<ul>
<li>There are already tests covering tag removal</li>
</ul> Ansible Plugin - Test #5000 (CLOSED - COMPLETE): Collection - Test that sucessive syncs do update...https://pulp.plan.io/issues/50002019-06-20T17:52:26Zkersom
<p>1 - Create multiple syncs using the same remote.<br>
2 - Assure that for every sync a new repo version was created.</p> Ansible Plugin - Test #4915 (CLOSED - COMPLETE): Test install a collection hosted by Pulp using m...https://pulp.plan.io/issues/49152019-06-04T19:54:58Zkersom
<p>Consume content hosted by Pulp using mazer.</p>
<p>Evaluate best way to add mazer as part of the test process.</p>
<p>mazer: <a href="https://github.com/ansible/mazer" class="external">https://github.com/ansible/mazer</a></p>
<p>See: <a href="https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html#collection-workflows" class="external">https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html#collection-workflows</a></p> Ansible Plugin - Test #4913 (CLOSED - COMPLETE): Test sync collections from Galaxyhttps://pulp.plan.io/issues/49132019-06-04T19:49:43Zkersom
<p>1 - Create a repository<br>
2 - Create a collection remote<br>
3 - Sync repository with collection remote</p> Python Support - Test #4682 (MODIFIED): Test to ensure pip installing a package workshttps://pulp.plan.io/issues/46822019-04-12T20:29:37ZCodeHeeler
<p>To prevent a repeat of issue 4677 [0] where pip installation fails, we should add a test that attempts to install a package with pip.</p>
<p>Our current test [1] only checks that direct download works, and thus all our tests were passing when no one could pip install packages we served.</p>
<p>I'm guessing the added code would be placed around L99 [2] to download using pip and then add an additional assert statement.</p>
<p>[0] <a href="https://pulp.plan.io/issues/4677" class="external">https://pulp.plan.io/issues/4677</a><br>
[1] <a href="https://github.com/pulp/pulp_python/blob/master/pulp_python/tests/functional/api/test_download_content.py#L29" class="external">https://github.com/pulp/pulp_python/blob/master/pulp_python/tests/functional/api/test_download_content.py#L29</a><br>
[2] <a href="https://github.com/pulp/pulp_python/blob/master/pulp_python/tests/functional/api/test_download_content.py#L99" class="external">https://github.com/pulp/pulp_python/blob/master/pulp_python/tests/functional/api/test_download_content.py#L99</a></p> Container Support - Test #4460 (CLOSED - COMPLETE): Test pull from container registryhttps://pulp.plan.io/issues/44602019-02-25T14:21:27Zbherring
<a name="Test-Notes"></a>
<h2 >Test Notes<a href="#Test-Notes" class="wiki-anchor">¶</a></h2>
<ul>
<li>testing `docker pull` against published docker repo... ;</li>
<li>API tests against a published distribution</li>
</ul> CertGuard - Test #4122 (CLOSED - COMPLETE): Test CRUD ContentGuardhttps://pulp.plan.io/issues/41222018-11-05T18:55:05Zkersom
<p>Add basic CRUD test for ContentGuard.</p>