Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-10-20T17:27:32ZPulp
Planio Ansible Plugin - Issue #9525 (CLOSED - WORKSFORME): Performing sync in Hub ignores sync toggle an...https://pulp.plan.io/issues/95252021-10-20T17:27:32Zawcrosby
<p>This is seen in galaxy_ng 4.3.3 performing a sync against c.rh.c:</p>
<ul>
<li>galaxy_ng==4.3.3 includes: pulpcore==3.11.2, pulp-ansible==0.7.3</li>
<li>c.rh.com includes: pulpcore==3.11.2 pulp-ansible==0.7.3</li>
<li>FWIW on console.rh.com, editing the sync toggles changes the <code>content_count</code> (527) at <a href="https://console.redhat.com/api/automation-hub/_ui/v1/my-distributions/" class="external">https://console.redhat.com/api/automation-hub/_ui/v1/my-distributions/</a> for <code>5910538-synclist</code> but it does not change the <code>count</code> (104) at <a href="https://console.redhat.com/api/automation-hub/content/5910538-synclist/v3/collections/" class="external">https://console.redhat.com/api/automation-hub/content/5910538-synclist/v3/collections/</a>
</li>
</ul>
<p>This is also seen in galaxy_ng <code>master</code> performing a sync against c.rh.c and c.stage.rh.c</p>
<ul>
<li>galaxy_ng==<code>master</code> includes: pulpcore==3.15.2, pulp-ansible==0.10.1</li>
<li>c.stage.rh.com includes: pulpcore==3.14.6, pulp-ansible==0.9.2</li>
<li>FWIW the count c.stage.rh.com <###>-synclist/v3/collections/ does change after sync toggles</li>
</ul> Ansible Plugin - Test #7955 (CLOSED - WONTFIX): Load Test - Collections V3 endpointhttps://pulp.plan.io/issues/79552020-12-08T21:41:37Zfao89
<p>We need a cron job for load testing collections v3 endpoint for:</p>
<ul>
<li>12 collections</li>
<li>24 collections</li>
<li>60 collections</li>
<li>120 collections</li>
</ul>
<p>Locust reference: <a href="https://coderbook.com/@marcus/load-test-your-django-website-using-locustio/" class="external">https://coderbook.com/@marcus/load-test-your-django-website-using-locustio/</a>
github action: <a href="https://github.com/marketplace/actions/locust-load-test" class="external">https://github.com/marketplace/actions/locust-load-test</a>
CI: <a href="https://github.com/apardo04/adrianpardo.dev-react/blob/676a44b715d865496082c12752f5c18f343575d5/.github/workflows/deployment.yml#L88" class="external">https://github.com/apardo04/adrianpardo.dev-react/blob/676a44b715d865496082c12752f5c18f343575d5/.github/workflows/deployment.yml#L88</a></p> Ansible Plugin - Issue #7651 (CLOSED - WORKSFORME): Pulp Error: Network is unreachable during Rep...https://pulp.plan.io/issues/76512020-10-05T16:18:14Zjdjeffers
<p>Attempting a sync via the pulp-ansible plugin, the following error is thrown:</p>
<pre><code>
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 936, in perform_job
rv = job.perform()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 684, in perform
self._result = self._execute()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 690, in _execute
return self.func(*self.args, **self.kwargs)
File "/usr/local/lib/pulp/src/pulp-ansible/pulp_ansible/app/tasks/collections.py", line 81, in sync
d_version.create()
File "/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/declarative_version.py", line 148, in create
loop.run_until_complete(pipeline)
File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
return future.result()
File "/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
await asyncio.gather(*futures)
File "/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/api.py", line 43, in __call__
await self.run()
File "/usr/local/lib/pulp/src/pulp-ansible/pulp_ansible/app/tasks/collections.py", line 288, in run
async for metadata in self._fetch_collections():
File "/usr/local/lib/pulp/src/pulp-ansible/pulp_ansible/app/tasks/collections.py", line 362, in _fetch_collections
initial_data = parse_metadata(await downloader.run())
File "/usr/local/lib/pulp/src/pulpcore/pulpcore/download/base.py", line 227, in run
return await self._run(extra_data=extra_data)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/backoff/_async.py", line 133, in retry
ret = await target(*args, **kwargs)
File "/usr/local/lib/pulp/src/pulp-ansible/pulp_ansible/app/downloaders.py", line 88, in _run
return await super()._run(extra_data=extra_data)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/backoff/_async.py", line 133, in retry
ret = await target(*args, **kwargs)
File "/usr/local/lib/pulp/src/pulpcore/pulpcore/download/http.py", line 209, in _run
async with self.session.get(self.url, proxy=self.proxy, auth=self.auth) as response:
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/client.py", line 1012, in __aenter__
self._resp = await self._coro
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/client.py", line 483, in _request
timeout=real_timeout
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/connector.py", line 523, in connect
proto = await self._create_connection(req, traces, timeout)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/connector.py", line 859, in _create_connection
req, traces, timeout)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/connector.py", line 1004, in _create_direct_connection
raise last_exc
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/connector.py", line 986, in _create_direct_connection
req=req, client_error=client_error)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/connector.py", line 943, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
description: Cannot connect to host galaxy-dev.ansible.com:443 ssl:default [Network
is unreachable]
</code></pre>
<p>The remote url provided was <a href="https://galaxy-dev.ansible.com/api/v2/collections/testing/ansible_testing_content" class="external">https://galaxy-dev.ansible.com/api/v2/collections/testing/ansible_testing_content</a>
These are the versions present:</p>
<pre><code>curl https://`hostname`/pulp/api/v3/status/ --cert /etc/pki/katello/certs/pulp-client.crt --key /etc/pki/katello/private/pulp-client.key | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1397 100 1397 0 0 7492 0 --:--:-- --:--:-- --:--:-- 7551
{
"versions": [
{
"component": "pulpcore",
"version": "3.7.1"
},
{
"component": "pulp_2to3_migration",
"version": "0.4.0.dev0"
},
{
"component": "pulp_file",
"version": "1.4.0.dev0"
},
{
"component": "pulp_container",
"version": "2.2.0.dev0"
},
{
"component": "pulp_ansible",
"version": "0.4.1"
},
{
"component": "pulp_certguard",
"version": "1.1.0.dev0"
},
{
"component": "pulp_rpm",
"version": "3.7.0"
}
],
"online_workers": [
{
"pulp_created": "2020-10-02T17:45:43.215436Z",
"pulp_href": "/pulp/api/v3/workers/974769cc-b099-469e-8694-4c73e56fa7f7/",
"name": "676@centos7-proxy-devel.jjeffers.example.com",
"last_heartbeat": "2020-10-05T16:13:36.733971Z"
},
{
"pulp_created": "2020-09-30T17:11:40.333349Z",
"pulp_href": "/pulp/api/v3/workers/4ad803de-500a-4061-9092-5dcd9f212b5e/",
"name": "resource-manager",
"last_heartbeat": "2020-10-05T16:13:38.437627Z"
},
{
"pulp_created": "2020-10-02T17:45:41.854532Z",
"pulp_href": "/pulp/api/v3/workers/6a7edf97-736d-49b6-9c28-dcbd12f51380/",
"name": "695@centos7-proxy-devel.jjeffers.example.com",
"last_heartbeat": "2020-10-05T16:13:38.636631Z"
}
],
"online_content_apps": [
{
"name": "1195@centos7-proxy-devel.jjeffers.example.com",
"last_heartbeat": "2020-10-05T16:13:37.690219Z"
},
{
"name": "1210@centos7-proxy-devel.jjeffers.example.com",
"last_heartbeat": "2020-10-05T16:13:37.691302Z"
}
],
"database_connection": {
"connected": true
},
"redis_connection": {
"connected": true
},
"storage": {
"total": 42927656960,
"used": 6078078976,
"free": 36849577984
}
}
</code></pre>
<p>Relevant extract from katello server log showing API call with RespositorySyncURL parameters:</p>
<pre><code>10:38:20 rails.1 | 2020-10-05T10:38:20 [D|kat|] Calling API: RepositoriesAnsibleApi.sync ...
10:38:20 rails.1 | 2020-10-05T10:38:20 [D|kat|] HTTP request body param ~BEGIN~
10:38:20 rails.1 | | {"remote":"/pulp/api/v3/remotes/ansible/collection/62b69866-9652-4243-a001-cb96c9cf3a10/","mirror":true}
</code></pre>
<p>Using cURL from the same instance that hosts the pulpserver:</p>
<pre><code>[vagrant@centos7-proxy-devel ~]$ curl -i -X GET https://galaxy-dev.ansible.com/api/v2/collections/testing/ansible_testing_content
HTTP/1.1 301 Moved Permanently
Date: Mon, 05 Oct 2020 16:17:20 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=d1e1ffeb69079587cc03db60aa3a4eb6a1601914640; expires=Wed, 04-Nov-20 16:17:20 GMT; path=/; domain=.ansible.com; HttpOnly; SameSite=Lax
Location: /api/v2/collections/testing/ansible_testing_content/
X-REQUEST-ID: e727e193a65f41ddb23696203369edbf
Set-Cookie: af91cb160f5f76cd3bc226a05339849a=1aabc03ebf238aabd8c18f839b624077; path=/; HttpOnly; Secure
Cache-control: private
CF-Cache-Status: DYNAMIC
cf-request-id: 059b26b096000007fa3a278200000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 5dd873c75c3107fa-ATL
</code></pre> Ansible Plugin - Task #6909 (CLOSED - WONTFIX): import_collection task to send filepath to galaxy...https://pulp.plan.io/issues/69092020-06-04T18:56:39Zawcrosby
<p>Currently the import_collection task sends the artifact fileobject to galaxy-importer. In galaxy-importer we want move from using the tarfile module to tar in shell.</p>
<p>We would like import_collection task to send the artifact filepath to galaxy-importer.</p>
<p>Also, currently in the fileobject we use the .storage.url property to download the artifact to a container to run "ansible-test sanity". So perhaps we can send the filepath + url instead of the opened fileobject.</p> Ansible Plugin - Issue #5905 (CLOSED - WORKSFORME): ansible-galaxy collection publish failshttps://pulp.plan.io/issues/59052019-12-19T19:06:37Zbmbouterbmbouter@redhat.com
<p>In a correctly configured pulp_ansible environment I try to upload a collection via ansible-galaxy (also correctly configured) and it fails.</p>
<p>1. Make a repository, e.g. http POST :80/pulp/api/v3/repositories/ansible/ansible/ name=foo<br>
2. Make a distribution for that repository, e.g. http POST :80/pulp/api/v3/distributions/ansible/ansible/ name='baz' base_path='foo' repository=/pulp/api/v3/repositories/ansible/ansible/53025e9e-9f30-4086-8214-8811028a396f/<br>
3. Try to upload:</p>
<pre><code>(pulp) [vagrant@pulp3-source-fedora31 ~]$ ansible-galaxy collection publish bmbouter-testing-1.0.0.tar.gz -vvvv
ansible-galaxy 2.9.2
config file = /home/vagrant/.ansible.cfg
configured module search path = ['/home/vagrant/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/pulp/lib64/python3.7/site-packages/ansible
executable location = /usr/local/lib/pulp/bin/ansible-galaxy
python version = 3.7.4 (default, Aug 12 2019, 14:45:07) [GCC 9.1.1 20190605 (Red Hat 9.1.1-2)]
Using /home/vagrant/.ansible.cfg as config file
Initial connection to galaxy_server: http://localhost:24817/pulp_ansible/galaxy/foo
Opened /home/vagrant/.ansible/galaxy_token
Calling Galaxy at http://localhost:24817/pulp_ansible/galaxy/foo
Opened /home/vagrant/.ansible/galaxy_token
Calling Galaxy at http://localhost:24817/pulp_ansible/galaxy/foo/api
Found API version 'v1, v2' with Galaxy server default (http://localhost:24817/pulp_ansible/galaxy/foo/api)
Publishing collection artifact '/home/vagrant/bmbouter-testing-1.0.0.tar.gz' to default http://localhost:24817/pulp_ansible/galaxy/foo/api
Opened /home/vagrant/.ansible/galaxy_token
Calling Galaxy at http://localhost:24817/pulp_ansible/galaxy/foo/api/v2/collections/
ERROR! Error when publishing collection to default (http://localhost:24817/pulp_ansible/galaxy/foo/api) (HTTP Code: 400, Message: Bad Request Code: Unknown)
</code></pre> Ansible Plugin - Test #5415 (CLOSED - WONTFIX): As a user, I can publish a Collection to pulp_ans...https://pulp.plan.io/issues/54152019-09-06T19:13:04Zdaviddavis
<p>There's documentation about how to publish (or upload) a collection to pulp_ansible using the ansible-galaxy cli (formerly mazer):</p>
<p><a href="https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html#collection-workflows" class="external">https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html#collection-workflows</a></p>
<p>Write a test to cover this workflow.</p> Ansible Plugin - Test #5269 (CLOSED - WONTFIX): Test - As a user, I have 'latest' as a boolean qu...https://pulp.plan.io/issues/52692019-08-13T18:42:11Zkersom
<p>In addition to the fields being searchable, a 'latest' parameter should be available that defaults to False. If True, it will only return the latest version when multiple versions of one Collection name are returned in the results.</p> Ansible Plugin - Test #5268 (CLOSED - WONTFIX): Test - As a user, I can upload a collection to th...https://pulp.plan.io/issues/52682019-08-13T18:38:10Zkersom
<a name="Problem"></a>
<h2 >Problem<a href="#Problem" class="wiki-anchor">¶</a></h2>
<p>Currently the one-shot uploader only is available via the galaxy bindings.</p>
<a name="Solution"></a>
<h3 >Solution<a href="#Solution" class="wiki-anchor">¶</a></h3>
<p>Add a one-shot uploader view that will use the same collection import task code. This needs to be available in the swagger API so it can be in the docs and bindings too.</p> Ansible Plugin - Test #5264 (CLOSED - WONTFIX): Test - Pulp3 Ansible Collection remote doesn't se...https://pulp.plan.io/issues/52642019-08-13T18:12:22Zkersom
<p>*Steps to Reproduce:*++<br>
Steps based on httpie and jq from documentation: <a href="https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html" class="external">https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html</a></p>
<p>#Create 2 separate repos:<br>
http POST $BASE_ADDR/pulp/api/v3/repositories/ name=test1<br>
http POST $BASE_ADDR/pulp/api/v3/repositories/ name=test2</p>
<p>export REPO_HREF1=$(http $BASE_ADDR/pulp/api/v3/repositories/ | \<br>
jq -r '.results[] | select(.name == "test1") | ._href')</p>
<p>export REPO_HREF2=$(http $BASE_ADDR/pulp/api/v3/repositories/ | \<br>
jq -r '.results[] | select(.name == "test2") | ._href')</p>
<p>#Create collection remote:</p>
<p>http POST $BASE_ADDR/pulp/api/v3/remotes/ansible/collection/ \<br>
name='bar' \<br>
url='<a href="https://galaxy-dev.ansible.com" class="external">https://galaxy-dev.ansible.com</a>' \<br>
whitelist='newswangerd.collection_demo'</p>
<p>export REMOTE_HREF=$(http $BASE_ADDR/pulp/api/v3/remotes/ansible/collection/ | jq -r '.results[] | select(.name == "bar") | ._href')</p>
<p>#Sync both repos with the remote</p>
<p>export TASK_URL1=$(http POST $BASE_ADDR$REMOTE_HREF'sync/' repository=$REPO_HREF1 \<br>
| jq -r '.task')</p>
<p>export TASK_URL2=$(http POST $BASE_ADDR$REMOTE_HREF'sync/' repository=$REPO_HREF2 \<br>
| jq -r '.task')</p>
<p>#Notice that a new repo version is created by 1st task but none by the 2nd.</p>
<p>http $BASE_ADDR$TASK_URL1</p>
<p>http $BASE_ADDR$TASK_URL2</p>
<p>*<strong>*</strong>*********************************************************</p>
<p>I tried with a second remote pointing to the same URL and with the same whitelist and it doesn't seem to create a repo version either.</p> Ansible Plugin - Test #5257 (CLOSED - WONTFIX): Test - As a user, I can have fulltext search on C...https://pulp.plan.io/issues/52572019-08-13T14:46:17Zkersom
<p>A fulltext search should be added using examples here: <a href="https://github.com/abarto/full_text_search_drf" class="external">https://github.com/abarto/full_text_search_drf</a></p> Ansible Plugin - Story #5038 (CLOSED - WONTFIX): As a user, I can submit one-time Remote options ...https://pulp.plan.io/issues/50382019-06-26T12:46:42Zbmbouterbmbouter@redhat.comAnsible Plugin - Test #4976 (CLOSED - WONTFIX): Collection - Test if creating duplicate content r...https://pulp.plan.io/issues/49762019-06-14T18:47:29Zkersom
<ul>
<li>Evaluate if we already have similar tests in place.</li>
<li>Create duplicate content</li>
<li>Assert that an HTTP Bad Request - 400 was raised</li>
<li>Assert that an error (e.g. "There is already a file content unit with relativepath 'foo.tar.gz' and artifact '/pulp/api/v3/artifacts/1/'.") is returned</li>
</ul> Ansible Plugin - Test #4914 (CLOSED - WONTFIX): Test upload contenthttps://pulp.plan.io/issues/49142019-06-04T19:52:30Zkersom
<p>1 - Upload content<br>
2 - Create a role from the artifact<br>
3 - Add content to a repository</p>
<p>See: <a href="https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html#collection-workflows" class="external">https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html#collection-workflows</a></p> Ansible Plugin - Test #4912 (CLOSED - WONTFIX): [Epic] Test collection content typehttps://pulp.plan.io/issues/49122019-06-04T19:43:58Zkersom
<p>Add tests to collection content type.</p>
<p><a href="https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html#collection-workflows" class="external">https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html#collection-workflows</a></p> Ansible Plugin - Test #4779 (CLOSED - WONTFIX): Test mazer 'install' won't get Content not in the...https://pulp.plan.io/issues/47792019-05-03T13:31:18Zbmbouterbmbouter@redhat.com
<p>This is the negative test related to <a href="https://pulp.plan.io/issues/4778" class="external">https://pulp.plan.io/issues/4778</a></p>
<p>Add the collection to Pulp but do <strong>not</strong> have the collection served by a Distribution.<br>
Try to have mazer install it and verify that it won't install even though the content unit is inside of Pulp (but not that Distribution).</p>
<p>This validates that content is only handed out through the correct Distributions.</p>
<p>I'm not sure if this is a very valuable test though since core is really providing this feature.</p>