Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-10-20T17:27:32ZPulp
Planio Ansible Plugin - Issue #9525 (CLOSED - WORKSFORME): Performing sync in Hub ignores sync toggle an...https://pulp.plan.io/issues/95252021-10-20T17:27:32Zawcrosby
<p>This is seen in galaxy_ng 4.3.3 performing a sync against c.rh.c:</p>
<ul>
<li>galaxy_ng==4.3.3 includes: pulpcore==3.11.2, pulp-ansible==0.7.3</li>
<li>c.rh.com includes: pulpcore==3.11.2 pulp-ansible==0.7.3</li>
<li>FWIW on console.rh.com, editing the sync toggles changes the <code>content_count</code> (527) at <a href="https://console.redhat.com/api/automation-hub/_ui/v1/my-distributions/" class="external">https://console.redhat.com/api/automation-hub/_ui/v1/my-distributions/</a> for <code>5910538-synclist</code> but it does not change the <code>count</code> (104) at <a href="https://console.redhat.com/api/automation-hub/content/5910538-synclist/v3/collections/" class="external">https://console.redhat.com/api/automation-hub/content/5910538-synclist/v3/collections/</a>
</li>
</ul>
<p>This is also seen in galaxy_ng <code>master</code> performing a sync against c.rh.c and c.stage.rh.c</p>
<ul>
<li>galaxy_ng==<code>master</code> includes: pulpcore==3.15.2, pulp-ansible==0.10.1</li>
<li>c.stage.rh.com includes: pulpcore==3.14.6, pulp-ansible==0.9.2</li>
<li>FWIW the count c.stage.rh.com <###>-synclist/v3/collections/ does change after sync toggles</li>
</ul> Ansible Plugin - Issue #7651 (CLOSED - WORKSFORME): Pulp Error: Network is unreachable during Rep...https://pulp.plan.io/issues/76512020-10-05T16:18:14Zjdjeffers
<p>Attempting a sync via the pulp-ansible plugin, the following error is thrown:</p>
<pre><code>
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 936, in perform_job
rv = job.perform()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 684, in perform
self._result = self._execute()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 690, in _execute
return self.func(*self.args, **self.kwargs)
File "/usr/local/lib/pulp/src/pulp-ansible/pulp_ansible/app/tasks/collections.py", line 81, in sync
d_version.create()
File "/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/declarative_version.py", line 148, in create
loop.run_until_complete(pipeline)
File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
return future.result()
File "/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
await asyncio.gather(*futures)
File "/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/api.py", line 43, in __call__
await self.run()
File "/usr/local/lib/pulp/src/pulp-ansible/pulp_ansible/app/tasks/collections.py", line 288, in run
async for metadata in self._fetch_collections():
File "/usr/local/lib/pulp/src/pulp-ansible/pulp_ansible/app/tasks/collections.py", line 362, in _fetch_collections
initial_data = parse_metadata(await downloader.run())
File "/usr/local/lib/pulp/src/pulpcore/pulpcore/download/base.py", line 227, in run
return await self._run(extra_data=extra_data)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/backoff/_async.py", line 133, in retry
ret = await target(*args, **kwargs)
File "/usr/local/lib/pulp/src/pulp-ansible/pulp_ansible/app/downloaders.py", line 88, in _run
return await super()._run(extra_data=extra_data)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/backoff/_async.py", line 133, in retry
ret = await target(*args, **kwargs)
File "/usr/local/lib/pulp/src/pulpcore/pulpcore/download/http.py", line 209, in _run
async with self.session.get(self.url, proxy=self.proxy, auth=self.auth) as response:
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/client.py", line 1012, in __aenter__
self._resp = await self._coro
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/client.py", line 483, in _request
timeout=real_timeout
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/connector.py", line 523, in connect
proto = await self._create_connection(req, traces, timeout)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/connector.py", line 859, in _create_connection
req, traces, timeout)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/connector.py", line 1004, in _create_direct_connection
raise last_exc
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/connector.py", line 986, in _create_direct_connection
req=req, client_error=client_error)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/connector.py", line 943, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
description: Cannot connect to host galaxy-dev.ansible.com:443 ssl:default [Network
is unreachable]
</code></pre>
<p>The remote url provided was <a href="https://galaxy-dev.ansible.com/api/v2/collections/testing/ansible_testing_content" class="external">https://galaxy-dev.ansible.com/api/v2/collections/testing/ansible_testing_content</a>
These are the versions present:</p>
<pre><code>curl https://`hostname`/pulp/api/v3/status/ --cert /etc/pki/katello/certs/pulp-client.crt --key /etc/pki/katello/private/pulp-client.key | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1397 100 1397 0 0 7492 0 --:--:-- --:--:-- --:--:-- 7551
{
"versions": [
{
"component": "pulpcore",
"version": "3.7.1"
},
{
"component": "pulp_2to3_migration",
"version": "0.4.0.dev0"
},
{
"component": "pulp_file",
"version": "1.4.0.dev0"
},
{
"component": "pulp_container",
"version": "2.2.0.dev0"
},
{
"component": "pulp_ansible",
"version": "0.4.1"
},
{
"component": "pulp_certguard",
"version": "1.1.0.dev0"
},
{
"component": "pulp_rpm",
"version": "3.7.0"
}
],
"online_workers": [
{
"pulp_created": "2020-10-02T17:45:43.215436Z",
"pulp_href": "/pulp/api/v3/workers/974769cc-b099-469e-8694-4c73e56fa7f7/",
"name": "676@centos7-proxy-devel.jjeffers.example.com",
"last_heartbeat": "2020-10-05T16:13:36.733971Z"
},
{
"pulp_created": "2020-09-30T17:11:40.333349Z",
"pulp_href": "/pulp/api/v3/workers/4ad803de-500a-4061-9092-5dcd9f212b5e/",
"name": "resource-manager",
"last_heartbeat": "2020-10-05T16:13:38.437627Z"
},
{
"pulp_created": "2020-10-02T17:45:41.854532Z",
"pulp_href": "/pulp/api/v3/workers/6a7edf97-736d-49b6-9c28-dcbd12f51380/",
"name": "695@centos7-proxy-devel.jjeffers.example.com",
"last_heartbeat": "2020-10-05T16:13:38.636631Z"
}
],
"online_content_apps": [
{
"name": "1195@centos7-proxy-devel.jjeffers.example.com",
"last_heartbeat": "2020-10-05T16:13:37.690219Z"
},
{
"name": "1210@centos7-proxy-devel.jjeffers.example.com",
"last_heartbeat": "2020-10-05T16:13:37.691302Z"
}
],
"database_connection": {
"connected": true
},
"redis_connection": {
"connected": true
},
"storage": {
"total": 42927656960,
"used": 6078078976,
"free": 36849577984
}
}
</code></pre>
<p>Relevant extract from katello server log showing API call with RespositorySyncURL parameters:</p>
<pre><code>10:38:20 rails.1 | 2020-10-05T10:38:20 [D|kat|] Calling API: RepositoriesAnsibleApi.sync ...
10:38:20 rails.1 | 2020-10-05T10:38:20 [D|kat|] HTTP request body param ~BEGIN~
10:38:20 rails.1 | | {"remote":"/pulp/api/v3/remotes/ansible/collection/62b69866-9652-4243-a001-cb96c9cf3a10/","mirror":true}
</code></pre>
<p>Using cURL from the same instance that hosts the pulpserver:</p>
<pre><code>[vagrant@centos7-proxy-devel ~]$ curl -i -X GET https://galaxy-dev.ansible.com/api/v2/collections/testing/ansible_testing_content
HTTP/1.1 301 Moved Permanently
Date: Mon, 05 Oct 2020 16:17:20 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=d1e1ffeb69079587cc03db60aa3a4eb6a1601914640; expires=Wed, 04-Nov-20 16:17:20 GMT; path=/; domain=.ansible.com; HttpOnly; SameSite=Lax
Location: /api/v2/collections/testing/ansible_testing_content/
X-REQUEST-ID: e727e193a65f41ddb23696203369edbf
Set-Cookie: af91cb160f5f76cd3bc226a05339849a=1aabc03ebf238aabd8c18f839b624077; path=/; HttpOnly; Secure
Cache-control: private
CF-Cache-Status: DYNAMIC
cf-request-id: 059b26b096000007fa3a278200000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 5dd873c75c3107fa-ATL
</code></pre> Ansible Plugin - Task #6909 (CLOSED - WONTFIX): import_collection task to send filepath to galaxy...https://pulp.plan.io/issues/69092020-06-04T18:56:39Zawcrosby
<p>Currently the import_collection task sends the artifact fileobject to galaxy-importer. In galaxy-importer we want move from using the tarfile module to tar in shell.</p>
<p>We would like import_collection task to send the artifact filepath to galaxy-importer.</p>
<p>Also, currently in the fileobject we use the .storage.url property to download the artifact to a container to run "ansible-test sanity". So perhaps we can send the filepath + url instead of the opened fileobject.</p> Ansible Plugin - Issue #5905 (CLOSED - WORKSFORME): ansible-galaxy collection publish failshttps://pulp.plan.io/issues/59052019-12-19T19:06:37Zbmbouterbmbouter@redhat.com
<p>In a correctly configured pulp_ansible environment I try to upload a collection via ansible-galaxy (also correctly configured) and it fails.</p>
<p>1. Make a repository, e.g. http POST :80/pulp/api/v3/repositories/ansible/ansible/ name=foo<br>
2. Make a distribution for that repository, e.g. http POST :80/pulp/api/v3/distributions/ansible/ansible/ name='baz' base_path='foo' repository=/pulp/api/v3/repositories/ansible/ansible/53025e9e-9f30-4086-8214-8811028a396f/<br>
3. Try to upload:</p>
<pre><code>(pulp) [vagrant@pulp3-source-fedora31 ~]$ ansible-galaxy collection publish bmbouter-testing-1.0.0.tar.gz -vvvv
ansible-galaxy 2.9.2
config file = /home/vagrant/.ansible.cfg
configured module search path = ['/home/vagrant/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/pulp/lib64/python3.7/site-packages/ansible
executable location = /usr/local/lib/pulp/bin/ansible-galaxy
python version = 3.7.4 (default, Aug 12 2019, 14:45:07) [GCC 9.1.1 20190605 (Red Hat 9.1.1-2)]
Using /home/vagrant/.ansible.cfg as config file
Initial connection to galaxy_server: http://localhost:24817/pulp_ansible/galaxy/foo
Opened /home/vagrant/.ansible/galaxy_token
Calling Galaxy at http://localhost:24817/pulp_ansible/galaxy/foo
Opened /home/vagrant/.ansible/galaxy_token
Calling Galaxy at http://localhost:24817/pulp_ansible/galaxy/foo/api
Found API version 'v1, v2' with Galaxy server default (http://localhost:24817/pulp_ansible/galaxy/foo/api)
Publishing collection artifact '/home/vagrant/bmbouter-testing-1.0.0.tar.gz' to default http://localhost:24817/pulp_ansible/galaxy/foo/api
Opened /home/vagrant/.ansible/galaxy_token
Calling Galaxy at http://localhost:24817/pulp_ansible/galaxy/foo/api/v2/collections/
ERROR! Error when publishing collection to default (http://localhost:24817/pulp_ansible/galaxy/foo/api) (HTTP Code: 400, Message: Bad Request Code: Unknown)
</code></pre> Ansible Plugin - Test #5415 (CLOSED - WONTFIX): As a user, I can publish a Collection to pulp_ans...https://pulp.plan.io/issues/54152019-09-06T19:13:04Zdaviddavis
<p>There's documentation about how to publish (or upload) a collection to pulp_ansible using the ansible-galaxy cli (formerly mazer):</p>
<p><a href="https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html#collection-workflows" class="external">https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html#collection-workflows</a></p>
<p>Write a test to cover this workflow.</p> Ansible Plugin - Issue #5155 (CLOSED - WORKSFORME): Ansible Collection Is Installed With Wrong Ve...https://pulp.plan.io/issues/51552019-07-18T17:36:24Zompragash
<p><strong>OS / ENVIRONMENT</strong></p>
<p>- Fedora release 30 (Thirty)<br>
- Ansible == 2.8.1<br>
- pulpcore == 3.0.0rc3<br>
- pulpcore-plugin == 0.1.0rc3<br>
- pulp_ansible == 0.2.0b1</p>
<p><strong>STEPS TO REPRODUCE</strong></p>
<p>- Deploy pulp_ansible</p>
<p>- Create a Repo, Sync/Upload Ansible collection</p>
<p>- Publish the uploaded Collection</p>
<p>- In my case I have uploaded three different collection each with different version as you can see below:</p>
<p>testing.ansible_testing_content,4.0.6<br>
ompragash.collections,1.0.12<br>
greetings_namespace.hello,11.11.11</p>
<p>- Also uploaded collection_list.txt which has the API output of `/pulp/api/v3/content/ansible/collections/` endpoint.</p>
<p>- Install mazer and configure ~/.ansible/mazer.yml file</p>
<p>server:<br>
ignore_certs: false<br>
url: <a href="http://localhost:24817/pulp_ansible/galaxy/galaxy_base" class="external">http://localhost:24817/pulp_ansible/galaxy/galaxy_base</a></p>
<p>- Now run `mazer install NAMESPACE_name.COLLECTION_name` to install uploaded collections from pulp_ansible</p>
<p><strong>EXPECTED RESULTS</strong></p>
<p>- Collection should be installed and if no version is specified `mazer` should install the latest available version and if version is specified it should install the required version.</p>
<p><strong>ACTUAL RESULTS</strong></p>
<p>- Collection is installed with the wrong version i.e. with the version of another collection</p>
<p>- As you can see I have three collections and when I try to install `ompragash.collections` or `greetings_namespace.hello` it is always installed with version 4.0.6.</p>
<p><strong>Observations:</strong></p>
<p>- Trying to install an collection with the specific version but got the below error:</p>
<p>~~~<br>
[<a href="mailto:root@X.X.X.X" class="email">root@X.X.X.X</a> ~]# mazer install ompragash.collections,1.0.12</p>
<p>Collection specs to install:<br>
ompragash.collections,==1.0.12</p>
<p>Installing spec: ompragash.collections,==1.0.12<br>
Unable to find a collection that matches the spec: ompragash.collections,==1.0.12:<br>
- you can use --ignore-errors to skip failed collections and finish processing the list.<br>
~~~</p>
<p>- When I only call the collection name like below:</p>
<p>~~~<br>
[<a href="mailto:root@X.X.X.X" class="email">root@X.X.X.X</a> ~]# mazer install ompragash.collections</p>
<p>Collection specs to install:<br>
ompragash.collections,*</p>
<p>Installing spec: ompragash.collections,*<br>
Found: ompragash.collections,4.0.6 (for spec ompragash.collections,*)<br>
Installed: ompragash.collections,4.0.6 (to /root/.ansible/collections/ansible_collections/ompragash/collections)<br>
~~~</p>
<p>here^ if you see `ompragash.collections` is installed with version 4.0.6 which is the version of another collection (i.e. testing.ansible_testing_content)</p> Ansible Plugin - Story #5038 (CLOSED - WONTFIX): As a user, I can submit one-time Remote options ...https://pulp.plan.io/issues/50382019-06-26T12:46:42Zbmbouterbmbouter@redhat.comAnsible Plugin - Test #4779 (CLOSED - WONTFIX): Test mazer 'install' won't get Content not in the...https://pulp.plan.io/issues/47792019-05-03T13:31:18Zbmbouterbmbouter@redhat.com
<p>This is the negative test related to <a href="https://pulp.plan.io/issues/4778" class="external">https://pulp.plan.io/issues/4778</a></p>
<p>Add the collection to Pulp but do <strong>not</strong> have the collection served by a Distribution.<br>
Try to have mazer install it and verify that it won't install even though the content unit is inside of Pulp (but not that Distribution).</p>
<p>This validates that content is only handed out through the correct Distributions.</p>
<p>I'm not sure if this is a very valuable test though since core is really providing this feature.</p> Ansible Plugin - Test #4777 (CLOSED - WONTFIX): Test mazer 'publish' to Pulphttps://pulp.plan.io/issues/47772019-05-03T13:27:48Zbmbouterbmbouter@redhat.com
<p>This test follows these docs: <a href="https://github.com/pulp/pulp_ansible#mazer-publish" class="external">https://github.com/pulp/pulp_ansible#mazer-publish</a></p>
<p>You should be testing against the latest checkout of <a href="https://github.com/ansible/mazer/" class="external">https://github.com/ansible/mazer/</a> and specifically the 'devel' branch.</p>
<p>1. Build the 'hello' collection like the docs say (with the mazer source, from the mazer source)<br>
2. configure mazer to point to a Distribution on Pulp. (note for this test the Distribution doesn't yet have to exist)<br>
3. mazer publish<br>
4. Verify the content is listed at <a href="http://localhost:24817/pulp/api/v3/content/ansible/collections/" class="external">http://localhost:24817/pulp/api/v3/content/ansible/collections/</a></p> Ansible Plugin - Test #4776 (CLOSED - WONTFIX): Test installing a role directly via urlhttps://pulp.plan.io/issues/47762019-05-03T13:22:40Zbmbouterbmbouter@redhat.com
<p>1. Have a role be served up by a Distribution. This is probably easiest done by uploading one, adding it to a repo version, and configuring the Distribution to serve that repo version.</p>
<p>2. Test that you can have <code>ansible-galaxy</code> client install that role directly via a pre-known URL for that unit in that distribution's url. See the example in the docs here: <a href="https://github.com/pulp/pulp_ansible#using-a-direct-path" class="external">https://github.com/pulp/pulp_ansible#using-a-direct-path</a></p> Ansible Plugin - Test #4775 (CLOSED - WONTFIX): Test Installing using the Galaxy APIhttps://pulp.plan.io/issues/47752019-05-03T13:22:35Zbmbouterbmbouter@redhat.com
<p>1. Have a role be served up by a Distribution. This is probably easiest done by uploading one, adding it to a repo version, and configuring the Distribution to serve that repo version.</p>
<p>2. Test that you can have <code>ansible-galaxy</code> client install by name (not directly by URL). You do this by configuring ansible-galaxy to point at the distribution and then installing by name. See <a href="https://github.com/pulp/pulp_ansible#using-the-pulp-galaxy-api" class="external">these docs</a> for example.</p> Ansible Plugin - Test #4774 (CLOSED - WONTFIX): Ensure upload of a Role workshttps://pulp.plan.io/issues/47742019-05-03T13:12:14Zbmbouterbmbouter@redhat.com
<p>Use the pg.tar.gz as the fixture. <a href="https://github.com/pulp/pulp_ansible#upload-a-role-to-pulp" class="external">https://github.com/pulp/pulp_ansible#upload-a-role-to-pulp</a></p>
<p>Ensure you can:<br>
1. upload the Artifact <a href="https://github.com/pulp/pulp_ansible#upload-a-role-to-pulp" class="external">https://github.com/pulp/pulp_ansible#upload-a-role-to-pulp</a><br>
2. create the Role content unit <a href="https://github.com/pulp/pulp_ansible#create-a-role-content-unit" class="external">https://github.com/pulp/pulp_ansible#create-a-role-content-unit</a></p> Ansible Plugin - Task #4772 (CLOSED - WONTFIX): Add table of contents to docshttps://pulp.plan.io/issues/47722019-05-03T12:48:09Zbmbouterbmbouter@redhat.com
<p>The docs would benefit from a table of contents to help users find what they need more easily. The page is sufficiently long now that it's a problem.</p> Ansible Plugin - Test #3967 (CLOSED - WONTFIX): Create a test fixture for Ansiblehttps://pulp.plan.io/issues/39672018-09-05T12:51:42Zdalleydalley@redhat.com
<p>Smash tests currently use Ansible Galaxy, which could change at any time. If one of the projects being tested (ansible and pulp) uploads a new role version then the sync tests will fail. We should try to set up a test fixture with static data in order to avoid test breakage.</p> Ansible Plugin - Story #3947 (CLOSED - WONTFIX): Support syncing of repositories and other conten...https://pulp.plan.io/issues/39472018-08-29T15:08:47Zdaviddavis
<p>We currently sync roles from the <code>/api/v1/roles/</code> endpoint. However, we should instead start syncing repositories and their contents from <code>/api/v1/repositories/</code> and <code>/api/v1/content/</code> endpoints as the <code>/api/v1/roles/</code> endpoint is considered deprecated.</p>