Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-10-04T10:30:48ZPulp
Planio Ansible Plugin - Issue #9483 (CLOSED - CURRENTRELEASE): dispatch calls need to specify locked res...https://pulp.plan.io/issues/94832021-10-04T10:30:48ZmdellwegAnsible Plugin - Issue #9476 (CLOSED - CURRENTRELEASE): Syncing twice w/ mirror=True and optimize...https://pulp.plan.io/issues/94762021-09-30T19:25:31Zgerrod
<p>The optimize=True setting is causing the sync pipeline to think all the content has been removed from the remote and thus produce a new repo version with no content in it.</p> Ansible Plugin - Issue #9442 (CLOSED - CURRENTRELEASE): Deprecation status isn't synced correctly...https://pulp.plan.io/issues/94422021-09-23T21:02:06Zfao89
<p>If a repo with deprecated collections sync from a remote without deprecations, the repo will keep its deprecations regardless the new remote</p> Ansible Plugin - Issue #9405 (CLOSED - CURRENTRELEASE): Download links for V3 unpaginated collect...https://pulp.plan.io/issues/94052021-09-15T22:47:36Zgerrod
<p>The download links for collection versions from the v3 unpaginated endpoint don't match the download links for the same collection versions at their href endpoints. I've tested this on AH staging environment, not sure if present elsewhere.</p> Ansible Plugin - Issue #9404 (CLOSED - CURRENTRELEASE): V3 unpaginated sync fails when trying to ...https://pulp.plan.io/issues/94042021-09-15T22:44:48Zgerrod
<p>This line is the offender if the collection version does not exist: <a href="https://github.com/pulp/pulp_ansible/blob/3c2a4fe6e51eacc478534f795cb959a74a639383/pulp_ansible/app/tasks/collections.py#L546-L550" class="external">https://github.com/pulp/pulp_ansible/blob/3c2a4fe6e51eacc478534f795cb959a74a639383/pulp_ansible/app/tasks/collections.py#L546-L550</a>, a key error will happen.</p> Ansible Plugin - Issue #9390 (CLOSED - CURRENTRELEASE): Proxy authentication is not working in co...https://pulp.plan.io/issues/93902021-09-13T18:42:17Znewswangerd
<p>We're seeing issues with customers getting <code>aiohttp.client_exceptions.ClientHttpProxyError: 407, message='Proxy Authentication Required'</code> error on Automation Hub 4.2 using pulpcore 3.7.5 and pulp_ansible 0.5.8 during collection sync.</p>
<p>This appears to be happening with Squid.</p>
<p>Squid logs:</p>
<pre><code>1627559170.932 0 xxx.xxx.xxx.xxx TCP_DENIED/407 4125 CONNECT galaxy.ansible.com:443 - HIER_NONE/- text/html
</code></pre> Ansible Plugin - Issue #9368 (CLOSED - CURRENTRELEASE): pulp-ansible is failing to synchttps://pulp.plan.io/issues/93682021-09-09T13:38:46Zfao89
<p>Currently, syncing is not async safe:
<a href="https://github.com/pulp/pulp_ansible/actions/runs/1215767114" class="external">https://github.com/pulp/pulp_ansible/actions/runs/1215767114</a></p> Ansible Plugin - Issue #9260 (CLOSED - CURRENTRELEASE): Syncing is running out of DB connectionshttps://pulp.plan.io/issues/92602021-08-19T21:46:18Zfao89
<pre><code>pulp [bd3e6e1fe5bb4586b75163ddd7ad1521]: pulpcore.tasking.pulpcore_worker:INFO: File "/usr/local/lib/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py", line 274, in _perform_task
result = func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 130, in sync
repo_version = d_version.create()
File "/usr/local/lib/python3.8/site-packages/pulpcore/plugin/stages/declarative_version.py", line 151, in create
loop.run_until_complete(pipeline)
File "/usr/lib64/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/local/lib/python3.8/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
await asyncio.gather(*futures)
File "/usr/local/lib/python3.8/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
await self.run()
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 778, in run
await asyncio.gather(*tasks)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 592, in _fetch_collection_metadata
await self._fetch_paginated_collection_metadata(
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 548, in _fetch_paginated_collection_metadata
await asyncio.gather(*tasks)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 441, in _fetch_collection_version_metadata
await self._add_collection_version(api_version, collection_version_url, metadata)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 469, in _add_collection_version
await asyncio.gather(*tasks)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 592, in _fetch_collection_metadata
await self._fetch_paginated_collection_metadata(
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 548, in _fetch_paginated_collection_metadata
await asyncio.gather(*tasks)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 441, in _fetch_collection_version_metadata
await self._add_collection_version(api_version, collection_version_url, metadata)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 496, in _add_collection_version
self.parsing_metadata_progress_bar.increment()
File "/usr/local/lib/python3.8/site-packages/pulpcore/app/models/progress.py", line 185, in increment
self.increase_by(1)
File "/usr/local/lib/python3.8/site-packages/pulpcore/app/models/progress.py", line 198, in increase_by
self.save()
File "/usr/local/lib/python3.8/site-packages/pulpcore/app/models/progress.py", line 142, in save
super().save(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django_lifecycle/mixins.py", line 134, in save
save(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 726, in save
self.save_base(using=using, force_insert=force_insert,
File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 763, in save_base
updated = self._save_table(
File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 845, in _save_table
updated = self._do_update(base_qs, using, pk_val, values, update_fields,
File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 899, in _do_update
return filtered._update(values) > 0
File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 802, in _update
return query.get_compiler(self.db).execute_sql(CURSOR)
File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1559, in execute_sql
cursor = super().execute_sql(result_type)
File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1173, in execute_sql
cursor = self.connection.cursor()
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 259, in cursor
return self._cursor()
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 235, in _cursor
self.ensure_connection()
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
self.connect()
File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
self.connect()
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 200, in connect
self.connection = self.get_new_connection(conn_params)
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 187, in get_new_connection
connection = Database.connect(**conn_params)
File "/usr/local/lib64/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
</code></pre>
<pre><code>Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
self.connect()
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 200, in connect
self.connection = self.get_new_connection(conn_params)
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 187, in get_new_connection
connection = Database.connect(**conn_params)
File "/usr/local/lib64/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL: remaining connection slots are reserved for non-replication superuser connections
</code></pre> Ansible Plugin - Issue #9250 (CLOSED - CURRENTRELEASE): improper validation when passing json as ...https://pulp.plan.io/issues/92502021-08-18T15:41:42Zjsherril@redhat.comjsherril@redhat.com
<p>Json seems to be a subset of yaml in some cases and with some libraries. Pulp's parsing seems to allow it as something like this:</p>
<p>{"collections": "theforeman.foreman"}</p>
<p>works fine as a requirements.yaml on a remote. However when you sync, you get a traceback:</p>
<pre><code> {"traceback"=>
" File \"/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpcore_worker.py\", line 272, in _perform_task\n" +
" result = func(*args, **kwargs)\n" +
" File \"/usr/local/lib/python3.6/site-packages/pulp_ansible/app/tasks/collections.py\", line 130, in sync\n" +
" repo_version = d_version.create()\n" +
" File \"/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 151, in create\n" +
" loop.run_until_complete(pipeline)\n" +
" File \"/usr/lib64/python3.6/asyncio/base_events.py\", line 484, in run_until_complete\n" +
" return future.result()\n" +
" File \"/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 225, in create_pipeline\n" +
" await asyncio.gather(*futures)\n" +
" File \"/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 43, in __call__\n" +
" await self.run()\n" +
" File \"/usr/local/lib/python3.6/site-packages/pulp_ansible/app/tasks/collections.py\", line 778, in run\n" +
" await asyncio.gather(*tasks)\n" +
" File \"/usr/local/lib/python3.6/site-packages/pulp_ansible/app/tasks/collections.py\", line 587, in _fetch_collection_metadata\n" +
" namespace, name = requirements_entry.name.split(\".\")\n",
"description"=>"not enough values to unpack (expected 2, got 1)"},
</code></pre>
<p>So it seems like that format should either error on save (like other invalid yaml does), or work for syncing</p> Ansible Plugin - Issue #9220 (CLOSED - CURRENTRELEASE): content_ansible_collection_versions_creat...https://pulp.plan.io/issues/92202021-08-09T17:07:09Zsajha
<p>The <a href="https://docs.pulpproject.org/pulp_ansible/restapi.html#operation/upload_collection" class="external">https://docs.pulpproject.org/pulp_ansible/restapi.html#operation/upload_collection</a> endpoint allows you to upload a Collection tarball. This operation results in an artifact which can then be used to create a new collection content.</p>
<p>However, the endpoint to do so <a href="https://docs.pulpproject.org/pulp_ansible/restapi.html#operation/content_ansible_collection_versions_create" class="external">https://docs.pulpproject.org/pulp_ansible/restapi.html#operation/content_ansible_collection_versions_create</a> seems to have a lot of required fields which are presumably avaialble on the uploaded tarball itself. Is there a way to modify the flow so that associating an artifact from an uploaded collection tarball doesn't need to have all the other information passed.</p>
<p>I am assuming the flow would be to upload tarball, create a collection content using the artifact, add content created to a collection repository.</p> Ansible Plugin - Issue #9137 (CLOSED - CURRENTRELEASE): As a user I want sync to be more efficienthttps://pulp.plan.io/issues/91372021-07-23T15:51:29Zfao89
<p>Currently, the progress report is doing lots of transactions, slowing down the sync process</p> Ansible Plugin - Issue #9075 (CLOSED - CURRENTRELEASE): Downloads do not seem to use proxy authen...https://pulp.plan.io/issues/90752021-07-13T13:59:34Zfao89
<p>I tried with both file and rpm repos, if i specify a proxy url, username and password, i can check pulps' db and see:</p>
<pre><code>pulpcore=# select url, proxy_url, proxy_username, proxy_password from core_remote;
url | proxy_url | proxy_username | proxy_password
---------------------------------------------------------------+-----------------------------+----------------+----------------
https://jlsherrill.fedorapeople.org/fake-repos/needed-errata/ | http://192.168.122.182:8888 | admin | redhat
</code></pre>
<p>so i am fairly confident that the username and password are being set properly. However, when i sync, i get:</p>
<p>407, message='Proxy Authentication Required', url=URL('<a href="http://192.168.122.182:8888" class="external">http://192.168.122.182:8888</a>')</p>
<p>looking at my squid logs, i see:</p>
<p>1625605398.930 0 192.168.122.143 TCP_DENIED/407 4235 CONNECT jlsherrill.fedorapeople.org:443 - HIER_NONE/- text/html</p>
<p>So I believe pulp isn't properly sending the proxy auth correctly?</p>
<p>Note that my curl command:</p>
<pre><code> curl -vv -X GET https://jlsherrill.fedorapeople.org/fake-repos/needed-errata/repodata/repomd.xml --proxy http://192.168.122.182:8888 -H 'Proxy-Authorization: Basic YWRtaW46cmVkaGF0'
</code></pre>
<p>works without issue</p> Ansible Plugin - Issue #9006 (CLOSED - CURRENTRELEASE): As a user I want to be able to use the ne...https://pulp.plan.io/issues/90062021-07-02T18:08:19Zfao89Ansible Plugin - Issue #8971 (CLOSED - CURRENTRELEASE): ImportError: cannot import name 'PulpAnsi...https://pulp.plan.io/issues/89712021-06-25T13:46:35Zdaviddavis
<p>The tests on pulp_ansible are failing with this error. See last night's test run:</p>
<p><a href="https://github.com/pulp/pulp_ansible/actions/runs/970115331" class="external">https://github.com/pulp/pulp_ansible/actions/runs/970115331</a></p>
<p>I believe this is caused by <a href="https://github.com/pulp/pulpcore/commit/21494e17763736aa949970d9051190f5e5821948" class="external">a recent change in pulpcore to how bindings are generated</a>. I looked through the bindings and I believe the new bindings class name for the galaxy v3 collections endpoint is <code>PulpAnsibleApiV3CollectionsApi</code>. We could update to use this class name although it doesn't seem quite right (ie notice the lack of "galaxy" in the class name). That said, the old class name <code>PulpAnsibleGalaxyApiCollectionsApi</code> doesn't seem quite right either (notice there's no V3 in the name).</p> Ansible Plugin - Issue #8957 (CLOSED - CURRENTRELEASE): URL validation error on remote displays r...https://pulp.plan.io/issues/89572021-06-23T18:27:37Zsajha
<p>The url validation is somehow leading to this: ["Invalid URL https://galaxy.ansible.com/. Ensure the URL ends '/'."]} This happens when I have some source specified in the requirements.yml without the trailing "/".. However the error displays the remote URL in the error which happens to have the trailing /..</p>
<p>(02:14:32 PM) rochacbruno: error is on <a href="https://github.com/pulp/pulp_ansible/blob/master/pulp_ansible/app/serializers.py#L166" class="external">https://github.com/pulp/pulp_ansible/blob/master/pulp_ansible/app/serializers.py#L166</a>
(02:14:44 PM) rochacbruno: should be url=url instead of data["url"]</p>