Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-11-09T13:48:32ZPulp
Planio Ansible Plugin - Issue #9561 (CLOSED - DUPLICATE): Update galaxy-importer requirementshttps://pulp.plan.io/issues/95612021-11-09T13:48:32Znewswangerd
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_ansible/749":<a href="https://github.com/pulp/pulp_ansible/issues/749" class="external">https://github.com/pulp/pulp_ansible/issues/749</a></p>
<hr>
<p>Pulp ansible currently depends on galaxy-importer>=0.3.1, but relies on a function that's only available in 0.4.1: <a href="https://github.com/pulp/pulp_ansible/commit/db592b44ed367c3c1866456d012a24886d9c55a5#diff-ee4bd390338ab9adff17b9b09f05e8dc56e09a3964633c104402998971723bffR6" class="external">https://github.com/pulp/pulp_ansible/commit/db592b44ed367c3c1866456d012a24886d9c55a5#diff-ee4bd390338ab9adff17b9b09f05e8dc56e09a3964633c104402998971723bffR6</a></p> Ansible Plugin - Issue #9546 (CLOSED - DUPLICATE): When syncing a collection version, all its sig...https://pulp.plan.io/issues/95462021-10-27T18:51:23Zrochacbruno
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_ansible/748":<a href="https://github.com/pulp/pulp_ansible/issues/748" class="external">https://github.com/pulp/pulp_ansible/issues/748</a></p>
<hr>
<p>Related to <a href="https://pulp.plan.io/issues/9543" class="external">https://pulp.plan.io/issues/9543</a></p>
<p>Story:</p>
<ul>
<li>All collections are signed on the host system</li>
<li>When syncing to another instance, together with all the artifacts/versions, signatures will also need to be synced</li>
<li>If for example a signature is corrupted on the source system, user may redo the signature, invalidate the current, do signature rotation and then the clients will need to re-sync to fetch new valid signatures</li>
</ul> Ansible Plugin - Issue #9534 (CLOSED - DUPLICATE): Ansible Galaxy Repo failed to synchronize case...https://pulp.plan.io/issues/95342021-10-25T15:15:01Zstosem
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_ansible/747":<a href="https://github.com/pulp/pulp_ansible/issues/747" class="external">https://github.com/pulp/pulp_ansible/issues/747</a></p>
<hr>
<p>The galaxy.ansible.com API uses case-insensitive fields for the owner username and role name. The Ansible Galaxy Hub source code: <a href="https://github.com/ansible/galaxy/blob/e0e4b909171ddc6ca40f0ef2f7d4dce5680777ce/galaxy/api/views/roles.py#L50" class="external">https://github.com/ansible/galaxy/blob/e0e4b909171ddc6ca40f0ef2f7d4dce5680777ce/galaxy/api/views/roles.py#L50</a></p>
<p>But pulp usesnamespace=namespace. It causes the error on case-sensitive repositories (404 not found, or empty set). An example:</p>
<ul>
<li><a href="https://galaxy.ansible.com/api/v1/roles/?owner__username=Oefenweb" class="external">https://galaxy.ansible.com/api/v1/roles/?owner__username=Oefenweb</a></li>
<li><a href="https://galaxy.ansible.com/api/v1/roles/?owner__username=oefenweb" class="external">https://galaxy.ansible.com/api/v1/roles/?owner__username=oefenweb</a></li>
</ul>
<p>Galaxy Ansible returns the same result. But pulp isn't.</p>
<p>So we can't install roles like this using regular <code>ansible-galaxy install oefenweb.postfix</code> command with the Pulp repo.</p> Ansible Plugin - Issue #9525 (CLOSED - WORKSFORME): Performing sync in Hub ignores sync toggle an...https://pulp.plan.io/issues/95252021-10-20T17:27:32Zawcrosby
<p>This is seen in galaxy_ng 4.3.3 performing a sync against c.rh.c:</p>
<ul>
<li>galaxy_ng==4.3.3 includes: pulpcore==3.11.2, pulp-ansible==0.7.3</li>
<li>c.rh.com includes: pulpcore==3.11.2 pulp-ansible==0.7.3</li>
<li>FWIW on console.rh.com, editing the sync toggles changes the <code>content_count</code> (527) at <a href="https://console.redhat.com/api/automation-hub/_ui/v1/my-distributions/" class="external">https://console.redhat.com/api/automation-hub/_ui/v1/my-distributions/</a> for <code>5910538-synclist</code> but it does not change the <code>count</code> (104) at <a href="https://console.redhat.com/api/automation-hub/content/5910538-synclist/v3/collections/" class="external">https://console.redhat.com/api/automation-hub/content/5910538-synclist/v3/collections/</a>
</li>
</ul>
<p>This is also seen in galaxy_ng <code>master</code> performing a sync against c.rh.c and c.stage.rh.c</p>
<ul>
<li>galaxy_ng==<code>master</code> includes: pulpcore==3.15.2, pulp-ansible==0.10.1</li>
<li>c.stage.rh.com includes: pulpcore==3.14.6, pulp-ansible==0.9.2</li>
<li>FWIW the count c.stage.rh.com <###>-synclist/v3/collections/ does change after sync toggles</li>
</ul> Ansible Plugin - Issue #9524 (CLOSED - DUPLICATE): docs blob missing from CollectionVersion when ...https://pulp.plan.io/issues/95242021-10-19T23:08:51Zdkliban@redhat.com
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_ansible/746":<a href="https://github.com/pulp/pulp_ansible/issues/746" class="external">https://github.com/pulp/pulp_ansible/issues/746</a></p> Ansible Plugin - Issue #9483 (CLOSED - CURRENTRELEASE): dispatch calls need to specify locked res...https://pulp.plan.io/issues/94832021-10-04T10:30:48ZmdellwegAnsible Plugin - Issue #9476 (CLOSED - CURRENTRELEASE): Syncing twice w/ mirror=True and optimize...https://pulp.plan.io/issues/94762021-09-30T19:25:31Zgerrod
<p>The optimize=True setting is causing the sync pipeline to think all the content has been removed from the remote and thus produce a new repo version with no content in it.</p> Ansible Plugin - Issue #9473 (CLOSED - DUPLICATE): CI is failing intermittently on v3 test_serial...https://pulp.plan.io/issues/94732021-09-29T23:13:56Zgerrod
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_ansible/745":<a href="https://github.com/pulp/pulp_ansible/issues/745" class="external">https://github.com/pulp/pulp_ansible/issues/745</a></p>
<hr>
<p>The test <code>test_v3_updated_at</code> fails randomly when it fails to assert the latest_version collection stays the same after the modify action. The serializer seems to choose a different latest version sometimes even when the highest version collections don't change, but only after the modify action creates a new repository_version. (i.e. refreshing the <code>/collections/</code> endpoint doesn't change the latest_version).</p> Ansible Plugin - Issue #9442 (CLOSED - CURRENTRELEASE): Deprecation status isn't synced correctly...https://pulp.plan.io/issues/94422021-09-23T21:02:06Zfao89
<p>If a repo with deprecated collections sync from a remote without deprecations, the repo will keep its deprecations regardless the new remote</p> Ansible Plugin - Issue #9405 (CLOSED - CURRENTRELEASE): Download links for V3 unpaginated collect...https://pulp.plan.io/issues/94052021-09-15T22:47:36Zgerrod
<p>The download links for collection versions from the v3 unpaginated endpoint don't match the download links for the same collection versions at their href endpoints. I've tested this on AH staging environment, not sure if present elsewhere.</p> Ansible Plugin - Issue #9404 (CLOSED - CURRENTRELEASE): V3 unpaginated sync fails when trying to ...https://pulp.plan.io/issues/94042021-09-15T22:44:48Zgerrod
<p>This line is the offender if the collection version does not exist: <a href="https://github.com/pulp/pulp_ansible/blob/3c2a4fe6e51eacc478534f795cb959a74a639383/pulp_ansible/app/tasks/collections.py#L546-L550" class="external">https://github.com/pulp/pulp_ansible/blob/3c2a4fe6e51eacc478534f795cb959a74a639383/pulp_ansible/app/tasks/collections.py#L546-L550</a>, a key error will happen.</p> Ansible Plugin - Issue #9390 (CLOSED - CURRENTRELEASE): Proxy authentication is not working in co...https://pulp.plan.io/issues/93902021-09-13T18:42:17Znewswangerd
<p>We're seeing issues with customers getting <code>aiohttp.client_exceptions.ClientHttpProxyError: 407, message='Proxy Authentication Required'</code> error on Automation Hub 4.2 using pulpcore 3.7.5 and pulp_ansible 0.5.8 during collection sync.</p>
<p>This appears to be happening with Squid.</p>
<p>Squid logs:</p>
<pre><code>1627559170.932 0 xxx.xxx.xxx.xxx TCP_DENIED/407 4125 CONNECT galaxy.ansible.com:443 - HIER_NONE/- text/html
</code></pre> Ansible Plugin - Issue #9368 (CLOSED - CURRENTRELEASE): pulp-ansible is failing to synchttps://pulp.plan.io/issues/93682021-09-09T13:38:46Zfao89
<p>Currently, syncing is not async safe:
<a href="https://github.com/pulp/pulp_ansible/actions/runs/1215767114" class="external">https://github.com/pulp/pulp_ansible/actions/runs/1215767114</a></p> Ansible Plugin - Issue #9260 (CLOSED - CURRENTRELEASE): Syncing is running out of DB connectionshttps://pulp.plan.io/issues/92602021-08-19T21:46:18Zfao89
<pre><code>pulp [bd3e6e1fe5bb4586b75163ddd7ad1521]: pulpcore.tasking.pulpcore_worker:INFO: File "/usr/local/lib/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py", line 274, in _perform_task
result = func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 130, in sync
repo_version = d_version.create()
File "/usr/local/lib/python3.8/site-packages/pulpcore/plugin/stages/declarative_version.py", line 151, in create
loop.run_until_complete(pipeline)
File "/usr/lib64/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/local/lib/python3.8/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
await asyncio.gather(*futures)
File "/usr/local/lib/python3.8/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
await self.run()
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 778, in run
await asyncio.gather(*tasks)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 592, in _fetch_collection_metadata
await self._fetch_paginated_collection_metadata(
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 548, in _fetch_paginated_collection_metadata
await asyncio.gather(*tasks)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 441, in _fetch_collection_version_metadata
await self._add_collection_version(api_version, collection_version_url, metadata)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 469, in _add_collection_version
await asyncio.gather(*tasks)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 592, in _fetch_collection_metadata
await self._fetch_paginated_collection_metadata(
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 548, in _fetch_paginated_collection_metadata
await asyncio.gather(*tasks)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 441, in _fetch_collection_version_metadata
await self._add_collection_version(api_version, collection_version_url, metadata)
File "/usr/local/lib/python3.8/site-packages/pulp_ansible/app/tasks/collections.py", line 496, in _add_collection_version
self.parsing_metadata_progress_bar.increment()
File "/usr/local/lib/python3.8/site-packages/pulpcore/app/models/progress.py", line 185, in increment
self.increase_by(1)
File "/usr/local/lib/python3.8/site-packages/pulpcore/app/models/progress.py", line 198, in increase_by
self.save()
File "/usr/local/lib/python3.8/site-packages/pulpcore/app/models/progress.py", line 142, in save
super().save(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django_lifecycle/mixins.py", line 134, in save
save(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 726, in save
self.save_base(using=using, force_insert=force_insert,
File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 763, in save_base
updated = self._save_table(
File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 845, in _save_table
updated = self._do_update(base_qs, using, pk_val, values, update_fields,
File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 899, in _do_update
return filtered._update(values) > 0
File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 802, in _update
return query.get_compiler(self.db).execute_sql(CURSOR)
File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1559, in execute_sql
cursor = super().execute_sql(result_type)
File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1173, in execute_sql
cursor = self.connection.cursor()
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 259, in cursor
return self._cursor()
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 235, in _cursor
self.ensure_connection()
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
self.connect()
File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
self.connect()
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 200, in connect
self.connection = self.get_new_connection(conn_params)
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 187, in get_new_connection
connection = Database.connect(**conn_params)
File "/usr/local/lib64/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
</code></pre>
<pre><code>Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
self.connect()
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 200, in connect
self.connection = self.get_new_connection(conn_params)
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 187, in get_new_connection
connection = Database.connect(**conn_params)
File "/usr/local/lib64/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL: remaining connection slots are reserved for non-replication superuser connections
</code></pre> Ansible Plugin - Issue #9250 (CLOSED - CURRENTRELEASE): improper validation when passing json as ...https://pulp.plan.io/issues/92502021-08-18T15:41:42Zjsherril@redhat.comjsherril@redhat.com
<p>Json seems to be a subset of yaml in some cases and with some libraries. Pulp's parsing seems to allow it as something like this:</p>
<p>{"collections": "theforeman.foreman"}</p>
<p>works fine as a requirements.yaml on a remote. However when you sync, you get a traceback:</p>
<pre><code> {"traceback"=>
" File \"/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpcore_worker.py\", line 272, in _perform_task\n" +
" result = func(*args, **kwargs)\n" +
" File \"/usr/local/lib/python3.6/site-packages/pulp_ansible/app/tasks/collections.py\", line 130, in sync\n" +
" repo_version = d_version.create()\n" +
" File \"/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 151, in create\n" +
" loop.run_until_complete(pipeline)\n" +
" File \"/usr/lib64/python3.6/asyncio/base_events.py\", line 484, in run_until_complete\n" +
" return future.result()\n" +
" File \"/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 225, in create_pipeline\n" +
" await asyncio.gather(*futures)\n" +
" File \"/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 43, in __call__\n" +
" await self.run()\n" +
" File \"/usr/local/lib/python3.6/site-packages/pulp_ansible/app/tasks/collections.py\", line 778, in run\n" +
" await asyncio.gather(*tasks)\n" +
" File \"/usr/local/lib/python3.6/site-packages/pulp_ansible/app/tasks/collections.py\", line 587, in _fetch_collection_metadata\n" +
" namespace, name = requirements_entry.name.split(\".\")\n",
"description"=>"not enough values to unpack (expected 2, got 1)"},
</code></pre>
<p>So it seems like that format should either error on save (like other invalid yaml does), or work for syncing</p>