Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-10-25T15:15:01ZPulp
Planio Ansible Plugin - Issue #9534 (CLOSED - DUPLICATE): Ansible Galaxy Repo failed to synchronize case...https://pulp.plan.io/issues/95342021-10-25T15:15:01Zstosem
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_ansible/747":<a href="https://github.com/pulp/pulp_ansible/issues/747" class="external">https://github.com/pulp/pulp_ansible/issues/747</a></p>
<hr>
<p>The galaxy.ansible.com API uses case-insensitive fields for the owner username and role name. The Ansible Galaxy Hub source code: <a href="https://github.com/ansible/galaxy/blob/e0e4b909171ddc6ca40f0ef2f7d4dce5680777ce/galaxy/api/views/roles.py#L50" class="external">https://github.com/ansible/galaxy/blob/e0e4b909171ddc6ca40f0ef2f7d4dce5680777ce/galaxy/api/views/roles.py#L50</a></p>
<p>But pulp usesnamespace=namespace. It causes the error on case-sensitive repositories (404 not found, or empty set). An example:</p>
<ul>
<li><a href="https://galaxy.ansible.com/api/v1/roles/?owner__username=Oefenweb" class="external">https://galaxy.ansible.com/api/v1/roles/?owner__username=Oefenweb</a></li>
<li><a href="https://galaxy.ansible.com/api/v1/roles/?owner__username=oefenweb" class="external">https://galaxy.ansible.com/api/v1/roles/?owner__username=oefenweb</a></li>
</ul>
<p>Galaxy Ansible returns the same result. But pulp isn't.</p>
<p>So we can't install roles like this using regular <code>ansible-galaxy install oefenweb.postfix</code> command with the Pulp repo.</p> Ansible Plugin - Issue #9220 (CLOSED - CURRENTRELEASE): content_ansible_collection_versions_creat...https://pulp.plan.io/issues/92202021-08-09T17:07:09Zsajha
<p>The <a href="https://docs.pulpproject.org/pulp_ansible/restapi.html#operation/upload_collection" class="external">https://docs.pulpproject.org/pulp_ansible/restapi.html#operation/upload_collection</a> endpoint allows you to upload a Collection tarball. This operation results in an artifact which can then be used to create a new collection content.</p>
<p>However, the endpoint to do so <a href="https://docs.pulpproject.org/pulp_ansible/restapi.html#operation/content_ansible_collection_versions_create" class="external">https://docs.pulpproject.org/pulp_ansible/restapi.html#operation/content_ansible_collection_versions_create</a> seems to have a lot of required fields which are presumably avaialble on the uploaded tarball itself. Is there a way to modify the flow so that associating an artifact from an uploaded collection tarball doesn't need to have all the other information passed.</p>
<p>I am assuming the flow would be to upload tarball, create a collection content using the artifact, add content created to a collection repository.</p> Ansible Plugin - Issue #9075 (CLOSED - CURRENTRELEASE): Downloads do not seem to use proxy authen...https://pulp.plan.io/issues/90752021-07-13T13:59:34Zfao89
<p>I tried with both file and rpm repos, if i specify a proxy url, username and password, i can check pulps' db and see:</p>
<pre><code>pulpcore=# select url, proxy_url, proxy_username, proxy_password from core_remote;
url | proxy_url | proxy_username | proxy_password
---------------------------------------------------------------+-----------------------------+----------------+----------------
https://jlsherrill.fedorapeople.org/fake-repos/needed-errata/ | http://192.168.122.182:8888 | admin | redhat
</code></pre>
<p>so i am fairly confident that the username and password are being set properly. However, when i sync, i get:</p>
<p>407, message='Proxy Authentication Required', url=URL('<a href="http://192.168.122.182:8888" class="external">http://192.168.122.182:8888</a>')</p>
<p>looking at my squid logs, i see:</p>
<p>1625605398.930 0 192.168.122.143 TCP_DENIED/407 4235 CONNECT jlsherrill.fedorapeople.org:443 - HIER_NONE/- text/html</p>
<p>So I believe pulp isn't properly sending the proxy auth correctly?</p>
<p>Note that my curl command:</p>
<pre><code> curl -vv -X GET https://jlsherrill.fedorapeople.org/fake-repos/needed-errata/repodata/repomd.xml --proxy http://192.168.122.182:8888 -H 'Proxy-Authorization: Basic YWRtaW46cmVkaGF0'
</code></pre>
<p>works without issue</p> Ansible Plugin - Issue #8957 (CLOSED - CURRENTRELEASE): URL validation error on remote displays r...https://pulp.plan.io/issues/89572021-06-23T18:27:37Zsajha
<p>The url validation is somehow leading to this: ["Invalid URL https://galaxy.ansible.com/. Ensure the URL ends '/'."]} This happens when I have some source specified in the requirements.yml without the trailing "/".. However the error displays the remote URL in the error which happens to have the trailing /..</p>
<p>(02:14:32 PM) rochacbruno: error is on <a href="https://github.com/pulp/pulp_ansible/blob/master/pulp_ansible/app/serializers.py#L166" class="external">https://github.com/pulp/pulp_ansible/blob/master/pulp_ansible/app/serializers.py#L166</a>
(02:14:44 PM) rochacbruno: should be url=url instead of data["url"]</p> Ansible Plugin - Issue #8742 (CLOSED - DUPLICATE): Ansible playbooks fail on database migrationshttps://pulp.plan.io/issues/87422021-05-13T07:09:03Zsli720
<p>I run the ansible playbooks from pulp-installer 3.12.2 (also once with the latest commit of the master branch) on a fresh CentOS 8 as I did last week but suddenly it breaks on database migration with the error:</p>
<pre><code class="text syntaxhl" data-language="text">File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/click/core.py\", line 2108, in __init__", " ) from None", "ValueError: 'default' must be a list when 'multiple' is true."], "stdout": "", "stdout_lines": []}
</code></pre>
<p>Is the click package incompatible? I really don't know now which version of the pulp-installer really works, cause on every rerun of the playbooks another issue appears mostly due to 3rd party incompatibilities (also django upgrade recently broke the installation) . Would it somehow possible to fix all this dependencies somewhere (which file?) cause when I check out 3.12.2 I always expect the same result? Or is there something I do wrong?</p> Ansible Plugin - Issue #8290 (CLOSED - CURRENTRELEASE): Pulp_ansible unable to sync from galaxy e...https://pulp.plan.io/issues/82902021-02-22T18:00:20Zsajha
<p>Steps to reproduce:
Set up a remote with a token and point to a galaxy instance.</p>
<pre><code> {
"auth_url": null,
"ca_cert": null,
"client_cert": null,
"client_key": null,
"download_concurrency": 10,
"name": "hub_1-128797",
"password": null,
"policy": "immediate",
"proxy_url": null,
"pulp_created": "2021-02-19T19:05:47.231796Z",
"pulp_href": "/pulp/api/v3/remotes/ansible/collection/91c20a7d-8477-48d8-86df-46755cdffcae/",
"pulp_last_updated": "2021-02-22T16:53:00.377091Z",
"requirements_file": "---\ncollections:\n- ibm.cloudcollection",
"tls_validation": false,
"token": "xxx-valid-token-xxx",
"url": "https://hostname/api/galaxy/content/community/",
"username": null
},
</code></pre>
<p>When syncing this, I get a 403 from the endpoint with the following logs:</p>
<pre><code>Feb 22 17:13:53 hub-sat1-v4-2-1-jhardy gunicorn[74032]: pulp: django.request:WARNING: Forbidden: /api/galaxy/content/community/
Feb 22 17:13:53 hub-sat1-v4-2-1-jhardy gunicorn[74032]: pulp [-]: - - [22/Feb/2021:17:13:53 +0000] "GET /api/galaxy/content/community/ HTTP/1.0" 403 112 "-" "pulpcore/3.7.3 (cpython 3.6.8-final0, Linux x86_64) (aiohttp 3.7.2)"
Feb 22 17:13:54 hub-sat1-v4-2-1-jhardy gunicorn[74032]: pulp: django.request:WARNING: Forbidden: /api/galaxy/content/community/api/
Feb 22 17:13:54 hub-sat1-v4-2-1-jhardy gunicorn[74032]: pulp [-]: - - [22/Feb/2021:17:13:54 +0000] "GET /api/galaxy/content/community/api/ HTTP/1.0" 403 112 "-" "pulpcore/3.7.3 (cpython 3.6.8-final0, Linux x86_64) (aiohttp 3.7.2)"
</code></pre>
<p>PS: "tls_validation": false is set on the remote.</p>
<p>A ansible-galaxy client with the below cfg file:</p>
<p>[galaxy]
server_list = community_repo</p>
<p>[galaxy_server.community_repo]
url=<a href="https://hostname/api/galaxy/content/community/" class="external">https://hostname/api/galaxy/content/community/</a>
token=xxx-valid-token-xxx</p>
<p>gives the following results:</p>
<p>Secure:</p>
<pre><code># ansible-galaxy collection install -p ./ ibm.cloudcollection
Process install dependency map
ERROR! Unknown error when attempting to call Galaxy at 'https://hub-sat1-v4-2-1-jhardy.c.ansible-tower-engineering.internal/api/galaxy/content/community/api': <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)>
</code></pre>
<p>Ignore SSL certificate validation errors with -c :</p>
<pre><code># ansible-galaxy collection install -c -p ./ ibm.cloudcollection
Process install dependency map
Starting collection install process
Installing 'ibm.cloudcollection:1.21.0' to '/root/ansible_collections/ibm/cloudcollection'
</code></pre>
<p>Expected result:
Pulp_Ansible should be able to ignore cert validations and sync successfully when tls_validation is off.</p> Ansible Plugin - Issue #5783 (CLOSED - CURRENTRELEASE): Ansible repositories API doesn't have a /...https://pulp.plan.io/issues/57832019-11-22T15:59:14ZiballouAnsible Plugin - Issue #5572 (CLOSED - CURRENTRELEASE): pulp_ansible/app/tasks/collections.py:imp...https://pulp.plan.io/issues/55722019-10-15T17:33:25Zalikins
<pre><code>pulp-worker_1 | django.core.exceptions.FieldError: Cannot resolve keyword '_id' into field. Choices are: _artifacts, authors, collection, collection_id, content_ptr, content_ptr_id, contentartifact, contents, dependencies, description, docs_blob, documentation, homepage, is_certified, is_highest, issues, license, name, namespace, pulp_created, pulp_id, pulp_last_updated, pulp_type, repositories, repository, search_vector, tags, version, version_memberships
pulp-worker_1 | Traceback (most recent call last):
pulp-worker_1 | File "/venv/lib64/python3.6/site-packages/rq/worker.py", line 822, in perform_job
pulp-worker_1 | rv = job.perform()
pulp-worker_1 | File "/venv/lib64/python3.6/site-packages/rq/job.py", line 605, in perform
pulp-worker_1 | self._result = self._execute()
pulp-worker_1 | File "/venv/lib64/python3.6/site-packages/rq/job.py", line 611, in _execute
pulp-worker_1 | return self.func(*self.args, **self.kwargs)
pulp-worker_1 | File "/code/pulp-ansible/pulp_ansible/app/tasks/collections.py", line 183, in import_collection
pulp-worker_1 | content_q = CollectionVersion.objects.filter(_id=collection_version.pk)
pulp-worker_1 | File "/venv/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
pulp-worker_1 | return getattr(self.get_queryset(), name)(*args, **kwargs)
pulp-worker_1 | File "/venv/lib64/python3.6/site-packages/django/db/models/query.py", line 892, in filter
pulp-worker_1 | return self._filter_or_exclude(False, *args, **kwargs)
pulp-worker_1 | File "/venv/lib64/python3.6/site-packages/django/db/models/query.py", line 910, in _filter_or_exclude
pulp-worker_1 | clone.query.add_q(Q(*args, **kwargs))
pulp-worker_1 | File "/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py", line 1290, in add_q
pulp-worker_1 | clause, _ = self._add_q(q_object, self.used_aliases)
pulp-worker_1 | File "/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py", line 1318, in _add_q
pulp-worker_1 | split_subq=split_subq, simple_col=simple_col,
pulp-worker_1 | File "/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py", line 1190, in build_filter
pulp-worker_1 | lookups, parts, reffed_expression = self.solve_lookup_type(arg)
pulp-worker_1 | File "/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py", line 1049, in solve_lookup_type
pulp-worker_1 | _, field, _, lookup_parts = self.names_to_path(lookup_splitted, self.get_meta())
pulp-worker_1 | File "/venv/lib64/python3.6/site-packages/django/db/models/sql/query.py", line 1420, in names_to_path
pulp-worker_1 | "Choices are: %s" % (name, ", ".join(available)))
pulp-worker_1 | django.core.exceptions.FieldError: Cannot resolve keyword '_id' into field. Choices are: _artifacts, authors, collection, collection_id, content_ptr, content_ptr_id, contentartifact, contents, dependencies, description, docs_blob, documentation, homepage, is_certified, is_highest, issues, license, name, namespace, pulp_created, pulp_id, pulp_last_updated, pulp_type, repositories, repository, search_vector, tags, version, version_memberships
</code></pre> Ansible Plugin - Issue #5555 (CLOSED - CURRENTRELEASE): Error when uploading a collection to v3https://pulp.plan.io/issues/55552019-10-08T13:36:46Zdaviddavis
<p>If you try to upload a collection to the v3 api, you get:</p>
<pre><code>'CollectionVersionViewSet' object has no attribute 'create'
</code></pre>
<p>.</p> Ansible Plugin - Test #5333 (CLOSED - COMPLETE): As a user, I can sync collections from another P...https://pulp.plan.io/issues/53332019-08-22T17:28:11Zdaviddavis
<p>In lieu of setting up two Pulp servers, I would just use two repositories.</p>
<p>Test steps:</p>
<p>1. Create a collections repo, remote, sync and publish<br>
2. Create a second repository and a remote which points to the distribution in step 1<br>
3. Sync the second repo and confirm it has the same contents as the first.</p> Ansible Plugin - Issue #5299 (CLOSED - CURRENTRELEASE): Bindings for pulp \ pulp ansible return p...https://pulp.plan.io/issues/52992019-08-19T16:05:10Zosapryki
<p>Fields described in DRF serializers as `JSONField()` [1] are included in swagger schema as `string` [2] fields, which leads to invalid deserialization on a client side when using pulp bindings.</p>
<p>Those fields deserialized as a `str` type containing <code>repr()</code> result of json field passed from server.</p>
<p>This issue is a blocker, because it cannot be reliably workarounded on a client side as there is no reliable way to deserialize python <code>repr()</code> back to a normal object.</p>
<p>[1] <a href="https://github.com/pulp/pulp_ansible/blob/master/pulp_ansible/app/serializers.py#L197" class="external">https://github.com/pulp/pulp_ansible/blob/master/pulp_ansible/app/serializers.py#L197</a><br>
[2] <a href="https://pulp-ansible.readthedocs.io/en/latest/restapi.html#operation/content_ansible_collections_list" class="external">https://pulp-ansible.readthedocs.io/en/latest/restapi.html#operation/content_ansible_collections_list</a></p> Ansible Plugin - Issue #5226 (CLOSED - CURRENTRELEASE): Migration 0004_add_fulltext_search_indexe...https://pulp.plan.io/issues/52262019-08-06T20:09:00Zosapryki
<p>Migration <code>0004_add_fulltext_search_indexes</code> fails with the following exception when trying to execute against database table that actually contains data.</p>
<pre><code>Traceback (most recent call last):
File "/venv/bin/django-admin", line 11, in <module>
sys.exit(execute_from_command_line())
File "/venv/lib64/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/venv/lib64/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/venv/lib64/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "/venv/lib64/python3.6/site-packages/django/core/management/base.py", line 364, in execute
output = self.handle(*args, **options)
File "/venv/lib64/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
res = handle_func(*args, **kwargs)
File "/venv/lib64/python3.6/site-packages/django/core/management/commands/migrate.py", line 234, in handle
fake_initial=fake_initial,
File "/venv/lib64/python3.6/site-packages/django/db/migrations/executor.py", line 117, in migrate
state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/venv/lib64/python3.6/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/venv/lib64/python3.6/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
state = migration.apply(state, schema_editor)
File "/venv/lib64/python3.6/site-packages/django/db/migrations/migration.py", line 124, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File "/venv/lib64/python3.6/site-packages/django/db/migrations/operations/special.py", line 105, in database_forwards
self._run_sql(schema_editor, self.sql)
File "/venv/lib64/python3.6/site-packages/django/db/migrations/operations/special.py", line 130, in _run_sql
schema_editor.execute(statement, params=None)
File "/venv/lib64/python3.6/site-packages/django/db/backends/base/schema.py", line 137, in execute
cursor.execute(sql, params)
File "/venv/lib64/python3.6/site-packages/django/db/backends/utils.py", line 99, in execute
return super().execute(sql, params)
File "/venv/lib64/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/venv/lib64/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/venv/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/venv/lib64/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/venv/lib64/python3.6/site-packages/django/db/backends/utils.py", line 82, in _execute
return self.cursor.execute(sql)
django.db.utils.ProgrammingError: more than one row returned by a subquery used as an expression
</code></pre>
<p>The SQL query that causes this exception is this:</p>
<pre><code> UPDATE ansible_collectionversion AS c
SET search_vector = (
SELECT
setweight(to_tsvector(coalesce(namespace,'')), 'A')
|| setweight(to_tsvector(coalesce(name, '')), 'A')
|| (
SELECT
setweight(to_tsvector(
coalesce(string_agg("ansible_tag"."name", ' '), '')
), 'B')
FROM
"ansible_tag" INNER JOIN "ansible_collectionversion_tags" ON ("ansible_tag"."_id" = "ansible_collectionversion_tags"."tag_id")
)
|| (
SELECT
setweight(to_tsvector(
coalesce(string_agg(cvc ->> 'name', ' '), '')
), 'C')
FROM jsonb_array_elements(cv.contents) AS cvc
)
|| setweight(to_tsvector(coalesce(description, '')), 'D')
FROM ansible_collectionversion cv
)
</code></pre> Ansible Plugin - Issue #5001 (CLOSED - CURRENTRELEASE): Provide content_ansible_collection_read m...https://pulp.plan.io/issues/50012019-06-20T18:37:10Zsajha
<p>We need the ability to read ansible collection content similar to other content types ex: <a href="https://pulp-ansible.readthedocs.io/en/latest/restapi.html#operation/content_ansible_roles_read" class="external">https://pulp-ansible.readthedocs.io/en/latest/restapi.html#operation/content_ansible_roles_read</a>.</p> Ansible Plugin - Issue #4997 (CLOSED - CURRENTRELEASE): Pulp3 Ansible Collection remote doesn't s...https://pulp.plan.io/issues/49972019-06-20T16:19:01Zsajha
<p>*Steps to Reproduce:*++<br>
Steps based on httpie and jq from documentation: <a href="https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html" class="external">https://pulp-ansible.readthedocs.io/en/latest/workflows/index.html</a></p>
<p>#Create 2 separate repos:<br>
http POST $BASE_ADDR/pulp/api/v3/repositories/ name=test1<br>
http POST $BASE_ADDR/pulp/api/v3/repositories/ name=test2</p>
<p>export REPO_HREF1=$(http $BASE_ADDR/pulp/api/v3/repositories/ | \<br>
jq -r '.results[] | select(.name == "test1") | ._href')</p>
<p>export REPO_HREF2=$(http $BASE_ADDR/pulp/api/v3/repositories/ | \<br>
jq -r '.results[] | select(.name == "test2") | ._href')</p>
<p>#Create collection remote:</p>
<p>http POST $BASE_ADDR/pulp/api/v3/remotes/ansible/collection/ \<br>
name='bar' \<br>
url='<a href="https://galaxy-dev.ansible.com" class="external">https://galaxy-dev.ansible.com</a>' \<br>
whitelist='newswangerd.collection_demo'</p>
<p>export REMOTE_HREF=$(http $BASE_ADDR/pulp/api/v3/remotes/ansible/collection/ | jq -r '.results[] | select(.name == "bar") | ._href')</p>
<p>#Sync both repos with the remote</p>
<p>export TASK_URL1=$(http POST $BASE_ADDR$REMOTE_HREF'sync/' repository=$REPO_HREF1 \<br>
| jq -r '.task')</p>
<p>export TASK_URL2=$(http POST $BASE_ADDR$REMOTE_HREF'sync/' repository=$REPO_HREF2 \<br>
| jq -r '.task')</p>
<p>#Notice that a new repo version is created by 1st task but none by the 2nd.</p>
<p>http $BASE_ADDR$TASK_URL1</p>
<p>http $BASE_ADDR$TASK_URL2</p>
<p>*<strong>*</strong>*********************************************************</p>
<p>I tried with a second remote pointing to the same URL and with the same whitelist and it doesn't seem to create a repo version either.</p> Ansible Plugin - Issue #3898 (CLOSED - CURRENTRELEASE): Filters inherit from a Django's FilterSet...https://pulp.plan.io/issues/38982018-08-02T15:13:08Zvdusek
<p>AnsibleRoleFilter and AnsibleRoleVersionFilter inherit from Django's filterset.FilterSet class not from our BaseFilterSet. I suppose this could cause problems in generating documentation.</p>