Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2020-08-26T20:17:43ZPulp
Planio Ansible Plugin - Story #7397 (CLOSED - CURRENTRELEASE): Expose docs_blob information for collecti...https://pulp.plan.io/issues/73972020-08-26T20:17:43Znewswangerd
<p>Right now collection sync doesn't pull down enough metadata to be useful to the UI.</p>
<p>We need to</p>
<ul>
<li>update the automation hub V3 api to provide all the information the ui needs via the collection version endpoint at <code>v3/collections/<namespace>/<collection>/versions/<version_number>/</code>. (issue <a href="https://github.com/ansible/galaxy_ng/issues/403" class="external">403</a>).</li>
<li>update the pulp sync process to pull metadata from the details endpoints during syncs</li>
</ul> Ansible Plugin - Story #6648 (CLOSED - CURRENTRELEASE): As a user, I can use longer urls in model...https://pulp.plan.io/issues/66482020-05-05T14:00:18Zosapryki
<p>At the moment URL fields (documentation, homepage, issues, repository) in <code>CollectionVersion</code> model are defined as <code>models.URLField(max_length=128, ...)</code>.
This is not enough sometimes as URL length can easily exceed 128 symbols length.</p>
<p>We had several reports from our customers that collection import failed due to bigger URL length used in collection metadata.</p>
<p>Fields length should be increased to reasonably bigger size.</p> Ansible Plugin - Task #5300 (CLOSED - CURRENTRELEASE): Add data models for tracking collection im...https://pulp.plan.io/issues/53002019-08-19T16:36:08ZosaprykiAnsible Plugin - Story #5288 (CLOSED - CURRENTRELEASE): As a user, I can sync collections from an...https://pulp.plan.io/issues/52882019-08-15T20:02:49Zsajha
<p>This is from before the requirements.yaml PR went in..</p>
<p><strong>STEPS TO REPRODUCE:</strong></p>
<pre><code>#Create repo
http POST $BASE_ADDR/pulp/api/v3/repositories/ name=foo
export REPO_HREF=$(http $BASE_ADDR/pulp/api/v3/repositories/ | \
jq -r '.results[] | select(.name == "foo") | ._href')
#Create remote
http POST $BASE_ADDR/pulp/api/v3/remotes/ansible/collection/ name='bar' url='https://galaxy-dev.ansible.com/api/v2/collections/testing/ansible_testing_content'
export REMOTE_HREF=$(http $BASE_ADDR/pulp/api/v3/remotes/ansible/collection/ | jq -r '.results[] | select(.name == "bar") | ._href')
#Sync repo with Remote
export TASK_URL=$(http POST $BASE_ADDR$REMOTE_HREF'sync/' repository=$REPO_HREF \
| jq -r '.task')
export REPOVERSION_HREF=$(http $BASE_ADDR$TASK_URL | jq -r '.created_resources[]')
#Create Distribution for the synced Repo
export TASK_URL=$(http POST $BASE_ADDR/pulp/api/v3/distributions/ansible/ansible/ \
name='baz' \
base_path='foo' \
repository=${REPO_HREF} | jq -r '.task')
export DIST_PATH=$(http $BASE_ADDR$TASK_URL | jq -r '.created_resources[]')
#Form SYNC_URL
export GALAXY_URL=$(http $BASE_ADDR$DIST_PATH | jq -r '.mazer_url')
export SYNC_URL=$GALAXY_URL'/api/v2/collections/testing/ansible_testing_content'
http $SYNC_URL
# Notice Error in /var/log/messages around multiple collection versions being returned.
http $SYNC_URL'/versions'
#-----------------Try to use $SYNC_URL as remote for 2nd repo----------
#Create 2nd repo to sync from the sync url
export REPO_HREF=$(http $BASE_ADDR/pulp/api/v3/repositories/ | \
jq -r '.results[] | select(.name == "foo1") | ._href')
#Create remote with url pointing to pulp's galaxy endpoint of 1st repo
http POST $BASE_ADDR/pulp/api/v3/remotes/ansible/collection/ name='bar1' url=$SYNC_URL
export REMOTE_HREF=$(http $BASE_ADDR/pulp/api/v3/remotes/ansible/collection/ | jq -r '.results[] | select(.name == "bar1") | ._href')
#Notice url in the Remote
http $BASE_HREF$REMOTE_HREF
#Sync repo2 with remote poinitng to 1st repos pulp-galaxy url
export TASK_URL=$(http POST $BASE_ADDR$REMOTE_HREF'sync/' repository=$REPO_HREF \
| jq -r '.task')
#Notice failure in /var/log/messages
http $BASE_HREF$TASK_URL
#http://localhost:24817/pulp_ansible/galaxy/foo/api/v2/collections/testing/ansible_testing_content/versions/4.0.4/ as sync URL fails too
</code></pre>
<p>Tried this with a collection <a href="https://galaxy.ansible.com/api/v2/collections/devoperate/base/versions/" class="external">https://galaxy.ansible.com/api/v2/collections/devoperate/base/versions/</a> having only one version. Got the error:</p>
<pre><code>"error": {
"code": null,
"description": "string indices must be integers",
"traceback": " File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 822, in perform_job\n rv = job.perform()\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 605, in perform\n self._result = self._execute()\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 611, in _execute\n return self.func(*self.args, **self.kwargs)\n File \"/usr/local/lib/pulp/src/pulp-ansible/pulp_ansible/app/tasks/collections.py\", line 66, in sync\n d_version.create()\n File \"/usr/local/lib/pulp/src/pulpcore-plugin/pulpcore/plugin/stages/declarative_version.py\", line 169, in create\n loop.run_until_complete(pipeline)\n File \"/usr/lib64/python3.6/asyncio/base_events.py\", line 484, in run_until_complete\n return future.result()\n File \"/usr/local/lib/pulp/src/pulpcore-plugin/pulpcore/plugin/stages/api.py\", line 209, in create_pipeline\n await asyncio.gather(*futures)\n File \"/usr/local/lib/pulp/src/pulpcore-plugin/pulpcore/plugin/stages/api.py\", line 43, in __call__\n await self.run()\n File \"/usr/local/lib/pulp/src/pulp-ansible/pulp_ansible/app/tasks/collections.py\", line 195, in run\n async for metadata in self._fetch_collections():\n File \"/usr/local/lib/pulp/src/pulp-ansible/pulp_ansible/app/tasks/collections.py\", line 242, in _fetch_collections\n \"namespace\": result[\"namespace\"][\"name\"],\n"
},
</code></pre> Ansible Plugin - Task #5287 (CLOSED - CURRENTRELEASE): Add certified status and related endpoint ...https://pulp.plan.io/issues/52872019-08-15T16:04:49ZosaprykiAnsible Plugin - Story #5251 (CLOSED - DUPLICATE): Dependency Downloading for Ansible Collection ...https://pulp.plan.io/issues/52512019-08-12T16:30:40Zsajha
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_ansible/685":<a href="https://github.com/pulp/pulp_ansible/issues/685" class="external">https://github.com/pulp/pulp_ansible/issues/685</a></p>
<hr>
<a name="Dependency-Downloading-when-syncing-collections"></a>
<h3 >Dependency Downloading when syncing collections:<a href="#Dependency-Downloading-when-syncing-collections" class="wiki-anchor">¶</a></h3>
<p>When syncing a collection, we would like to have the collections listed as dependencies to be synced into the repository as well.<br>
Example: <a href="https://docs.ansible.com/ansible/devel/dev_guide/collections_galaxy_meta.html#examples" class="external">https://docs.ansible.com/ansible/devel/dev_guide/collections_galaxy_meta.html#examples</a></p>
<p>For a collection to be synced, having the below dependencies:</p>
<pre><code>dependencies:
"other_namespace.collection1": ">=1.0.0"
"other_namespace.collection2": ">=2.0.0,<3.0.0"
"anderson55.my_collection": "*" # note: "*" selects the highest version available
</code></pre>
<p>The 3 listed collections should also get synced when syncing the collection.</p> Ansible Plugin - Story #5250 (CLOSED - CURRENTRELEASE): Add requirements.yaml specification suppo...https://pulp.plan.io/issues/52502019-08-12T16:22:23Zsajha
<a name="Requirements-with-requirementsyml"></a>
<h3 >Requirements with requirements.yml:<a href="#Requirements-with-requirementsyml" class="wiki-anchor">¶</a></h3>
<p>We would like to have the ability to use the requirements.yml documented here: <a href="https://docs.ansible.com/ansible/devel/dev_guide/collections_tech_preview.html#install-multiple-collections-with-a-requirements-file" class="external">https://docs.ansible.com/ansible/devel/dev_guide/collections_tech_preview.html#install-multiple-collections-with-a-requirements-file</a></p>
<pre><code>---
collections:
# With just the collection name
- my_namespace.my_collection
# With the collection name, version, and source options
- name: my_namespace.my_other_collection
version: 'version range identifiers (default: ``*``)'
source: 'The Galaxy URL to pull the collection from (default: ``--api-server`` from cmdline)'
</code></pre>
<p>This would sync the collection my_namespace.my_collection and the specified version of my_namespace.my_other_collection. The version specification would be similar to <a href="https://docs.ansible.com/ansible/devel/dev_guide/collections_tech_preview.html#installing-an-older-version-of-a-collection" class="external">https://docs.ansible.com/ansible/devel/dev_guide/collections_tech_preview.html#installing-an-older-version-of-a-collection</a></p>
<a name="Field-Name"></a>
<h3 >Field Name<a href="#Field-Name" class="wiki-anchor">¶</a></h3>
<p>The field on the CollectionRemote should be called <code>requirements_file</code>.</p>
<a name="Validation-of-data-Requirement"></a>
<h3 >Validation of data Requirement<a href="#Validation-of-data-Requirement" class="wiki-anchor">¶</a></h3>
<p>Note that Pulp expects to have all content from a single source and does not expect to have sources defined per content item (as the "source" field allows in this spec). As such, Pulp should not accept a requirements.yaml that contains a definition for "source". If specified when creating/saving a remote it should be rejected as invalid with a message from the user.</p> Ansible Plugin - Task #5244 (CLOSED - CURRENTRELEASE): Make links returned by pulp_ansible endpoi...https://pulp.plan.io/issues/52442019-08-09T19:00:45Zosapryki
<p>URLs to related resources and pagination links currently are automatically generated from Django views and url configs.<br>
These links include `/pulp_ansible` prefix, that when proxied by galaxy should not be included in order to return valid Galaxy URLs.</p> Ansible Plugin - Task #5243 (CLOSED - CURRENTRELEASE): One shot uploader for Galaxy API v3https://pulp.plan.io/issues/52432019-08-09T18:57:37Zosapryki
<p>Implement endpoint <code>POST /pulp_ansible/galaxy/galaxy/api/v3/artifacts/collections/</code> that accepts artifact file, pushes it to storage and triggers importer task.</p> Ansible Plugin - Task #5225 (CLOSED - CURRENTRELEASE): Implement collections and collection versi...https://pulp.plan.io/issues/52252019-08-06T11:03:04ZosaprykiAnsible Plugin - Task #5224 (CLOSED - CURRENTRELEASE): Implement custom pagination and error hand...https://pulp.plan.io/issues/52242019-08-06T11:02:10ZosaprykiAnsible Plugin - Task #5223 (CLOSED - CURRENTRELEASE): Add highest version mark to CollectionVers...https://pulp.plan.io/issues/52232019-08-06T11:01:02ZosaprykiAnsible Plugin - Story #5105 (CLOSED - CURRENTRELEASE): Implement advanced collection modelshttps://pulp.plan.io/issues/51052019-07-10T12:37:43Zosapryki
<p>Update collection models to store metadata retrieved from collection as a result of artifact importing and linting process.<br>
Collection models should be match when possible existing Galaxy collection models.</p> Ansible Plugin - Story #5023 (CLOSED - CURRENTRELEASE): Ansible Collection sync does not have dat...https://pulp.plan.io/issues/50232019-06-24T19:44:50Zsajha
<p>The progress report in the collection sync task is empty. We should ideally receive data in a structure similar to files. <a href="https://docs.pulpproject.org/en/3.0/nightly/restapi.html#operation/tasks_read" class="external">https://docs.pulpproject.org/en/3.0/nightly/restapi.html#operation/tasks_read</a></p> Ansible Plugin - Story #5018 (CLOSED - CURRENTRELEASE): Add artifact sha details to the content a...https://pulp.plan.io/issues/50182019-06-24T19:39:05Zsajha
<p>It would be helpful to see artifact sha details on the content details/list results to avoid an additional call to artifacts API to fetch sha details.<br>
This endpoint should ideally provide sha details in results: <a href="https://pulp-ansible.readthedocs.io/en/latest/restapi.html#operation/content_ansible_collections_list" class="external">https://pulp-ansible.readthedocs.io/en/latest/restapi.html#operation/content_ansible_collections_list</a></p>