Pulp: Issueshttps://pulp.plan.io/https://pulp.plan.io/favicon.ico2021-09-13T08:29:41ZPulp
Planio Pulp - Story #9383 (CLOSED - DUPLICATE): As an administrator, I'd like the ability to easily dele...https://pulp.plan.io/issues/93832021-09-13T08:29:41Zwibbit
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulpcore/2044":<a href="https://github.com/pulp/pulpcore/issues/2044" class="external">https://github.com/pulp/pulpcore/issues/2044</a></p>
<hr>
<p>Using RPM as an example.</p>
<p>Currently if there is a "bad" rpm, that we want to remove from pulp, to ensure their is no ability for it to be republished, it's far from straight forward.</p>
<p>Current process, I believe is.</p>
<ol>
<li>Identify content to be removed.</li>
<li>Locate content in each version of each repository.</li>
<li>Delete content from each version of each repository.</li>
<li>Update publications, and distributions where appropriate.</li>
<li>Run delete orphaned data.</li>
</ol>
<p>It would be great if there was a simple "do it" button that I can metaphorically click, but I doubt that's realistic.</p>
<p>I think what may be more realistic is, some way of finding out which repositories/versions of a repository has the content and providing a report, that could then be used to feed into some one's internal machinery to delete/publish/update distributors and finally clean up orphaned content.</p>
<p>I had a quick look but couldn't find a pre-existing request for this feature.</p> RPM Support - Story #9131 (CLOSED - DUPLICATE): As an administrator, I'd like RPM repository sync...https://pulp.plan.io/issues/91312021-07-23T06:13:18Zwibbit
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2286":<a href="https://github.com/pulp/pulp_rpm/issues/2286" class="external">https://github.com/pulp/pulp_rpm/issues/2286</a></p>
<hr>
<p>Currently, though Pulp3 supports mirror lists, it does not currently support re-trying against a different host in the mirror list in the event of a package sync failure.</p>
<p>While attempting to use mirror lists while syncing fedora34 updates, after ~15 attempts I was not able to get to version 1 of the repository, as each time it would try and it would get a new mirror, and there would be a failed package of some kind.</p>
<p>Anecdotally I see this a lot when running a dnf update/upgrade where packages will fail and DNF will happily go off and try a new mirror, without this logic, for larger repositories that may have a lot of change, I'm unsure of the value of supporting mirror lists.</p> Pulp - Task #8789 (CLOSED - COMPLETE): Take down readthedocs siteshttps://pulp.plan.io/issues/87892021-05-20T16:05:09Zwibbit
<p>There are a few readthedocs.io sites relating to pulp3 and it's plugins, posting them below.</p>
<p>Not sure if they should all be removed, but certainly the core ones probably should be.</p>
<p><a href="https://readthedocs.org/projects/pulp-installer/" class="external">https://readthedocs.org/projects/pulp-installer/</a>
<a href="https://readthedocs.org/projects/pulp-certguard/" class="external">https://readthedocs.org/projects/pulp-certguard/</a>
<a href="https://readthedocs.org/projects/pulp-2to3-migration/" class="external">https://readthedocs.org/projects/pulp-2to3-migration/</a>
<a href="https://readthedocs.org/projects/pulp-rpm/" class="external">https://readthedocs.org/projects/pulp-rpm/</a>
<a href="https://readthedocs.org/projects/pulp-operator/" class="external">https://readthedocs.org/projects/pulp-operator/</a></p> Pulp - Story #8463 (CLOSED - DUPLICATE): As an administrator, I'd like a simple way to compare re...https://pulp.plan.io/issues/84632021-03-29T10:31:26Zwibbit
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulpcore/1983":<a href="https://github.com/pulp/pulpcore/issues/1983" class="external">https://github.com/pulp/pulpcore/issues/1983</a></p>
<hr>
<p>As an administrator of multiple Pulp3 instances, I would like a simple way to check to see if two publications of a repository are identical.</p>
<p>Expected use case.</p>
<p>Pulp Server 1:
I have x repositories, each with multiple publications and distributors (think 1 distributor/publication per "release").</p>
<p>This work is synchronised across multiple Pulp3 instances.</p>
<p>I would like a simple way to ensure that the presented content (current, don't care about added/removed), is identical amongst these instances.</p>
<p>Bonus points if it's accessible without the use of the API :)</p>
<p>Possibly related to <a href="https://pulp.plan.io/issues/5615" class="external">https://pulp.plan.io/issues/5615</a></p> Pulp - Task #8297 (CLOSED - DUPLICATE): Documentation relating to Directory Structure & File Systemshttps://pulp.plan.io/issues/82972021-02-23T13:44:13Zwibbit
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulpcore/1974":<a href="https://github.com/pulp/pulpcore/issues/1974" class="external">https://github.com/pulp/pulpcore/issues/1974</a></p>
<hr>
<p>Good morning all.</p>
<p>TLDR: Should the documentation be updated to reference DEPLOY_ROOT, and recommend that it is on a single partition, further ammendments to MEDIA_ROOT, WORKING_DIRECTORY, CHUNKED_UPLOAD_DIR, STATIC_ROOT or FILE_UPLOAD_TEMP_DIR are limited/restrictred to where it's actually pertinent and detailed information as to why you might choose to make that change with associated risks and caveats supplied where appropriate for that deployment style.</p>
<p>And the rest...</p>
<p>Re-reading the installation instructions and settings for Pulp3, I think there is a hole in relation to recommended partition layout, and associated settings.</p>
<p>We appear to have 3 documented variables (suggesting, that these are things we want to consider setting)</p>
<p>MEDIA_ROOT '/var/lib/pulp/media'
WORKING_DIRECTORY PosixPath('/var/lib/pulp/tmp')
CHUNKED_UPLOAD_DIR 'upload'</p>
<p>Being the curious fellow that I am, looking through dynaconf I also find the following variables.</p>
<p>DEPLOY_ROOT PosixPath('/var/lib/pulp')
STATIC_ROOT PosixPath('/var/lib/pulp/assets')
FILE_UPLOAD_TEMP_DIR PosixPath('/var/lib/pulp/tmp')</p>
<p>This appears to indicate the following layout.</p>
<p>/var/lib/pulp
/var/lib/pulp/assets
/var/lib/pulp/media/
/var/lib/pulp/media/??? (artifacts? 1TB on my current test install)
/var/lib/pulp/media/upload
/var/lib/pulp/tmp</p>
<p>The below note recommends MEDIA_ROOT and WORKING_DIRECTORY share the same filesystem.</p>
<p><a href="https://docs.pulpproject.org/pulpcore/settings.html#working-directory" class="external">https://docs.pulpproject.org/pulpcore/settings.html#working-directory</a></p>
<p>The only other "documentation" that I can find is <a href="https://pulp.plan.io/issues/7178" class="external">https://pulp.plan.io/issues/7178</a></p>
<p>This suggests that there is an expectation that at a minimum DEPLOY_ROOT is a single partition, sized appropriately to store all of pulp's content.</p>
<p>I'm purprosefully ignoring the question of whether CHUNKED_UPLOAD, WORKING_DIRECTORY or FILE_UPLOAD_TEMP_DIR would not be better suited to /var/spool or /var/cache</p> Pulp - Story #7977 (CLOSED - DUPLICATE): As a user I'd like the pulp content landing page to be l...https://pulp.plan.io/issues/79772020-12-10T16:27:39Zwibbit
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulpcore/1951":<a href="https://github.com/pulp/pulpcore/issues/1951" class="external">https://github.com/pulp/pulpcore/issues/1951</a></p>
<hr>
<p>Currently when view the pulp content landing page /pulp/content/ you are greeted with a list of all distributors that are hosted on the system (I believe).</p>
<p>Though the distributors support a directory structure, the landing page does not make use of this information.</p>
<p>Within our estate, within a year we could easily see ~30k distributors being created based on our current cadence (not including file and container distributors).</p>
<p>This will result in the landing page both being slow to load, and arguably unusable.</p>
<p>An example.</p>
<p>Say we have 4 distributors below.</p>
<pre><code>/release/rpm/rhel7/some_repo
/release/rpm/rhel7/some_repo2
/trunk/rpm/rhel7/some_repo
/upstream/rpm/rhel7/some_repo
</code></pre>
<p>The landing page as it stands will have 4 hyper links.</p>
<pre><code>/release/rpm/rhel7/some_repo
/release/rpm/rhel7/some_repo2
/trunk/rpm/rhel7/some_repo
/upstream/rpm/rhel7/some_repo
</code></pre>
<p>Would it be possible to have this landing page to look more like.</p>
<pre><code>/release
/trunk
/upstream
</code></pre>
<p>Which links to the next depth (just as traversing a normal directory, or web site).</p> RPM Support - Story #7708 (CLOSED - DUPLICATE): Improve content creation experiencehttps://pulp.plan.io/issues/77082020-10-14T11:39:39Zwibbit
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulp_rpm/2257":<a href="https://github.com/pulp/pulp_rpm/issues/2257" class="external">https://github.com/pulp/pulp_rpm/issues/2257</a></p>
<hr>
<p>Currently when attempting to create RPM content from an already uploaded artefact, if a NEVRA already exists for that content, the task fails.</p>
<p>There are many reasons why a single NEVRA could translate to multiple SHA's depending on build time/host etc.</p>
<p>From within Pulp, I don't believe (outside of passing a string), there is no way to find the HREF of the conflicting package.</p>
<ol>
<li>An artefact has no knowledge of its content (I can't query the artefact for its NEVRA, and then query the content)</li>
<li>Though you can see the filename (fairly reliable, but not guaranteed), when looking at content, you can't query by file, you'd need to iterate over the entire content, or pass the filename (not reliable as I believe filename and metadata can and often do vary)</li>
<li>When a task fails due to conflicting NEVRA, the Description string in the error message <em>DOES</em> include NEVRA + conflicting pkgId, however I'd rather avoid having to pass strings if at all possible.</li>
</ol>
<p>So some initial thoughts (not prescribing just initial brainstorming that would work for me).</p>
<ol>
<li>The task does not fail but instead returns the pkgId/href of the conflicting package.</li>
<li>Provide a method to query content for a would-be created content from an artifact, and <em>IF</em> it were to conflict provide.
a) The NEVRA of the conflicting package, which can then be used to query the content for said package.
b) All of the metadata from the would be created content (not the existing content).
b) Simply return the href for the pre-existing package.</li>
</ol>
<p>A & B, should have no implications for RBAC, as it's only returning the information for the artefact being referenced, and also allows the developer to subsequently query the content using the NEVRA and see if there are more concerning differences between the pre-existing content and the desired new content, and choose what to do (delete pre-existing, and replace with new, throw away new, and keep old, etc etc etc)</p> Pulp - Story #7544 (CLOSED - DUPLICATE): As a user of the python client libraries, I would like t...https://pulp.plan.io/issues/75442020-09-22T11:08:05Zwibbit
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulpcore/1935":<a href="https://github.com/pulp/pulpcore/issues/1935" class="external">https://github.com/pulp/pulpcore/issues/1935</a></p>
<hr>
<p>As a user of the python client libraries, while monitoring the state of tasks, in the event a publish task were to fail I would like to be able to report back to the user which publication and version the publish task failed on.</p> Container Support - Story #7419 (CLOSED - WONTFIX): As a user I can integrate with DOC,SEC: Docke...https://pulp.plan.io/issues/74192020-08-29T05:14:21Zwesturner
<p>Hey, are there and are there docs for Docker Notary / TUF features?</p>
<p>From <a href="https://twitter.com/westurner/status/1299542433106202625" class="external">https://twitter.com/westurner/status/1299542433106202625</a> ::</p>
<blockquote>
<p>#DockerNotary (#TUF) is justified and necessary because container images sidestep normal OS package integrity verifications (with e.g. GPG) and nobody runs debsums or rpm --verify on images they're trusting with all of their ops.
"Trusted Install Media"</p>
</blockquote>
<p><a href="https://docs.docker.com/notary/getting_started/" class="external">https://docs.docker.com/notary/getting_started/</a></p>
<p><a href="https://github.com/theupdateframework/specification/blob/master/tuf-spec.md" class="external">https://github.com/theupdateframework/specification/blob/master/tuf-spec.md</a></p>
<p><a href="https://en.wikipedia.org/wiki/The_Update_Framework_(TUF)" class="external">https://en.wikipedia.org/wiki/The_Update_Framework_(TUF)</a></p>
<p>(FWIU, <a class="issue tracker-3 status-12 priority-6 priority-default closed" title="Story: [Epic] Add security scanner integration (CLOSED - DUPLICATE)" href="https://pulp.plan.io/issues/6871">#6871</a> is also an opportunity for better security. )</p> Pulp - Story #7114 (CLOSED - DUPLICATE): Improve Artifact upload experiencehttps://pulp.plan.io/issues/71142020-07-09T13:38:17Zwibbit
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulpcore/1914":<a href="https://github.com/pulp/pulpcore/issues/1914" class="external">https://github.com/pulp/pulpcore/issues/1914</a></p>
<hr>
<p>As a user, when uploading an artifact, if that artifact already exists, provide a pulp_href to the pre-existing entity, thus allowing the user to pass that error output, and use the pre-existing artifact.</p> Pulp - Story #7036 (CLOSED - DUPLICATE): As a user, I would like to be able to filter all publica...https://pulp.plan.io/issues/70362020-06-23T15:59:31Zwibbit
<p><strong>Ticket moved to GitHub</strong>: "pulp/pulpcore/1912":<a href="https://github.com/pulp/pulpcore/issues/1912" class="external">https://github.com/pulp/pulpcore/issues/1912</a></p>
<hr>
<p>As a user, I would like to be able to filter all publications for a given repository.</p> Container Support - Task #7018 (CLOSED - COMPLETE): Broken Documentation Linkshttps://pulp.plan.io/issues/70182020-06-19T15:39:09Zwibbit
<p>Not sure if this is the right place to raise this, however...</p>
<p><a href="https://pypi.org/project/pulp-docker/" class="external">https://pypi.org/project/pulp-docker/</a>
This links to
<a href="https://pulp-docker.readthedocs.io/en/latest/" class="external">https://pulp-docker.readthedocs.io/en/latest/</a></p>
<p>Which is currently broken.</p> Pulp - Story #7015 (CLOSED - CURRENTRELEASE): As a user, I can associate a remote with a repositoryhttps://pulp.plan.io/issues/70152020-06-19T14:34:16Zwibbit
<p>Currently there is no way to link a repository with a remote, and as such, each time you wish to sync a repository against it's upstream link, you have to keep a record (or query for) the appropriate remote for that repo.</p>
<p>This can be done with careful consideration around naming conventions, however it would be nice if an optional link between a Repository and Remote was possible.</p>
<p>As such, once the Repository and Remote have been created an paired, a simply repository sync would be able to work out which remote to use, and sync.</p>
<a name="Design"></a>
<h1 >Design<a href="#Design" class="wiki-anchor">¶</a></h1>
<a name="Model"></a>
<h2 >Model<a href="#Model" class="wiki-anchor">¶</a></h2>
<p>Add a many-to-many foreign key between Repositories and Remotes. This association should be optional. For now Repositories can only have 0 or 1 remotes. Remotes can belong to any number of repositories.</p>
<p>Note: In the future we want to be able to support multiple remotes for repositories that can be synced at once. See <a href="https://pulp.plan.io/issues/7031" class="external">https://pulp.plan.io/issues/7031</a>. This is the purpose of having a many-to-many instead of one-to-many.</p>
<a name="REMOTE_TYPES"></a>
<h3 >REMOTE_TYPES<a href="#REMOTE_TYPES" class="wiki-anchor">¶</a></h3>
<p>Add an iterable to Repository similar to CONTENT_TYPES. This iterable will check that when a remote is added to a repo, it is of one of these types.</p>
<a name="API"></a>
<h2 >API<a href="#API" class="wiki-anchor">¶</a></h2>
<a name="Repositories"></a>
<h3 >Repositories<a href="#Repositories" class="wiki-anchor">¶</a></h3>
<p>Expose a remote hrefs field for repository endpoints. Users should be able to create/read/update this field. Validate the remote type against <code>REMOTE_TYPES</code> and that a user is only supplying 0..1 remote. Also, indicate in the help text that this field is optional.</p>
<a name="Remotes"></a>
<h3 >Remotes<a href="#Remotes" class="wiki-anchor">¶</a></h3>
<p>Expose a repository hrefs field for remotes. Users should be able to view this field to see which repos a remote belongs to. Also, indicate in the help text that this field is optional.</p>
<a name="Syncing"></a>
<h3 >Syncing<a href="#Syncing" class="wiki-anchor">¶</a></h3>
<p>Make the remote parameter on sync optional. If no remote is supplied, the sync task uses the remote field on Repository when syncing.</p>
<p>If a repository does not have a remote and no remote is supplied, raise an error.</p> RPM Support - Story #4229 (CLOSED - WONTFIX): Create yum repos with simple-md-filenames createrep...https://pulp.plan.io/issues/42292018-12-05T09:26:13Zvpapalia
<p>When creating a new yum repo, for metadata it appears pulp just uses default settings of --unique-md-filenames which prefixes metadata files with their checksum. We have a requirement to override this setting with --simple-md-filenames which will create metadata files without being prefixed with the files checksum. From searching it doesn't appear possible?</p> Pulp - Story #3885 (CLOSED - WONTFIX): As a user, I can specify distributor_id via pulp-admin dur...https://pulp.plan.io/issues/38852018-07-30T09:58:49Zwibbit
<p>Good morning all.</p>
<p>I'm currently creating rpm repo's via the api, and attach distributors as part of the process.</p>
<p>We are naming these distributors based on "the release" they represent, but of course, they are of distributor_type yum_distributor.</p>
<p>I've noticed that if I create a repo via the above approach, the pulp-admin tools can not be used to publish the repository, as they are expecting to have a distributor to be present with distributo_id: yum_distributor.</p>
<p>running a pulp-admin rpm repo sync successfully syncs, and publishes the repo.</p>
<p>Is there a reason that on a publish request, we would not be publishing based on distributor_type, as opposed to distributore_id?</p>
<p>Example repo</p>
<pre><code>[dfurlong@chlv-repo01 ~]$ pulp-admin repo list --details --repo-id=upstream_sles12.2-pool
+----------------------------------------------------------------------+
Repositories
+----------------------------------------------------------------------+
Id: upstream_sles12.2-pool
Display Name: pool
Description: None
Content Unit Counts:
Rpm: 3683
Yum Repo Metadata File: 2
Notes:
Parent: None
Platform: sles12.2
Release: upstream
Scratchpad:
Checksum Type: sha256
Importers:
Config:
Feed: https://updates.suse.com/SUSE/Products/SLE-SERVER/12-SP2/x
86_64/product
Num Retries: 5
Query Auth Token: myeSwPPGvfUqdMgBHlNWhOsog-xa4FbM4vn95es-W2qLIxrAks0jY8XnYQ
UwqDDbwTJJGJ_FBjw3DmZrQxF22zTdsj6eH0dqad1C0Oi55V3a_6CyiKyI
Pqil7vNrvIp8kYqJppZc3OgBgTmAcQ
Validate: True
Id: yum_importer
Importer Type Id: yum_importer
Last Override Config:
Type Skip List: iso, distribution
Last Sync: 2018-07-27T08:26:34Z
Last Updated: 2018-07-26T15:17:06Z
Repo Id: upstream_sles12.2-pool
Scratchpad:
Repomd Revision: 1477945506
Distributors:
Auto Publish: True
Config:
Generate Sqlite: True
Http: True
Https: True
Relative URL: sles12.2/upstream/sles12.2-pool/
Repoview: True
Distributor Type Id: yum_distributor
Id: upstream
Last Override Config:
Last Publish: None
Last Updated: 2018-07-30T09:36:51Z
Repo Id: upstream_sles12.2-pool
Scratchpad:
[dfurlong@chlv-repo01 ~]$ pulp-admin rpm repo publish run --repo-id=upstream_sles12.2-pool
+----------------------------------------------------------------------+
Publishing Repository [upstream_sles12.2-pool]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Task Failed
Missing resource(s): repo_id=upstream_sles12.2-pool,
distributor_id=yum_distributor
</code></pre>