https://pulp.plan.io/https://pulp.plan.io/favicon.ico2016-11-28T22:24:59ZPulpPulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=160572016-11-28T22:24:59Zbmbouterbmbouter@redhat.com
<ul><li><strong>Blocked by</strong> <i><a class="issue tracker-2 status-9 priority-6 priority-default closed" href="/issues/2443">Task #2443</a>: create the 'ansible-pulp' project inside pulp GitHub organization</i> added</li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=160582016-11-28T22:26:35Zbmbouterbmbouter@redhat.com
<ul></ul><p>Pulp today does a lot of system configuration at the RPM level. We should generalize that by relying on Ansible playbook to handle two things:</p>
<ul>
<li>non Python dependency installation</li>
<li>system configuration</li>
</ul>
<p>That Ansible work is being tracked as part of issue 2443 so I've named that as a blocker for this issue</p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=160592016-11-28T22:27:02Zbmbouterbmbouter@redhat.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-3 status-11 priority-6 priority-default closed child" href="/issues/278">Story #278</a>: Add documentation about Pulp's setup.py requirements</i> added</li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=161112016-11-30T20:48:24Zbmbouterbmbouter@redhat.com
<ul></ul><p>In Pulp3 we will have several setup.py files. I believe to publish on PyPI each published package corresponds with 1 setup.py file. Is that correct?</p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=162062016-12-03T17:25:24Zbmbouterbmbouter@redhat.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-11 priority-6 priority-default closed" href="/issues/2463">Task #2463</a>: Reorganize 3.0-dev branch</i> added</li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=162802016-12-06T20:55:49Zbmbouterbmbouter@redhat.com
<ul></ul><p>Answering my own question here, yes I believe each setup.py we maintain will need to be published separately on PyPI.</p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=171172017-01-23T20:22:31Zbmbouterbmbouter@redhat.com
<ul></ul><p>There are two main questions we need answers to:</p>
<ul>
<li>What are the PyPI names going to be?</li>
<li>Will all packages install under a top level directory or not?</li>
</ul>
<a name="What-are-the-PyPI-names-going-to-be"></a>
<h2 >What are the PyPI names going to be?<a href="#What-are-the-PyPI-names-going-to-be" class="wiki-anchor">¶</a></h2>
<p>Pulp will need to register a lot of names on PyPI because we have many distinct Python packages. For example platform needs at least 4: one for the server, the cli, the common bits, and the streamer. This ticket is only for platform, but the decisions here will likely be applied to any given plugin the same way which have similar needs. For example, each plugin we maintain will likely need 3: one for the plugin, one for its CLI extensions, and another for the plugins common bits. With 5 actively maintained plugins that 15 packages and 4 for platform for an expected total of 19 PyPI registrations.</p>
<p>Through some in-person brainstorming, <a class="user active" href="https://pulp.plan.io/users/7">mhrivnak</a> and I came up with the ones below. All of these are available on PyPI at this time.</p>
<a name="Idea-1-Use-a-prefix-with-underscores-after-it"></a>
<h3 >Idea 1: Use a prefix with underscores after it<a href="#Idea-1-Use-a-prefix-with-underscores-after-it" class="wiki-anchor">¶</a></h3>
<p>Three prefixes we thought of are <code>pulpproj</code>, <code>pulpproject</code>, or <code>pulp</code>. Any $PREFIX would be used like <code>$PREFIX_platform</code>, <code>$PREFIX_cli</code>, <code>$PREFIX_common</code>, and <code>$PREFIX_streamer</code>.</p>
<p>For the <code>pulpproj</code> prefix, you would pip install with:</p>
<pre><code>pip install pulpproj_platform # Install the server which brings in pulpproj_common as a dependency
pip install pulpproj_cli # Install the cli which brings in pulpproj_common as a dependency
pip install pulpproj_streamer # Install the streamer which brings in pulpproj_platform as a dependency
</code></pre>
<p>For the <code>pulpproject</code> prefix, you would pip install with:</p>
<pre><code>pip install pulpproject_platform # Install the server which brings in pulpproject_common as a dependency
pip install pulpproject_cli # Install the cli which brings in pulpproject_common as a dependency
pip install pulpproject_streamer # Install the streamer which brings in pulpproject_platform as a dependency
</code></pre>
<p>For the <code>pulp</code> prefix, you would pip install with:</p>
<pre><code>pip install pulp_platform # Install the server which brings in pulp_common as a dependency
pip install pulp_cli # Install the cli which brings in pulp_common as a dependency
pip install pulp_streamer # Install the streamer which brings in pulp_platform as a dependency
</code></pre>
<a name="Idea-2-A-variation-on-Idea-1-except-dont-append-_platform-for-platform"></a>
<h3 >Idea 2: A variation on Idea 1 ^, except don't append <code>_platform</code> for platform<a href="#Idea-2-A-variation-on-Idea-1-except-dont-append-_platform-for-platform" class="wiki-anchor">¶</a></h3>
<p>This only works for the prefix of <code>pulpproj</code> or <code>pulpproject</code>. For that one, platform would use the following names:</p>
<pre><code>pip install pulpproject # Install the server which brings in pulpproject_common as a dependency
pip install pulpproject_cli # Install the cli which brings in pulpproject_common as a dependency
pip install pulpproject_streamer # Install the streamer which brings in pulpproject as a dependency
</code></pre>
<p>This does not work for the prefix <code>pulp</code> because the <a href="https://pypi.python.org/pypi/PuLP" class="external">existing pulp on PyPI</a> already reserves it.</p>
<a name="Will-all-packages-install-under-a-top-level-directory-or-not"></a>
<h2 >Will all packages install under a top level directory or not?<a href="#Will-all-packages-install-under-a-top-level-directory-or-not" class="wiki-anchor">¶</a></h2>
<p>This is not something we need to decide to move forward, but our name choices above will restrict our layout choices, so I include it here also. Note that the way Pulp packages are imported are mostly decided by this decision.</p>
<a name="yes-all-packages-will-install-under-a-top-level-dir"></a>
<h3 >'yes' all packages will install under a top level dir<a href="#yes-all-packages-will-install-under-a-top-level-dir" class="wiki-anchor">¶</a></h3>
<p>With this, we would have to use the <code>pulpproj</code> or <code>pulpproject</code> prefixes. For example it would look like this:</p>
<pre><code>[bmbouter@dhcp129-10 ~]$ tree /usr/lib/python2.7/site-packages/pulpproject/
/usr/lib/python2.7/site-packages/pulpproject/
├── cli
├── common
├── platform
└── streamer
</code></pre>
<p>With this type of layout, an import in platform of common bits would be: <code>from pulpproject.common import ...</code></p>
<p>Note, we cannot use the <code>pulp</code> as the top level directory because we do not own the registration for <code>pulp</code> on PyPI.</p>
<a name="no-all-packages-will-install-in-their-own-site-packages-folders"></a>
<h3 >'no' all packages will install in their own <code>site-packages</code> folders<a href="#no-all-packages-will-install-in-their-own-site-packages-folders" class="wiki-anchor">¶</a></h3>
<p>With this, every package would be installed as a top level package on its own. Any prefix would work for this. For the <code>pulpproject</code> prefix it would have the following paths:</p>
<pre><code>/usr/lib/python2.7/site-packages/pulpproject_cli
/usr/lib/python2.7/site-packages/pulpproject_common
/usr/lib/python2.7/site-packages/pulpproject_platform
/usr/lib/python2.7/site-packages/pulpproject_streamer
</code></pre>
<p>In this case an import in platform importing from common would import as: <code>from pulpproject_common import ...</code></p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=171202017-01-23T20:53:35Zmhrivnakmhrivnak@redhat.com
<ul></ul><p>Good summary. Two small additions:</p>
<p>1. The length problem really shows itself with plugins.</p>
<pre><code>pip install pulp_rpm_extensions
pip install pulpproject_rpm_extensions
</code></pre>
<p>2. The platform packages are likely to be a little different, but those details don't matter much for this discussion. But just to illustrate it, I expect we would start from the idea of one python package and one rpm package per second-level namespace in today's tree. For example:</p>
<p>pulp.app becomes pulp_app<br>
pulp.bindings becomes pulp_bindings</p>
<p>and so on for pulp.common, pulp.plugin, pulp.tasking, etc.</p>
<p>Then we will likely find that several of those namespaces always need to be delivered together, and could bundle them into one package and RPM, such as pulp_platform. I would anticipate pulp_platform to include "app", "plugin", and "tasking" namespaces at least, if not more.</p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=190702017-04-10T17:50:35Zbizhangbizhang@redhat.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-9 priority-6 priority-default closed" href="/issues/2699">Task #2699</a>: Ensure all default CA_PATH settings to work with multiple distribution default locations in Pulp3</i> added</li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=197082017-05-17T18:03:22Zbmbouterbmbouter@redhat.com
<ul><li><strong>Subject</strong> changed from <i>Publish Pulp on PyPI</i> to <i>Create PyPI accounts for Pulp3 core and all plugins</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/19708/diff?detail_id=20333">diff</a>)</li></ul><p>Rewriting based on lots of discussion. See the recap of that discussion here: <a href="https://www.redhat.com/archives/pulp-dev/2017-May/msg00030.html" class="external">https://www.redhat.com/archives/pulp-dev/2017-May/msg00030.html</a></p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=197092017-05-17T18:26:24Zttereshcttereshc@redhat.com
<ul><li><strong>Groomed</strong> changed from <i>No</i> to <i>Yes</i></li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=197592017-05-18T14:48:48Zbmbouterbmbouter@redhat.com
<ul><li><strong>Subject</strong> changed from <i>Create PyPI accounts for Pulp3 core and all plugins</i> to <i>Create a PyPI account for Pulp3 core and all plugins</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/19759/diff?detail_id=20385">diff</a>)</li><li><strong>Sprint/Milestone</strong> set to <i>39</i></li></ul><p>We talked about the possibility of pre-registering names for more plugins. We decided that we need more info about how to delegate access if we pre-register a plugin that then needs to be pushed and maintained by a plugin writer.</p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=199632017-06-01T15:27:46Zbizhangbizhang@redhat.com
<ul><li><strong>Assignee</strong> set to <i>bizhang</i></li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=199662017-06-01T17:32:01Zbizhangbizhang@redhat.com
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>ASSIGNED</i></li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=200212017-06-05T16:00:28Zbizhangbizhang@redhat.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/20021/diff?detail_id=20627">diff</a>)</li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=200222017-06-05T16:17:34Zbizhangbizhang@redhat.com
<ul><li><strong>Status</strong> changed from <i>ASSIGNED</i> to <i>POST</i></li></ul><p>PR:</p>
<p><a href="https://github.com/pulp/pulp/pull/3044" class="external">https://github.com/pulp/pulp/pull/3044</a><br>
<a href="https://github.com/pulp/devel/pull/58" class="external">https://github.com/pulp/devel/pull/58</a><br>
<a href="https://github.com/pulp/pulp_ostree/pull/92" class="external">https://github.com/pulp/pulp_ostree/pull/92</a><br>
<a href="https://github.com/pulp/pulp_docker/pull/197" class="external">https://github.com/pulp/pulp_docker/pull/197</a><br>
<a href="https://github.com/pulp/pulp_puppet/pull/263" class="external">https://github.com/pulp/pulp_puppet/pull/263</a><br>
<a href="https://github.com/pulp/pulp_python/pull/108" class="external">https://github.com/pulp/pulp_python/pull/108</a><br>
<a href="https://github.com/pulp/pulp_rpm/pull/1055" class="external">https://github.com/pulp/pulp_rpm/pull/1055</a><br>
<a href="https://github.com/pulp/pulp_file/pull/3" class="external">https://github.com/pulp/pulp_file/pull/3</a></p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=200732017-06-06T19:14:47Zbizhangbizhang@redhat.com
<ul><li><strong>Status</strong> changed from <i>POST</i> to <i>MODIFIED</i></li></ul><p><a href="https://pypi.python.org/pypi/pulpcore" class="external">https://pypi.python.org/pypi/pulpcore</a></p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=200762017-06-06T19:27:22Zbmbouterbmbouter@redhat.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-12 priority-6 priority-default closed" href="/issues/2531">Issue #2531</a>: pulp python modules use other project's namespace</i> added</li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=200772017-06-06T20:03:37Zbizhangbizhang@redhat.com
<ul></ul><p>Mhrivnak found out that it's super easy to transfer ownership of pypi packages, so I am going to preemptively register some plugins names, and we can transfer ownership to community members later:</p>
<p>pulp-deb<br>
pulp-deb-common<br>
pulp-deb-cli</p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=200882017-06-07T14:55:33Zbmbouterbmbouter@redhat.com
<ul></ul><p>-0 to registering additional plugins. I think the plugin owner should handle their own plugin start to finish. They may not even want our names.</p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=200992017-06-07T17:35:56Zbizhangbizhang@redhat.com
<ul></ul><p><a class="user active" href="https://pulp.plan.io/users/1">bmbouter</a>, yeah I only ended up registering the pulp-deb packages. mhrivnak and I could not decided on the other package names (pulp-npm vs pulp-js, pulp-ruby vs pulp-gem and pulp-whateverjavauses)</p>
<p>The original motivation of registering the additional packages was so no one else could encroach on our package names. but because there was no obvious name for the other plugins, I decided not to, since I didn't want squat on names that might not be used in the future.</p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=201002017-06-07T17:39:59Zbmbouterbmbouter@redhat.com
<ul></ul><p>That sounds fine; there is a pulp-deb plugin at least so whenever @misa is ready to publish on PyPI we can hand it over to him. Thanks for registering all of these. This was well done.</p> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=259742018-03-08T23:14:19Zbmbouterbmbouter@redhat.com
<ul><li><strong>Sprint</strong> set to <i>Sprint 20</i></li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=259982018-03-08T23:15:44Zbmbouterbmbouter@redhat.com
<ul><li><strong>Sprint/Milestone</strong> deleted (<del><i>39</i></del>)</li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=417812019-04-25T16:47:11Zdaviddavis
<ul><li><strong>Sprint/Milestone</strong> set to <i>3.0.0</i></li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=429922019-04-26T20:39:02Zbmbouterbmbouter@redhat.com
<ul><li><strong>Tags</strong> deleted (<del><i>Pulp 3</i></del>)</li></ul> Pulp - Story #2444: Create a PyPI account for Pulp3 core and all pluginshttps://pulp.plan.io/issues/2444?journal_id=506542019-12-13T17:16:28Zbmbouterbmbouter@redhat.com
<ul><li><strong>Status</strong> changed from <i>MODIFIED</i> to <i>CLOSED - CURRENTRELEASE</i></li></ul>