Story #1852: Move Pulp to a Software Collection on EL6

Story #1852

Updated by jcline@redhat.com over 8 years ago
This story is the result of https://pulp.plan.io/issues/931. Currently, all Pulp code must be Python 2.6 compatible. This makes transitioning to Python 3 compatible code difficult, as many features introduced in 2.7 are specifically for writing code that runs on 2.7 and 3.x. Furthermore, we are stuck with a very old version of Django (1.4). Therefore, it would be helpful if we could raise our minimum to requirements to Python 2.7 and Django 1.6 (for now). Both these ship with RHEL 7 so in order to support RHEL6, we can use Software Collections. 

 Software Collections[0] install into /opt/ or /var/opt, so they don't interfere with the versions that ship with the OS. We need to extend the existing Python 2.7 SCL[1] with all our Python dependencies. For each Python dependency we have that isn't provided by the Python 2.7 SCL, we must create an RPM package that uses the SCL. This can be done by acquiring the existing spec files for these dependencies and tweaking them (potentially with spec2scl[2]). Note that this applies not just to our direct dependencies, but also the dependencies of _those_ dependencies, all the way to the packages that have no dependencies beyond those already provided by the Python 2.7 SCL or the platform itself. 

 Once all the dependencies are packaged for the SCL, Pulp itself needs its spec file modified to use the SCL. It also needs the init scripts and commands installed into /usr/bin to use the SCL paths. All these changes can occur as patches applied by the spec file (or even as install steps in the spec itself). 

 All these spec files (and their associated source tarballs) need to be tracked. The Foreman has a SCL and they use Git with git-annex[3]. We could probably do the same. My recommendation is that we move our spec files out of the pulp and pulp_<plugin> repos and into a package repository. Within that repository we should have branches for each platform, much like Fedora does (so an "el6", "el7", "f22", "f23", and "f24" branch). For the "el6" branch, we check in all the SCL-related files. This has two advantages: we get rid of all our nasty if/else blocks for various platforms and we don't have to add more for the SCL work. 

 Here is a *potentially incomplete* dependency tree for *pulp-server* on EL6: 
 <pre> 
 pulp-server-2.8.2-1.el6.noarch [cmd line] 
  \_    Django14-1.4.21-1.el6.noarch [1: Django14] 
  \_    m2crypto-0.20.2-9.el6.x86_64 [1: m2crypto] 
  \_    mod_wsgi-3.4-2.pulp.el6.x86_64 [1: mod_wsgi >= 3.4-1.pulp] 
  \_    python-blinker-1.1-1.el6.noarch [1: python-blinker] 
  \_    python-celery-3.1.11-1.el6.noarch [2: python-celery < 3.2.0, python-celery >= 3.1.0] 
  |     \_    pyparsing-1.5.6-1.el6.noarch [1: pyparsing] 
  |     \_    python-amqp-1.4.9-1.el6.noarch [1: python-amqp] 
  |     \_    1:python-billiard-3.3.0.17-2.el6.x86_64 [1: python-billiard >= 3.3.0.17] 
  |     \_    python-dateutil-1.4.1-6.el6.noarch [1: python-dateutil] 
  |     |     \_    tzdata-2016c-1.el6.noarch [1: tzdata] 
  |     \_    python-importlib-1.0.2-1.el6.noarch [1: python-importlib] 
  |     \_    1:python-kombu-3.0.33-4.pulp.el6.noarch [1: python-kombu >= 3.0.15] 
  |     |     \_    python-amqp-1.4.9-1.el6.noarch [2: python-amqp < 2.0, python-amqp >= 1.4.9] 
  |     |     \_    python-libs-2.6.6-64.el6.i686 [1: python-ordereddict] 
  |     |     \_    python-libs-2.6.6-64.el6.x86_64 [1: python-ordereddict] 
  |     |     \_    python-ordereddict-1.1-2.el6.noarch [1: python-ordereddict] 
  |     \_    python-setuptools-0.6.10-3.el6.noarch [1: python-setuptools] 
  |     \_    pytz-2010h-2.el6.noarch [1: pytz] 
  \_    python-gofer-2.7.5-1.el6.noarch [1: python-gofer >= 2.5] 
  \_    python-httplib2-0.7.7-1.el6.noarch [1: python-httplib2] 
  \_    python-ldap-2.3.10-1.el6.x86_64 [1: python-ldap] 
  \_    python-mongoengine-0.10.5-1.el6.noarch [1: python-mongoengine >= 0.10.0] 
  |     \_    python-blinker-1.1-1.el6.noarch [1: python-blinker] 
  |     \_    python-pymongo-3.2-1.el6.x86_64 [1: python-pymongo >= 3.2] 
  |     \_    python-pymongo-gridfs-3.2-1.el6.x86_64 [1: python-pymongo-gridfs >= 3.2] 
  |     |     \_    python-pymongo-3.2-1.el6.x86_64 [1: python-pymongo(x86-64) = 3.2-1.el6] 
  \_    python-nectar-1.5.1-1.el6.noarch [1: python-nectar >= 1.5.0] 
  |     \_    python-requests-2.6.0-3.el6.noarch [1: python-requests >= 2.4.3] 
  |     |     \_    python-chardet-2.2.1-1.el6.noarch [1: python-chardet >= 2.2.1-1] 
  |     |     \_    python-libs-2.6.6-64.el6.i686 [1: python-ordereddict >= 1.1] 
  |     |     \_    python-libs-2.6.6-64.el6.x86_64 [1: python-ordereddict >= 1.1] 
  |     |     \_    python-ordereddict-1.1-2.el6.noarch [1: python-ordereddict >= 1.1] 
  |     |     \_    python-urllib3-1.10.2-1.el6.noarch [1: python-urllib3 >= 1.10.2-1] 
  |     |     |     \_    python-backports-ssl_match_hostname-3.4.0.2-2.el6.noarch [1: python-backports-ssl_match_hostname] 
  |     |     |     |     \_    python-backports-1.0-5.el6.x86_64 [1: python-backports] 
  |     |     |     \_    python-libs-2.6.6-64.el6.i686 [1: python-ordereddict] 
  |     |     |     \_    python-libs-2.6.6-64.el6.x86_64 [1: python-ordereddict] 
  |     |     |     \_    python-ordereddict-1.1-2.el6.noarch [1: python-ordereddict] 
  |     |     |     \_    python-six-1.9.0-2.el6.noarch [1: python-six] 
  \_    python-oauth2-1.5.211-8.el6.noarch [1: python-oauth2 >= 1.5.211] 
  |     \_    python-httplib2-0.7.7-1.el6.noarch [1: python-httplib2] 
  \_    python-pulp-common-2.8.2-1.el6.noarch [1: python-pulp-common = 2.8.2] 
  |     \_    python-iniparse-0.3.1-2.1.el6.noarch [1: python-iniparse] 
  \_    python-pulp-repoauth-2.8.2-1.el6.noarch [1: python-pulp-repoauth = 2.8.2] 
  |     \_    mod_wsgi-3.4-2.pulp.el6.x86_64 [1: mod_wsgi >= 3.4-1.pulp] 
  |     \_    python-pulp-common-2.8.2-1.el6.noarch [1: python-pulp-common = 2.8.2] 
  |     \_    python-rhsm-0.99.12-1.el6.noarch [1: python-rhsm] 
  |     |     \_    m2crypto-0.20.2-9.el6.x86_64 [1: m2crypto] 
  |     |     \_    python-iniparse-0.3.1-2.1.el6.noarch [1: python-iniparse] 
  |     \_    python-rhsm-1.14.3-1.el6.x86_64 [1: python-rhsm] 
  |     |     \_    m2crypto-0.20.2-9.el6.x86_64 [1: m2crypto] 
  |     |     \_    python-dateutil-1.4.1-6.el6.noarch [1: python-dateutil] 
  |     |     \_    python-iniparse-0.3.1-2.1.el6.noarch [1: python-iniparse] 
  |     |     \_    rpm-python-4.8.0-47.el6.x86_64 [1: rpm-python] 
  |     \_    python-setuptools-0.6.10-3.el6.noarch [1: python-setuptools] 
  \_    python-pymongo-3.2-1.el6.x86_64 [1: python-pymongo >= 3.0.0] 
  \_    python-qpid-0.14-11.el6_3.noarch [1: python-qpid] 
  |     \_    python-saslwrapper-0.14-1.el6.x86_64 [1: python-saslwrapper >= 0.10] 
  \_    python-semantic_version-2.2.0-6.el6.noarch [1: python-semantic_version >= 2.2.0] 
  \_    python-setuptools-0.6.10-3.el6.noarch [1: python-setuptools] 
 </pre> 
 In the above list I stripped out all dependencies to non-Python packages. It could be I accidentally got rid of a Python library I simply didn't recognize, so the tree should not be treated as an authoritative reference. 

 [0] https://access.redhat.com/documentation/en-US/Red_Hat_Software_Collections/2/html-single/Packaging_Guide/index.html 
 [1] https://access.redhat.com/documentation/en-US/Red_Hat_Software_Collections/2/html-single/Packaging_Guide/index.html#sect-Extending_the_python27_and_rh-python34_Software_Collections 
 [2] https://pypi.python.org/pypi/spec2scl 
 [3] https://github.com/theforeman/foreman-packaging/tree/rpm/1.11
Back
Project

Profile

Help

Pulp

Story #1852
