Infrastructure & Hosting » History » Revision 4
Revision 3 (bmbouter, 11/07/2017 11:21 PM) → Revision 4/13 (bmbouter, 11/21/2017 07:50 PM)
# Current Infrastructure & Hosting ## Websites pulpproject.org - The main website is a Jekyll based website. The content comes from the [pulp/pulpproject.org](https://github.com/pulp/pulpproject.org/) repo. It is currently built by the [OSCI](https://osci.io/) maintained Jekyll builder and hosted on OSCI infrastructure. via Github Pages. The OSCI also manages redirects, TLS cert renewal, domain name registration is owned by Red Hat IT, and the DNS for this site. is managed by Red Hat IT. docs.pulpproject.org - The docs website is built via Sphinx by Jenkins nightly using [this script](https://github.com/pulp/pulp_packaging/blob/master/ci/docs-builder.py). That builds a specific branch of the docs from pulp/pulp and then pushes them to the hosting environment on openshift V2. ## Testing Environments. Enivornments. **Jenkins Master** - Jenkins master is hosted by Red Hat central CI. This environment provides lots of functions: including PR test running, PR docs testing, nightly docs building, nightly rpm production, upstream/downstream automation, and other functions. **Jenkins Slaves** - All Jenkins work is coordinated by the master but performed by slaves. The Jenkins slaves run in an environment called ci-rhos which is a Red Hat openstack instance which provides slave resources. **Nodepool** - The slaves are requested by a nodepool instance. The Pulp team self-supports a nodepool instance which creates the slaves for Jenkins. **Travis** - Travis is the test runner for Pulp3 specifically. Those tests are not run by Jenkins. ## Distribution of Bits **PyPI** - Pulp3 is being distributed via PyPI. **repos.fedorapeople.org** - The official Pulp rpms are available via https://repos.fedorapeople.org/pulp/pulp/stable/ We rsync the bits to that environment whenever releases occur. ## Issue tracking **pulp.plan.io** - Hosted redmine instance for issues and features to be filed for Pulp and some plugins. All plugins maintained by the core team are expected to be there. Community plugins are also welcome to root their issue trackers there. **Github Issues** - Some parts of Pulp have their issues tracked via Github Issues. Specifically, pulp-smash and the repo tracking defects on pulpproject.org both use Github issues. # Infrastructure Needs ## Website pulpproject.org needs to be moved out of the environment it is hosted in currently and into a new environment. We need to get off Github Pages (pulpproject.org) because the site is [marked as spam](https://pulp.plan.io/issues/2477) and to allows us to add legitimate SSL which would avoid the scary name/certificate mismatch security error you get now. docs.pulpproject.org also needs to be moved out of the environment it is hosted in because that environment is being sunset within the coming months. Specifically it is hosted in openshift V2 and it needs to move. We can host docs.pulpproject.org both websites on one machine. Specifically these are the same OSCI infra as pulpproject.org requirements: A webhosting environment that can: - regular patching/updates for security - key based SSH only - the ability to push website changes via rsync which is how we deploy to the existing pulpproject.org environment already. - provide SNI based TLS - automatic renewal of letsencrypt based TLS certificates - redirect configuration to roll over all traffic from http to https. ## Vagrant VM building We want a machine to build Vagrant VMs. This needs to run virtualization on the machine, so it needs to be either virt-in-virt with hardware support or a bare-metal machine. Here are some basic requirements: - EL7 or similar - 50G of storage (we are building imagines inside this environment) - small CPU - no backups required - must be able to run virtualization, so either virt-in-virt or a bare metal machine. ## Distribute Pulp with Pulp We want an environment that can allow us to store/test/promote Pulp rpms/Python packages using Pulp. We are switching to Copr as a new build system and that won't retain our builds as long so we need to store them elsewhere. Also Pulp is built for package promotion so we should use it. We need: - An EL7 with - 30GB of space (10GB of storage for Pulp data, 20GB for Mongo) - small cpu - medium RAM, at least 4GB but 6 would be ideal. ## Code Performance Testing We've wanted to understand the performance impact of code changes for a long time. We want to run [tests like these](https://pulp.plan.io/projects/pulp/wiki/Performance_Test_Plan). Overall this idea is inspired by reports like [these](https://people.apache.org/\~mikemccand/lucenebench/index.html) for the Lucene project. We likely need one-bare metal machine that we can load fresh everyday and test on. We also need some way to freshly load that machine each day with a tool like oVirt or Foreman. It would be great if we could: - Have a bare-metal server box - That can have EL7 loaded onto it every night and updated to the latest - That Jenkins can load Pulp onto daily and run a performance test suite on it ## Nodepool We want CI-OSP to take over supporting Nodepool. They can host it wherever they want.
.