Project

Profile

Help

Infrastructure & Hosting » History » Sprint/Milestone 5

bmbouter, 03/02/2018 06:16 PM
Updating for the changes that recently happened.

1 1 bmbouter
# Current Infrastructure & Hosting
2
3
## Websites
4
5 5 bmbouter
pulpproject.org - The main website is a Jekyll based website. The content comes from the [pulp/pulpproject.org](https://github.com/pulp/pulpproject.org/) repo. It is currently built by the [OSCI](https://osci.io/) maintained Jekyll builder and hosted on OSCI infrastructure. The OSCI also manages redirects, TLS cert renewal, and DNS for the site.
6 1 bmbouter
7 5 bmbouter
docs.pulpproject.org - The docs website is built via Sphinx by Jenkins nightly using [this script](https://github.com/pulp/pulp_packaging/blob/master/ci/docs-builder.py). That builds a specific branch of the docs from pulp/pulp and then pushes them to the hosting environment hosted by OSCI infrastructure. The OSCI also manages redirects, TLS cert renewal, and DNS for the site.
8 1 bmbouter
9 4 bmbouter
## Testing Environments.
10 1 bmbouter
11
**Jenkins Master** - Jenkins master is hosted by Red Hat central CI. This environment provides lots of functions: including PR test running, PR docs testing, nightly docs building, nightly rpm production, upstream/downstream automation, and other functions.
12
13
**Jenkins Slaves** - All Jenkins work is coordinated by the master but performed by slaves. The Jenkins slaves run in an environment called ci-rhos which is a Red Hat openstack instance which provides slave resources.
14
15
**Nodepool** - The slaves are requested by a nodepool instance. The Pulp team self-supports a nodepool instance which creates the slaves for Jenkins.
16
17
**Travis** - Travis is the test runner for Pulp3 specifically. Those tests are not run by Jenkins.
18
19
## Distribution of Bits
20
21
**PyPI** - Pulp3 is being distributed via PyPI.
22
23
**repos.fedorapeople.org** - The official Pulp rpms are available via https://repos.fedorapeople.org/pulp/pulp/stable/ We rsync the bits to that environment whenever releases occur.
24
25
## Issue tracking
26
27
**pulp.plan.io** - Hosted redmine instance for issues and features to be filed for Pulp and some plugins. All plugins maintained by the core team are expected to be there. Community plugins are also welcome to root their issue trackers there.
28
29
**Github Issues** - Some parts of Pulp have their issues tracked via Github Issues. Specifically, pulp-smash and the repo tracking defects on pulpproject.org both use Github issues.
30
31
# Infrastructure Needs
32
33
## Vagrant VM building
34
35
We want a machine to build Vagrant VMs. This needs to run virtualization on the machine, so it needs to be either virt-in-virt with hardware support or a bare-metal machine. Here are some basic requirements:
36
37
  - EL7 or similar
38
  - 50G of storage (we are building imagines inside this environment)
39 2 dkliban@redhat.com
  - small CPU
40 1 bmbouter
  - no backups required
41
  - must be able to run virtualization, so either virt-in-virt or a bare metal machine.
42
43
## Distribute Pulp with Pulp
44
45
We want an environment that can allow us to store/test/promote Pulp rpms/Python packages using Pulp. We are switching to Copr as a new build system and that won't retain our builds as long so we need to store them elsewhere. Also Pulp is built for package promotion so we should use it. We need:
46
47
  - An EL7 with 
48
  - 30GB of space (10GB of storage for Pulp data, 20GB for Mongo)
49
  - small cpu
50
  - medium RAM, at least 4GB but 6 would be ideal.
51
52 3 bmbouter
## Code Performance Testing
53 1 bmbouter
54 3 bmbouter
We've wanted to understand the performance impact of code changes for a long time. We want to run [tests like these](https://pulp.plan.io/projects/pulp/wiki/Performance_Test_Plan). Overall this idea is inspired by reports like [these](https://people.apache.org/\~mikemccand/lucenebench/index.html) for the Lucene project.
55
56
We likely need one-bare metal machine that we can load fresh everyday and test on. We also need some way to freshly load that machine each day with a tool like oVirt or Foreman. It would be great if we could:
57 1 bmbouter
58
  - Have a bare-metal server box
59
  - That can have EL7 loaded onto it every night and updated to the latest
60
  - That Jenkins can load Pulp onto daily and run a performance test suite on it
61
62
## Nodepool
63
64
We want CI-OSP to take over supporting Nodepool. They can host it wherever they want.