Task #2325
closed
Distribute Pulp with Pulp
Description
We're planning to do out package builds using fedora's copr infrastructure from Pulp 3. We've identified two needs that need to be met for this to be viable:
- Old releases need to be archived, so that downstream folks like katello can pull specific release versions. This is also just a good thing to do. Currently, we only keep the latest release of a given x.y stream, and earlier releases can't be easily found online.
- Releases need to happen atomically. COPR supports this, but offers limited control over the exact moment a repository's metadata is regenerated.
Pulp meets both of these needs, and should be the tool we use to distribute Pulp. :)
This pulp instance will need to be secure and the following things should be ensured:
- Pulp's REST API should be ran on a non default port
- Pulp's content serving API should be run on ports 80 and 443
- mongo set up with authentication and listen locally (through sockets)
- message brokers also set up with authentication and configured to only listen locally
- The RHEL7 hardening guide is followed [0]
[0] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/pdf/Security_Guide/Red_Hat_Enterprise_Linux-7-Security_Guide-en-US.pdf
I added some checklist items. One, "Come up with an upgrade policy" is a little ambiguous, but I'm not sure how to phrase it more accurately. It refers to the upgrade policy of the Pulp installation being used to distribute Pulp, and whether we want to always keep it up to date with the latest release, or only update it as-needed. It doesn't refer to how we update the versions of pulp being distributed, that's a separate checklist item.
- Blocks Issue #2414: `ImproperlyConfigured` exception in for detail list view endpoint added
- Blocks deleted (Issue #2414: `ImproperlyConfigured` exception in for detail list view endpoint)
- Blocks Task #2145: Ansible playbooks need to pull from Pulp added
I think OS1 has been shutdown or nearly shutdown at this point. Open Source and Standards (OSAS) has offered some hosting for upstream Pulp's needs, like maybe hosting this environment. I'm emailing them to get some more details about hosting a Pulp inside the OSAS infrastructure.
- Sprint/Milestone set to 45
This is in-progress and will get groomed ASAP.
We will use an OSAS environment that they provide an EL7 box that we can manage and deploy Pulp onto it. I believe pcreech or bizhang have access to that.
- Description updated (diff)
- Description updated (diff)
I added that the broker should be configured to only listen locally.
I don't understand the middle bullet point.
Also will this be the url that users consumer directly from or are we distributing the bits elsewhere somehow?
- Description updated (diff)
bmbouter, since this is public facing people should be using it to consume our bits. I don't think we'll be distributing out the pulp3 bits to fedorapeople, but can be convinced otherwise.
As far as SNI goes pcreech mentioned hosting multiple https sites from the same IP, but I think that can (and should) be taken off, since this story only deals with getting pulp up and running.
Can I assume that "Pulp should be ran on a non default port" is referring to the REST API service?
What is the goal of running it on a non-default port?
The intent behind moving the REST api to a different port is that doing such would allow us to have more access control at the firewall level. This machine has a public IP, and therefore anything we set to listen will listen on the public IP. Having our web service listen at the same ip:port endpoint as our content will allow anyone coming in to attempt to access our rest api as well.
Moving to a separate port will allow us to implement stricter access controls on the rest api that we interface with, therefore helping reduce our attack surface.
The other option would be to have the rest api listen to local only, therefore necessitating that all interaction with pulp happens solely on that machine.
- Description updated (diff)
I clarified in the description about the ports saying that the rest API will be on one non-standard port and that content will be served via 80 and 443. Is that right?
- Sprint/Milestone changed from 45 to 46
- Groomed changed from No to Yes
- Sprint/Milestone changed from 46 to 47
- Status changed from NEW to ASSIGNED
- Assignee set to bizhang
- Sprint/Milestone changed from 47 to 48
- Sprint/Milestone changed from 48 to 52
- Sprint/Milestone changed from 52 to 53
- Sprint/Milestone changed from 53 to 54
- Sprint/Milestone changed from 54 to 56
- Sprint/Milestone deleted (
56)
- Sprint changed from Sprint 33 to Sprint 34
- Sprint deleted (
Sprint 34)
- Assignee deleted (
bizhang)
- Status changed from ASSIGNED to NEW
I think we're moving away from self-distribution and towards container distribution in registries we don't operate and also PyPI delivery. Does doing this effort still make sense?
Note that the infra wiki shows this initative also: https://pulp.plan.io/projects/pulp/wiki/Infrastructure_&_Hosting#Distribute-Pulp-with-Pulp
Also AIUI the OSCI group has provisioned a machine in the community data center for this purpose. Since we're not using it, and if we decide to not go forward with it, we should ask them to deprovisioning it.
What do you all think?
- Status changed from NEW to CLOSED - WONTFIX
In pulp-dev mailing list discusison on this thread we are going to use PyPI and existing container registries to distribute Pulp. OSCI has deprovisioned this machine, and I removed its entry from the infra wiki.
- Sprint/Milestone set to 3.0.0
Also available in: Atom
PDF
.