Project

Profile

Help

Task #2325

Distribute Pulp with Pulp

Added by semyers about 3 years ago. Updated 6 months ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

0%

Platform Release:
Blocks Release:
Backwards Incompatible:
No
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

We're planning to do out package builds using fedora's copr infrastructure from Pulp 3. We've identified two needs that need to be met for this to be viable:
  1. Old releases need to be archived, so that downstream folks like katello can pull specific release versions. This is also just a good thing to do. Currently, we only keep the latest release of a given x.y stream, and earlier releases can't be easily found online.
  2. Releases need to happen atomically. COPR supports this, but offers limited control over the exact moment a repository's metadata is regenerated.

Pulp meets both of these needs, and should be the tool we use to distribute Pulp. :)

This pulp instance will need to be secure and the following things should be ensured:

  1. Pulp's REST API should be ran on a non default port
  2. Pulp's content serving API should be run on ports 80 and 443
  3. mongo set up with authentication and listen locally (through sockets)
  4. message brokers also set up with authentication and configured to only listen locally
  5. The RHEL7 hardening guide is followed [0]

[0] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/pdf/Security_Guide/Red_Hat_Enterprise_Linux-7-Security_Guide-en-US.pdf


Checklist


Related issues

Blocks Pulp - Task #2145: Ansible playbooks need to pull from Pulp NEW Actions

History

#1 Updated by semyers about 3 years ago

  • Checklist item Figure out where to host Pulp added
  • Checklist item Come up with an upgrade policy (i.e. do we pin it to the latest release?) added
  • Checklist item Come up with a pulp release workflow that supports archiving releases and atomic releases added
  • Checklist item Investigate High-Availability options for content distribution added

I added some checklist items. One, "Come up with an upgrade policy" is a little ambiguous, but I'm not sure how to phrase it more accurately. It refers to the upgrade policy of the Pulp installation being used to distribute Pulp, and whether we want to always keep it up to date with the latest release, or only update it as-needed. It doesn't refer to how we update the versions of pulp being distributed, that's a separate checklist item.

#2 Updated by dkliban@redhat.com almost 3 years ago

  • Blocks Issue #2414: `ImproperlyConfigured` exception in for detail list view endpoint added

#3 Updated by dkliban@redhat.com almost 3 years ago

  • Blocks deleted (Issue #2414: `ImproperlyConfigured` exception in for detail list view endpoint)

#4 Updated by dkliban@redhat.com almost 3 years ago

  • Blocks Task #2145: Ansible playbooks need to pull from Pulp added

#7 Updated by bmbouter about 2 years ago

I think OS1 has been shutdown or nearly shutdown at this point. Open Source and Standards (OSAS) has offered some hosting for upstream Pulp's needs, like maybe hosting this environment. I'm emailing them to get some more details about hosting a Pulp inside the OSAS infrastructure.

#8 Updated by mhrivnak about 2 years ago

  • Sprint/Milestone set to 45

This is in-progress and will get groomed ASAP.

#9 Updated by bmbouter about 2 years ago

We will use an OSAS environment that they provide an EL7 box that we can manage and deploy Pulp onto it. I believe @pcreech or @bizhang have access to that.

#10 Updated by bizhang about 2 years ago

  • Description updated (diff)

#11 Updated by bmbouter about 2 years ago

  • Description updated (diff)

I added that the broker should be configured to only listen locally.

I don't understand the middle bullet point.

Also will this be the url that users consumer directly from or are we distributing the bits elsewhere somehow?

#12 Updated by bizhang about 2 years ago

  • Description updated (diff)

@bmbouter, since this is public facing people should be using it to consume our bits. I don't think we'll be distributing out the pulp3 bits to fedorapeople, but can be convinced otherwise.

As far as SNI goes @pcreech mentioned hosting multiple https sites from the same IP, but I think that can (and should) be taken off, since this story only deals with getting pulp up and running.

#13 Updated by mhrivnak about 2 years ago

Can I assume that "Pulp should be ran on a non default port" is referring to the REST API service?

What is the goal of running it on a non-default port?

#14 Updated by pcreech about 2 years ago

The intent behind moving the REST api to a different port is that doing such would allow us to have more access control at the firewall level. This machine has a public IP, and therefore anything we set to listen will listen on the public IP. Having our web service listen at the same ip:port endpoint as our content will allow anyone coming in to attempt to access our rest api as well.

Moving to a separate port will allow us to implement stricter access controls on the rest api that we interface with, therefore helping reduce our attack surface.

The other option would be to have the rest api listen to local only, therefore necessitating that all interaction with pulp happens solely on that machine.

#15 Updated by bmbouter about 2 years ago

  • Description updated (diff)

I clarified in the description about the ports saying that the rest API will be on one non-standard port and that content will be served via 80 and 443. Is that right?

#16 Updated by jortel@redhat.com about 2 years ago

  • Sprint/Milestone changed from 45 to 46

#17 Updated by bizhang about 2 years ago

@bmbouter that sounds correct to me.

#18 Updated by bmbouter about 2 years ago

  • Groomed changed from No to Yes

Thanks @bizhang!

#19 Updated by mhrivnak almost 2 years ago

  • Sprint/Milestone changed from 46 to 47

#20 Updated by bizhang almost 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to bizhang

#21 Updated by rchan almost 2 years ago

  • Sprint/Milestone changed from 47 to 48

#22 Updated by rchan almost 2 years ago

  • Sprint/Milestone changed from 48 to 52

#23 Updated by rchan almost 2 years ago

  • Sprint/Milestone changed from 52 to 53

#24 Updated by jortel@redhat.com over 1 year ago

  • Sprint/Milestone changed from 53 to 54

#25 Updated by rchan over 1 year ago

  • Sprint/Milestone changed from 54 to 56

#26 Updated by bmbouter over 1 year ago

  • Sprint set to Sprint 33

#27 Updated by bmbouter over 1 year ago

  • Sprint/Milestone deleted (56)

#28 Updated by jortel@redhat.com over 1 year ago

  • Sprint changed from Sprint 33 to Sprint 34

#29 Updated by bmbouter over 1 year ago

  • Sprint deleted (Sprint 34)

Removing from sprint through email list discussion: https://www.redhat.com/archives/pulp-dev/2018-March/msg00080.html

#30 Updated by bizhang about 1 year ago

  • Assignee deleted (bizhang)

#31 Updated by amacdona@redhat.com 11 months ago

  • Status changed from ASSIGNED to NEW

#32 Updated by bmbouter 11 months ago

I think we're moving away from self-distribution and towards container distribution in registries we don't operate and also PyPI delivery. Does doing this effort still make sense?

Note that the infra wiki shows this initative also: https://pulp.plan.io/projects/pulp/wiki/Infrastructure_&_Hosting#Distribute-Pulp-with-Pulp

Also AIUI the OSCI group has provisioned a machine in the community data center for this purpose. Since we're not using it, and if we decide to not go forward with it, we should ask them to deprovisioning it.

What do you all think?

#33 Updated by bmbouter 10 months ago

  • Status changed from NEW to CLOSED - WONTFIX

In pulp-dev mailing list discusison on this thread we are going to use PyPI and existing container registries to distribute Pulp. OSCI has deprovisioned this machine, and I removed its entry from the infra wiki.

#34 Updated by daviddavis 6 months ago

  • Sprint/Milestone set to 3.0

#35 Updated by bmbouter 6 months ago

  • Tags deleted (Pulp 3)

Please register to edit this issue

Also available in: Atom PDF