Issue #5414
closed
Story #5132: [Epic] As a user, I can consume Pulp 3 from OperatorHub.io
HTTP requests to containers hang over IPv6
Description
Our individual Pulp containers providing Kubernetes external services (pulp-api & pulp-content) listen on only IPv4.
However, Kubernetes will forward both IPv4 and IPv6 often. Such as on Fedora 30 with k3s. An IPv6 connection will hang.
And many clients will prefer IPv6 over IPv4, including a Fedora 30 host connecting to "localhost".
This is observed during the development of the demo pulp install script (#5375):
https://github.com/mikedep333/Vagrantfiles/blob/master/pulp-insta-demo.sh
Related issues
Updated by mdepaulo@redhat.com over 5 years ago
Upon further investigation:
1. k3s is listening on IPv6, and due to the dual-stack behavior, this implies IPv4 listen as well. (Only IPv6 shows up in `netstat`).
2. Thus k3s is attempting to forwarding both IPv6 and IPv4 to the container, and the IPv6 hangs because nothing is listening on it in it.
3. HTTP clients can be configured to prefer IPv4 over IPv6 in /etc/gai.conf for when a host resolves to both (localhost), or have IPv6 disabled entirely (like in the Ubuntu 16.04 Vagrant box).
4. We should resolve this by having the containers listen on both IPv4 and IPv6. nginx & ansible-pulp should be updated later. It is TBD whether we will do separate binds, or IPv6 (with IPv4 implicit) "dual-stack" bind. The dual-stack bind will depend on /proc/sys/net/ipv6/bindv6only being set to 0, but accommodating that with kubernetes may unnecessary work.
Added by Mike DePaulo over 5 years ago
Updated by mdepaulo@redhat.com over 5 years ago
- Blocks Task #5375: As a user, I can use a single script to install k3s and launch pulp-operator added
Updated by Anonymous over 5 years ago
- Status changed from NEW to MODIFIED
Applied in changeset pulpcore|4e8765cb62e4ec5d7d8b6f5b9641bfbc49221d71.
Updated by mdepaulo@redhat.com over 5 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
It's been merged, and I did my own testing (connecting to localhost4 and localhost6 on a Fedora 30 k3s host as part of #5375's script & Vagrantfile.)
Closing.
Updated by bmbouter about 4 years ago
- Category set to Operator - Moved to Github Issues
.
Problem: HTTP requests to containers hang over IPv6
Solution: Bind to IPv6, and thus impliclty IPv4. This implicit listen can reliably happen in containers because of network namespaces.
Also, standardize container script syntax.
Fixes: #5414 HTTP requests to containers hang over IPv6 https://pulp.plan.io/issues/5414
re: #5375 As a user, I can use a single script to install k3s and launch pulp-operator https://pulp.plan.io/issues/5375
[noissue]