Project

Profile

Help

Story #9304

closed

As a user, I can configure https proxy using https://

Added by ttereshc over 2 years ago. Updated over 2 years ago.

Status:
CLOSED - NOTABUG
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 106
Quarter:

Description

If one configures https proxy, sync task fails with the following error.

pulp_tasks:
- pulp_href: "/pulp/api/v3/tasks/c3aadbfe-2b3d-449d-8e65-9666951476d2/"
  pulp_created: '2021-08-16T11:20:14.160+00:00'
  state: failed
  name: pulp_rpm.app.tasks.synchronizing.synchronize
  logging_cid: 0f189218-3ac5-4a3f-b183-0d6b916aa4f2
  started_at: '2021-08-16T11:20:14.249+00:00'
  finished_at: '2021-08-16T11:20:14.374+00:00'
  error:
    traceback: |2
        File "/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpcore_worker.py", line 272, in _perform_task
          result = func(*args, **kwargs)
        File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 363, in synchronize
          remote_url = fetch_remote_url(remote)
        File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 242, in fetch_remote_url
          get_repomd_file(remote, normalized_remote_url)
        File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 200, in get_repomd_file
          return downloader.fetch()
        File "/usr/lib/python3.6/site-packages/pulpcore/download/base.py", line 176, in fetch
          return done.pop().result()
        File "/usr/lib/python3.6/site-packages/pulpcore/download/http.py", line 258, in run
          return await download_wrapper()
        File "/usr/lib/python3.6/site-packages/backoff/_async.py", line 133, in retry
          ret = await target(*args, **kwargs)
        File "/usr/lib/python3.6/site-packages/pulpcore/download/http.py", line 256, in download_wrapper
          return await self._run(extra_data=extra_data)
        File "/usr/lib/python3.6/site-packages/pulp_rpm/app/downloaders.py", line 92, in _run
          url, proxy=self.proxy, proxy_auth=self.proxy_auth, auth=self.auth
        File "/usr/lib64/python3.6/site-packages/aiohttp/client.py", line 1117, in __aenter__
          self._resp = await self._coro
        File "/usr/lib64/python3.6/site-packages/aiohttp/client.py", line 513, in _request
          traces=traces,
        File "/usr/lib64/python3.6/site-packages/aiohttp/client_reqrep.py", line 311, in __init__
          self.update_proxy(proxy, proxy_auth, proxy_headers)
        File "/usr/lib64/python3.6/site-packages/aiohttp/client_reqrep.py", line 551, in update_proxy
          raise ValueError("Only http proxies are supported")
    description: Only http proxies are supported
  worker: "/pulp/api/v3/workers/204569e0-9f35-48ca-9e22-5cbfd8828dcc/"
  child_tasks: []
  progress_reports: []
  created_resources: []
  reserved_resources_record:
  - "/pulp/api/v3/repositories/rpm/rpm/abc9dd0c-d198-433c-af59-47b40de82db6/"
  - "/pulp/api/v3/remotes/rpm/rpm/d86a3ede-74f5-4f11-b979-34e9f0e6dd27/"
create_version: true
task_groups: []
poll_attempts:
  total: 1
  failed: 1

aiohttp doesn't support https proxies, specified with https://.
https://docs.aiohttp.org/en/stable/client_advanced.html#proxy-support

"aiohttp supports plain HTTP proxies and HTTP proxies that can be upgraded to HTTPS via the HTTP CONNECT method. aiohttp does not support proxies that must be connected to via https://."
HTTP CONNECT method https://www.ietf.org/rfc/rfc2817.txt

Pulp 2 supported it, so the lack of this feature is seen as a regression.


Related issues

Related to Pulp - Task #7921: Improve documentation on types of proxies supportedCLOSED - DUPLICATE

Actions
Actions #1

Updated by ttereshc over 2 years ago

It seems to be a low priority on the aiohttp list to complete https://github.com/aio-libs/aiohttp/issues/845#issuecomment-313141691. But reading the whole thread, it seems that they are not against a contribution, and the main problem is a test setup.

Actions #2

Updated by bmbouter over 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to bmbouter
Actions #3

Updated by bmbouter over 2 years ago

I setup squid on a dev box and configured it to also support https_port like this:

  1. Install with sudo dnf install squid -y
  2. Make certificates with these commands (and use your hostname):
sudo openssl genrsa -out /etc/ssl/key.pem 4096
sudo openssl req -new -x509 -key /etc/ssl/key.pem -out /etc/ssl/cert.pem -days 1826
  1. Add this line to the top of your /etc/squid/squid.conf: https_port 3130 tls-cert=/etc/ssl/cert.pem tls-key=/etc/ssl/key.pem

Then do some testing with curl.

Make a no proxy request: curl -v http://example.com Make a request going through the http-based proxy: curl -v --proxy https://localhost:3128 http://example.com Make a request going through the https proxy. curl -v --proxy-insecure --proxy https://localhost:3130 http://example.com

You have to use the --proxy-insecure because the certificate squid is using is self-signed. I couldn't easily get a request to https://example.com working, but that's not important for this bugfix because the proxy itself is TLS enabled.

Then have pulp_file sync through the http based proxy with:

#!/bin/bash

set -ev

repo_name="repo$RANDOM"
remote_name="remote$RANDOM"
distribution_name="distribution$RANDOM"
base_path="$RANDOM"
url="http://fixtures.pulpproject.org/file/PULP_MANIFEST"
proxy_url="http://pulp3-source-fedora33.fedora.example.com:3128/"

pulp file remote create --name $remote_name --url $url --proxy-url $proxy_url
pulp file repository create --name $repo_name --remote $remote_name
pulp file repository sync --name $repo_name
pulp file publication create --repository $repo_name
pulp file distribution create --base-path $base_path --repository $repo_name --name $distribution_name

echo "Created, synced, published repo $repo_name at distribution base path '$base_path'"

Then reproduce the issue by having it sync through the HTTPS proxy:

#!/bin/bash

set -ev

repo_name="repo$RANDOM"
remote_name="remote$RANDOM"
distribution_name="distribution$RANDOM"
base_path="$RANDOM"
url="http://fixtures.pulpproject.org/file/PULP_MANIFEST"
proxy_url="https://pulp3-source-fedora33.fedora.example.com:3130/"

pulp file remote create --name $remote_name --url $url --proxy-url $proxy_url
pulp file repository create --name $repo_name --remote $remote_name
pulp file repository sync --name $repo_name
pulp file publication create --repository $repo_name
pulp file distribution create --base-path $base_path --repository $repo_name --name $distribution_name

echo "Created, synced, published repo $repo_name at distribution base path '$base_path'"

It should show you: Error: Task /pulp/api/v3/tasks/ba5ecbf2-f7a8-48db-89e3-2c50af8fb8a6/ failed: 'Only http proxies are supported'

Actions #4

Updated by bmbouter over 2 years ago

Here's an update of the key points I've learned so far.

  • I've been mostly discussing this on this aiohttp PR https://github.com/aio-libs/aiohttp/pull/5992. Lots of details there
  • aiohttp doesn't do TLS in TLS correctly because Python doesn't do it correctly https://bugs.python.org/issue37179. This is close to being resolved and will be fixed in either 3.10 (maybe?) or 3.11 (likely).
  • Until this bug is fixed aiohttp and other web frameworks would have to carry a workaround. Other ones like httpx do carry this.

So there are two options I see:

  1. Continue to try to fix aiohttp's TLS handshake workaround code
  2. Accept this isn't working until aiohttp fixes it properly after Python fixes asyncio
Actions #5

Updated by rchan over 2 years ago

  • Sprint set to Sprint 105
Actions #6

Updated by bmbouter over 2 years ago

Here's another way to setup the proxy using a python-based proxy proxy.py

# Make your keys
sudo openssl genrsa -out /etc/ssl/key.pem 4096
sudo openssl req -new -x509 -key /etc/ssl/key.pem -out /etc/ssl/cert.pem -days 1826

# Install proxy and run it
pip install proxy
proxy --hostname 0.0.0.0 --cert-file /etc/ssl/cert.pem --key-file /etc/ssl/key.pem

# Test it with curl in another tty
curl -v --proxy-insecure --proxy https://127.0.0.1:8899 https://example.com
Actions #7

Updated by dalley over 2 years ago

  • Related to Task #7921: Improve documentation on types of proxies supported added
Actions #8

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 105 to Sprint 106
Actions #9

Updated by dalley over 2 years ago

  • Priority changed from Normal to High
Actions #10

Updated by bmbouter over 2 years ago

  • Status changed from ASSIGNED to CLOSED - NOTABUG

After working with the aiohttp maintainers, this can only be fixed in aiohttp directly (PRs links below). There isn't a code change that Pulp can make to resolve it. As such, I'm closing as NOTABUG given that the bug isn't actually in Pulp.

https://github.com/aio-libs/aiohttp/pull/5992/files https://github.com/aio-libs/aiohttp/pull/6002/files

Also available in: Atom PDF