Issue #1977
closedbasic auth in URL fails when using authenticated proxy
Description
If you perform a sync where all of these are true:
- the proxy requires auth
- the repo requires auth
- repo auth credentials are specified in the URL
You will see this error message:
RPM1004: Error retrieving metadata: Not found
If any of those three conditions are not true, the error will not occur.
Assuming you have an authenticated proxy configured in /etc/pulp/server/plugins.conf.d/yum_importer.json, this will work because credentials are not in the URL:
pulp-admin rpm repo create --repo-id=rplevka --feed=https://rplevka.fedorapeople.org/fakerepo01/ --basicauth-user=admin --basicauth-pass=changeme
pulp-admin rpm repo sync run --repo-id=rplevka
And this will fail:
pulp-admin rpm repo create --repo-id=rplevka --feed=https://admin:changeme@rplevka.fedorapeople.org/fakerepo01/
pulp-admin rpm repo sync run --repo-id=rplevka
Pulp previously did not have explicit options for setting the basic auth credentials, so users' only option was to put them in the URL. Even though that's deprecated per RFC and is generally not advisable, we should continue supporting those users.
This bug likely affects other plugins as well. I haven't tried reproducing, but at least one user reported the same problem doing a puppet sync.
Related issues
Updated by pcreech over 8 years ago
- Project changed from RPM Support to Nectar
- Version - Nectar set to 1.5.1
After investigation, I discovered that this particular combination has not been a viable pathway for quite some time (Doesn't work in pulp 2.6 or 2.7, introduced here [0] with python-nectar-1.1.5-1)
When you specify a proxy_password and proxy_username in the importer config, it invokes this code pathway here:
https://github.com/pulp/nectar/blob/master/nectar/downloaders/threaded.py#L412
Which replaces the authentication provider with the HTTPProxyAuth or HTTPBasicWithProxyAuth provider (for authenticated proxies). When this is done with requests, it breaks requests ability to authenticate the requested URL with HTTPBasicAuth when credentials are provided in the url (HTTPBasicAuth also intercepts the user:pass@host information)
With provided basic_auth_username and basic_auth_password credentials, it utilizes HTTPBasicWithProxyAuth instead, which is why the first example works.
There are two functioning workarounds, whch are:
- Use the first form provided in this issue
- Instead of providing proxy_username and proxy_password in the yum_importer.json file, provide the credentials in the proxy host i.e.
"proxy_host": "http://user:pass@host"
To fix the issue in the codebase, we should find a way to intercept credentials before
https://github.com/pulp/nectar/blob/python-nectar-1.5.1-1/nectar/downloaders/threaded.py#L369
Updated by amacdona@redhat.com over 8 years ago
- Sprint Candidate changed from No to Yes
Updated by amacdona@redhat.com over 8 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to amacdona@redhat.com
Updated by amacdona@redhat.com over 8 years ago
- Project changed from Nectar to Pulp
- Status changed from ASSIGNED to POST
I determined that this is not possible to fix in nectar. The reason is that at the time that a threaded downloader object is created, nectar is not aware of the urls, which are only passed in later as request objects. Instead, the proposed change moves the basic auth information to the importer config before creating the downloader.
Added by Austin Macdonald over 8 years ago
Added by Austin Macdonald over 8 years ago
Revision 206cf5fd | View on GitHub
Pull basic auth info out of feed url
Previous versions of Pulp allowed users to include their basic auth information in the url. This is obviously less ideal than passing this in the config, but in order to remain backwards compatibile, this removes the auth info from the url and adds it to the importer config. For importer configurations that were created before this change, we pass the configuration through the same cleaner function as we do during importer creation, so a temporary importer configuration includes the basic auth information.
closes #1977
Updated by Anonymous over 8 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulp|206cf5fd7bfd94cf994d528ac68019f143e92be8.
Updated by pthomas@redhat.com almost 8 years ago
- Status changed from 5 to 6
verified
pulp-admin rpm repo create --feed http://file.rdu.redhat.com/~cduryee/protected/ --basicauth-user testuser --basicauth-pass testpass --repo-id proxy1 --proxy-host="http://ginger.lab.eng.rdu2.redhat.com" --proxy-user="admin" --proxy-port 3128 --proxy-pass redhat
Successfully created repository [proxy1]
[root@cloud-qe-22 ~]# pulp-admin rpm repo sync run --repo-id proxy1
+----------------------------------------------------------------------+
Synchronizing Repository [proxy1]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[|]
... completed
Downloading repository content...
[==================================================] 100%
RPMs: 2/2 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[-]
... completed
Cleaning duplicate packages...
[-]
... completed
Task Succeeded
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[==================================================] 100%
2 of 2 items
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[-]
... completed
Publishing Comps file
[-]
... completed
Publishing Metadata.
[-]
... completed
Closing repo metadata
[-]
... completed
Generating sqlite files
... skipped
Generating HTML files
... skipped
Publishing files to web
[-]
... completed
Writing Listings File
[-]
... completed
Task Succeeded
Updated by pcreech almost 8 years ago
- Status changed from 6 to CLOSED - CURRENTRELEASE
Updated by mhrivnak almost 8 years ago
- Related to Issue #2520: credentials in feed URL are not url-unquoted added
Pull basic auth info out of feed url
Previous versions of Pulp allowed users to include their basic auth information in the url. This is obviously less ideal than passing this in the config, but in order to remain backwards compatibile, this removes the auth info from the url and adds it to the importer config. For importer configurations that were created before this change, we pass the configuration through the same cleaner function as we do during importer creation, so a temporary importer configuration includes the basic auth information.
closes #1977