Issue #8670
closed
URL escaping in yarl alters escaped characters
Description
After signing the URL when using the S3 backend (or presumably Azure), the call to generate the HTTP redirect re-parses the URL returned, sometimes resulting in characters being encoded as HTML entities or previously-encoded entities being decoded. While S3 is ok with ignoring an altered query string, CloudFront expects the entire URL to match what was signed exactly. This wasn't an issue before I patched django-storages to actually include the parameters passed in to the artifact_file.storage.url() call, but once that patch was applied, query strings were sent back with the URL for content-disposition.
It appears that providing a Yarl URL object to the raise HTTPFound() prevents re-parsing, resulting in the query string being left alone. If the object is created using URL() with encoded=True, that bypasses the reparsing in Yarl, and everyone's much happier. It's also verified to not break regular S3 operation.
Updated by pulpbot over 3 years ago
- Status changed from NEW to POST
Updated by fao89 over 3 years ago
- Triaged changed from No to Yes
- Sprint set to Sprint 95
Added by dannysauer over 3 years ago
Updated by dannysauer over 3 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulpcore|6066e5b11d8864cddb91164bf56f561230d33aed.
Updated by pulpbot over 3 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
.
Stabilize URLs before generating redirects
The aio URL object mangles query strings when they're passed in. Specifically, it unescapes http entities which are escaped but which don't need escaped. This is normally ok, but in the case of signed URLs, changing anything about the URL invalidates the signature. By pre-creating a yarl.URL object before raising the redirect exception, the parsing doesn't happen in aiohttp. Passing in the "encoding=True" stops the reparsing when the object is created.
fixes #8670 https://pulp.plan.io/issues/8670