Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #8865

closed

Range requests for on demand content return the full file (Kickstarts fail for on_demand repos)

Added by jsherril@redhat.com over 3 years ago. Updated about 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
bmbouter
Category:
-
Sprint/Milestone:
3.15.0
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 100
Quarter:

Description

When fetching an rpm from the content app and specifying the RANGE header, like so:

curl -k  http://foreman-nuc2.usersys.redhat.com/pulp/content/Demo/Library/custom/CentOS7/main/Packages/s/sg3_utils-1.37-19.el7.x86_64.rpm  -H "Range: bytes=1384-44339"   > foo.rpm

if the package is being lazily downloaded, the entire file is returned and not JUST the range:

$ ls -l foo.rpm -h
-rw-rw-r--. 1 jlsherri jlsherri 646K Jun  4 09:03 foo.rpm

Once the package is downloaded, it behaves as you'd expect:

$ curl -k  http://foreman-nuc2.usersys.redhat.com/pulp/content/Demo/Library/custom/CentOS7/main/Packages/s/sg3_utils-1.37-19.el7.x86_64.rpm  -H "Range: bytes=1384-44339"   > foo2.rpm 


$ ls -l foo2.rpm 
-rw-rw-r--. 1 jlsherri jlsherri 42956 Jun  4 09:04 foo2.rpm

For el7 (at least, probably more), this causes yum/anaconda to hang up the connection as soon as it gets the amount of requested data, which makes the content app really unhappy and leads to this error:

[2021-06-04 11:57:02 +0000] [27275] [ERROR] Error handling request
Traceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/aiohttp/web_protocol.py", line 422, in _handle_request
resp = await self._request_handler(request)
File "/usr/lib64/python3.6/site-packages/aiohttp/web_app.py", line 499, in _handle
resp = await handler(request)
File "/usr/lib/python3.6/site-packages/pulpcore/content/handler.py", line 138, in stream_content
return await self._match_and_stream(path, request)
File "/usr/lib/python3.6/site-packages/pulpcore/content/handler.py", line 387, in _match_and_stream
request, StreamResponse(headers=headers), ca
File "/usr/lib/python3.6/site-packages/pulpcore/content/handler.py", line 501, in _stream_content_artifact
response = await self._stream_remote_artifact(request, response, remote_artifact)
File "/usr/lib/python3.6/site-packages/pulpcore/content/handler.py", line 651, in _stream_remote_artifact
download_result = await downloader.run()
File "/usr/lib/python3.6/site-packages/pulpcore/download/base.py", line 227, in run
return await self._run(extra_data=extra_data)
File "/usr/lib/python3.6/site-packages/pulp_rpm/app/downloaders.py", line 90, in _run
to_return = await self._handle_response(response)
File "/usr/lib/python3.6/site-packages/pulpcore/download/http.py", line 189, in _handle_response
await self.handle_data(chunk)
File "/usr/lib/python3.6/site-packages/pulpcore/content/handler.py", line 636, in handle_data
await response.write(data)
File "/usr/lib64/python3.6/site-packages/aiohttp/web_response.py", line 470, in write
await self._payload_writer.write(data)
File "/usr/lib64/python3.6/site-packages/aiohttp/http_writer.py", line 107, in write
self._write(chunk)
File "/usr/lib64/python3.6/site-packages/aiohttp/http_writer.py", line 67, in _write
raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport
[04/Jun/2021:11:57:02 +0000] "GET /pulp/content/Demo/Library/custom/CentOS7/main/Packages/s/sg3_utils-1.37-19.el7.x86_64.rpm HTTP/1.1" 500 0 "-" "urlgrabber/3.10 yum/3.4.3"

and since anaconda receives the entire rpm instead of just the range it requested (the rpm header), it re-tries the request, and pulp continually just tries to return the entire file

Related issues

Related to Pulp - Backport #9057: Backport 8865 "incorrect responses to range requests for on_demand content" to 3.14.zCLOSED - CURRENTRELEASEdaviddavis

Actions
Actions
Send by e-mail Copy link

Also available in: Atom PDF