Issue #9078
closedAn internal server error is raised when querying content synced via the on_demand policy
Description
Steps to reproduce the behaviour:
-
Install pulp-manifest by issuing
python3 setup.py install
from the directory/home/vagrant/devel/pulp_file/pulp_manifest
. -
Download a random repository that contains nested directories:
cd ~
mkdir test
wget --recursive --no-parent --no-host-directories --directory-prefix /home/vagrant/test https://fixtures.pulpproject.org/rpm-alt-layout/
- Follow these steps:
cd ~/test/rpm-alt-layout/
pulp-manifest .
pulp file repository create --name test
pulp file remote create --name test --url file:///home/vagrant/test/rpm-alt-layout/PULP_MANIFEST --policy on_demand
pulp file repository sync --name test --remote test
pulp file distribution create --repository "test" --name "dist-test" --base-path "dist-test-base"
http pulp3-source-fedora33.localhost.example.com/pulp/content/dist-test-base/
Traceback:
File "/usr/local/lib/pulp/lib64/python3.9/site-packages/aiohttp/web_protocol.py", line 422, in _handle_request
resp = await self._request_handler(request)
File "/usr/local/lib/pulp/lib64/python3.9/site-packages/aiohttp/web_app.py", line 499, in _handle
resp = await handler(request)
File "/home/vagrant/devel/pulpcore/pulpcore/cache/cache.py", line 169, in cached_function
response = await self.make_entry(key, bk, func, args, self.default_expires_ttl)
File "/home/vagrant/devel/pulpcore/pulpcore/cache/cache.py", line 198, in make_entry
response = await handler(*args)
File "/home/vagrant/devel/pulpcore/pulpcore/content/handler.py", line 212, in stream_content
return await self._match_and_stream(path, request)
File "/home/vagrant/devel/pulpcore/pulpcore/content/handler.py", line 602, in _match_and_stream
return await self._stream_content_artifact(
File "/home/vagrant/devel/pulpcore/pulpcore/content/handler.py", line 670, in _stream_content_artifact
response = await self._stream_remote_artifact(request, response, remote_artifact)
File "/home/vagrant/devel/pulpcore/pulpcore/content/handler.py", line 876, in _stream_remote_artifact
download_result = await downloader.run()
File "/home/vagrant/devel/pulpcore/pulpcore/download/base.py", line 243, in run
return await self._run(extra_data=extra_data)
File "/home/vagrant/devel/pulpcore/pulpcore/download/file.py", line 58, in _run
await self.handle_data(chunk)
File "/home/vagrant/devel/pulpcore/pulpcore/content/handler.py", line 861, in handle_data
await response.write(data)
File "/usr/local/lib/pulp/lib64/python3.9/site-packages/aiohttp/web_response.py", line 468, in write
raise RuntimeError("Cannot call write() before prepare()")
RuntimeError: Cannot call write() before prepare()
127.0.0.1 [13/Jul/2021:14:43:08 +0000] "GET /pulp/content/dist-test-base/ HTTP/1.0" 500 231 "-" "HTTPie/2.4.0"
Updated by dkliban@redhat.com about 2 years ago
- Priority changed from Normal to High
- Triaged changed from No to Yes
- Sprint set to Sprint 100
Updated by pulpbot about 2 years ago
- Status changed from ASSIGNED to POST
Updated by dalley about 2 years ago
I somewhat question if we should obey the download policy for file:// repos. There's arguments both ways I think, and I'm not sure if it warrants changing the behavior now if we currently support it, but I'm not sure what the use case for it (on-demand local "downloads") would be?
Updated by lmjachky about 2 years ago
I somewhat question if we should obey the download policy for file:// repos. There's arguments both ways I think, and I'm not sure if it warrants changing the behavior now if we currently support it, but I'm not sure what the use case for it (on-demand local "downloads") would be?
I also do not think that there might be either real use-cases or benefits for on-demand downloading for file://
repositories. Exceptionally, I could imagine a scenario where remote NFS directories are mounted to local FS. Downloading/caching remote files locally to MEDIA_ROOT
would probably resemble the process of the standard on-demand downloading of http://
repositories. Is not this a valid use case?
Updated by dkliban@redhat.com about 2 years ago
We should not try to fix this issue at this time. Any changes we introduce to the downloaders API will most likely introduce some kind of a regression. The scenario from the bug report is not realistic. If a user opens this back up, we can look into it then.
Updated by dalley about 2 years ago
- Status changed from POST to CLOSED - WONTFIX
- Sprint/Milestone deleted (
3.15.0) - Sprint deleted (
Sprint 101)