Project

Profile

Help

Issue #9078

closed

An internal server error is raised when querying content synced via the on_demand policy

Added by lmjachky almost 3 years ago. Updated almost 3 years ago.

Status:
CLOSED - WONTFIX
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Steps to reproduce the behaviour:

  1. Install pulp-manifest by issuing python3 setup.py install from the directory /home/vagrant/devel/pulp_file/pulp_manifest.

  2. Download a random repository that contains nested directories:

cd ~

mkdir test

wget --recursive --no-parent --no-host-directories --directory-prefix /home/vagrant/test https://fixtures.pulpproject.org/rpm-alt-layout/
  1. Follow these steps:
cd ~/test/rpm-alt-layout/
pulp-manifest .

pulp file repository create --name test

pulp file remote create --name test --url file:///home/vagrant/test/rpm-alt-layout/PULP_MANIFEST --policy on_demand

pulp file repository sync --name test --remote test

pulp file distribution create --repository "test" --name "dist-test" --base-path "dist-test-base"

http pulp3-source-fedora33.localhost.example.com/pulp/content/dist-test-base/

Traceback:

  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/aiohttp/web_protocol.py", line 422, in _handle_request
    resp = await self._request_handler(request)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/aiohttp/web_app.py", line 499, in _handle
    resp = await handler(request)
  File "/home/vagrant/devel/pulpcore/pulpcore/cache/cache.py", line 169, in cached_function
    response = await self.make_entry(key, bk, func, args, self.default_expires_ttl)
  File "/home/vagrant/devel/pulpcore/pulpcore/cache/cache.py", line 198, in make_entry
    response = await handler(*args)
  File "/home/vagrant/devel/pulpcore/pulpcore/content/handler.py", line 212, in stream_content
    return await self._match_and_stream(path, request)
  File "/home/vagrant/devel/pulpcore/pulpcore/content/handler.py", line 602, in _match_and_stream
    return await self._stream_content_artifact(
  File "/home/vagrant/devel/pulpcore/pulpcore/content/handler.py", line 670, in _stream_content_artifact
    response = await self._stream_remote_artifact(request, response, remote_artifact)
  File "/home/vagrant/devel/pulpcore/pulpcore/content/handler.py", line 876, in _stream_remote_artifact
    download_result = await downloader.run()
  File "/home/vagrant/devel/pulpcore/pulpcore/download/base.py", line 243, in run 
    return await self._run(extra_data=extra_data)
  File "/home/vagrant/devel/pulpcore/pulpcore/download/file.py", line 58, in _run
    await self.handle_data(chunk)
  File "/home/vagrant/devel/pulpcore/pulpcore/content/handler.py", line 861, in handle_data
    await response.write(data)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/aiohttp/web_response.py", line 468, in write
    raise RuntimeError("Cannot call write() before prepare()")
RuntimeError: Cannot call write() before prepare()
127.0.0.1 [13/Jul/2021:14:43:08 +0000] "GET /pulp/content/dist-test-base/ HTTP/1.0" 500 231 "-" "HTTPie/2.4.0"
Actions #1

Updated by dkliban@redhat.com almost 3 years ago

  • Priority changed from Normal to High
  • Triaged changed from No to Yes
  • Sprint set to Sprint 100
Actions #2

Updated by gerrod almost 3 years ago

  • Assignee set to gerrod
Actions #3

Updated by dalley almost 3 years ago

  • Status changed from NEW to ASSIGNED
Actions #4

Updated by gerrod almost 3 years ago

  • Sprint/Milestone set to 3.15.0
Actions #5

Updated by pulpbot almost 3 years ago

  • Status changed from ASSIGNED to POST
Actions #6

Updated by dalley almost 3 years ago

I somewhat question if we should obey the download policy for file:// repos. There's arguments both ways I think, and I'm not sure if it warrants changing the behavior now if we currently support it, but I'm not sure what the use case for it (on-demand local "downloads") would be?

Actions #7

Updated by lmjachky almost 3 years ago

I somewhat question if we should obey the download policy for file:// repos. There's arguments both ways I think, and I'm not sure if it warrants changing the behavior now if we currently support it, but I'm not sure what the use case for it (on-demand local "downloads") would be?

I also do not think that there might be either real use-cases or benefits for on-demand downloading for file:// repositories. Exceptionally, I could imagine a scenario where remote NFS directories are mounted to local FS. Downloading/caching remote files locally to MEDIA_ROOT would probably resemble the process of the standard on-demand downloading of http:// repositories. Is not this a valid use case?

Actions #8

Updated by rchan almost 3 years ago

  • Sprint changed from Sprint 100 to Sprint 101
Actions #9

Updated by dkliban@redhat.com almost 3 years ago

We should not try to fix this issue at this time. Any changes we introduce to the downloaders API will most likely introduce some kind of a regression. The scenario from the bug report is not realistic. If a user opens this back up, we can look into it then.

Actions #10

Updated by dalley almost 3 years ago

  • Status changed from POST to CLOSED - WONTFIX
  • Sprint/Milestone deleted (3.15.0)
  • Sprint deleted (Sprint 101)

Also available in: Atom PDF