Issue #2587
closedon_demand downloads aren't being cached by Squid
Description
The on_demand download policy lets Pulp defer the download of a content unit until it's requested. When a content unit is requested for the first time, Pulp downloads the file and streams it to the requesting client as the content becomes available. When the same content unit is requested in the future, Squid serves the file directly. For whatever reason, Squid is not serving cached copies of content units.
The tests cases that demonstrate this issue are:
pulp_smash.tests.rpm.api_v2.test_download_policies.OnDemandTestCase.test_rpm_cache_control_header
pulp_smash.tests.rpm.api_v2.test_download_policies.OnDemandTestCase.test_same_rpm_cache_header
pulp_smash.tests.rpm.api_v2.test_download_policies.SwitchPoliciesTestCase.test_background_to_on_demand
pulp_smash.tests.rpm.api_v2.test_download_policies.SwitchPoliciesTestCase.test_immediate_to_on_demand
Attached is the output produced by Pulp Smash when these tests are executed (pending this pr). The test failures clearly describe cache misses occurring. I can reproduce this issue on at least Fedora 24 and Fedora 25 with Pulp 2.12 and Pulp 2.13. Manual investigation of all these systems shows that:
- Squid is running.
- All services are OK, according to
systemctl status
. - No error messages are logged, according to
journalctl
.
Here's some of the packages installed on my Fedora 24 system with Pulp 2.12 installed:
[root@fedora-24-pulp-2-12 ~]# rpm -qa | grep squid | sort
squid-3.5.20-1.fc24.x86_64
[root@fedora-24-pulp-2-12 ~]# rpm -qa | grep -i pulp | sort
pulp-admin-client-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
pulp-docker-admin-extensions-2.3.1-0.1.alpha.git.1.ad83c84.fc24.noarch
pulp-docker-plugins-2.3.1-0.1.alpha.git.1.ad83c84.fc24.noarch
pulp-ostree-admin-extensions-1.2.1-0.1.alpha.git.22.19760f7.fc24.noarch
pulp-ostree-plugins-1.2.1-0.1.alpha.git.22.19760f7.fc24.noarch
pulp-puppet-admin-extensions-2.12.1-0.1.alpha.git.1.d407541.fc24.noarch
pulp-puppet-plugins-2.12.1-0.1.alpha.git.1.d407541.fc24.noarch
pulp-python-admin-extensions-2.0.1-0.1.alpha.git.2.be7d11e.fc24.noarch
pulp-python-plugins-2.0.1-0.1.alpha.git.2.be7d11e.fc24.noarch
pulp-rpm-admin-extensions-2.12.1-0.1.alpha.git.23.f90340d.fc24.noarch
pulp-rpm-plugins-2.12.1-0.1.alpha.git.23.f90340d.fc24.noarch
pulp-selinux-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
pulp-server-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
python-kombu-3.0.33-6.pulp.fc24.noarch
python-pulp-bindings-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
python-pulp-client-lib-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
python-pulp-common-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
python-pulp-docker-common-2.3.1-0.1.alpha.git.1.ad83c84.fc24.noarch
python-pulp-oid_validation-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
python-pulp-ostree-common-1.2.1-0.1.alpha.git.22.19760f7.fc24.noarch
python-pulp-puppet-common-2.12.1-0.1.alpha.git.1.d407541.fc24.noarch
python-pulp-python-common-2.0.1-0.1.alpha.git.2.be7d11e.fc24.noarch
python-pulp-repoauth-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
python-pulp-rpm-common-2.12.1-0.1.alpha.git.23.f90340d.fc24.noarch
python-pulp-streamer-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
Files
Related issues
Updated by semyers almost 8 years ago
The version numbers are for a different build than the 2.12.1 beta, but these failures show up in the builds done against the 2.12.1 beta and do apply to that release.
Updated by mhrivnak almost 8 years ago
Is this behavior the same in 2.12.0? Or does it appear to be a regression only in 2.12.1?
Updated by mhrivnak almost 8 years ago
Seeing the system log and apache request logs from during this test case's run would be helpful.
Updated by mhrivnak almost 8 years ago
Wildly speculating, here is our top suspect for introducing the regression:
https://github.com/pulp/pulp/pull/2935
but this is also worth considering:
Updated by mhrivnak almost 8 years ago
- Status changed from NEW to POST
- Assignee set to jortel@redhat.com
- Sprint/Milestone set to 33
Updated by mhrivnak almost 8 years ago
@jortel's PR: https://github.com/pulp/pulp/pull/2953
Added by jortel@redhat.com almost 8 years ago
Added by jortel@redhat.com almost 8 years ago
Revision 0ced6570 | View on GitHub
Fix streamer http header management. closes #2587
Updated by jortel@redhat.com almost 8 years ago
- Project changed from RPM Support to Pulp
Updated by jortel@redhat.com almost 8 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulp|0ced65704875f9a4b65f1b2d8334f4533414fc43.
Updated by Ichimonji10 almost 8 years ago
This issue is fixed in the current 2.12.1 and 2.13.0 nightlies.
Updated by bizhang almost 8 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
Updated by dkliban@redhat.com about 7 years ago
- Has duplicate Issue #3140: pulp_streamer doesnt set HTTP response headers, causing squid cache is not used added
Fix streamer http header management. closes #2587