Project

Profile

Help

Issue #2587

closed

on_demand downloads aren't being cached by Squid

Added by Ichimonji10 about 7 years ago. Updated about 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
2.12.1
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 15
Quarter:

Description

The on_demand download policy lets Pulp defer the download of a content unit until it's requested. When a content unit is requested for the first time, Pulp downloads the file and streams it to the requesting client as the content becomes available. When the same content unit is requested in the future, Squid serves the file directly. For whatever reason, Squid is not serving cached copies of content units.

The tests cases that demonstrate this issue are:

  • pulp_smash.tests.rpm.api_v2.test_download_policies.OnDemandTestCase.test_rpm_cache_control_header
  • pulp_smash.tests.rpm.api_v2.test_download_policies.OnDemandTestCase.test_same_rpm_cache_header
  • pulp_smash.tests.rpm.api_v2.test_download_policies.SwitchPoliciesTestCase.test_background_to_on_demand
  • pulp_smash.tests.rpm.api_v2.test_download_policies.SwitchPoliciesTestCase.test_immediate_to_on_demand

Attached is the output produced by Pulp Smash when these tests are executed (pending this pr). The test failures clearly describe cache misses occurring. I can reproduce this issue on at least Fedora 24 and Fedora 25 with Pulp 2.12 and Pulp 2.13. Manual investigation of all these systems shows that:

  • Squid is running.
  • All services are OK, according to systemctl status.
  • No error messages are logged, according to journalctl.

Here's some of the packages installed on my Fedora 24 system with Pulp 2.12 installed:

[root@fedora-24-pulp-2-12 ~]# rpm -qa | grep squid | sort
squid-3.5.20-1.fc24.x86_64
[root@fedora-24-pulp-2-12 ~]# rpm -qa | grep -i pulp | sort
pulp-admin-client-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
pulp-docker-admin-extensions-2.3.1-0.1.alpha.git.1.ad83c84.fc24.noarch
pulp-docker-plugins-2.3.1-0.1.alpha.git.1.ad83c84.fc24.noarch
pulp-ostree-admin-extensions-1.2.1-0.1.alpha.git.22.19760f7.fc24.noarch
pulp-ostree-plugins-1.2.1-0.1.alpha.git.22.19760f7.fc24.noarch
pulp-puppet-admin-extensions-2.12.1-0.1.alpha.git.1.d407541.fc24.noarch
pulp-puppet-plugins-2.12.1-0.1.alpha.git.1.d407541.fc24.noarch
pulp-python-admin-extensions-2.0.1-0.1.alpha.git.2.be7d11e.fc24.noarch
pulp-python-plugins-2.0.1-0.1.alpha.git.2.be7d11e.fc24.noarch
pulp-rpm-admin-extensions-2.12.1-0.1.alpha.git.23.f90340d.fc24.noarch
pulp-rpm-plugins-2.12.1-0.1.alpha.git.23.f90340d.fc24.noarch
pulp-selinux-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
pulp-server-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
python-kombu-3.0.33-6.pulp.fc24.noarch
python-pulp-bindings-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
python-pulp-client-lib-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
python-pulp-common-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
python-pulp-docker-common-2.3.1-0.1.alpha.git.1.ad83c84.fc24.noarch
python-pulp-oid_validation-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
python-pulp-ostree-common-1.2.1-0.1.alpha.git.22.19760f7.fc24.noarch
python-pulp-puppet-common-2.12.1-0.1.alpha.git.1.d407541.fc24.noarch
python-pulp-python-common-2.0.1-0.1.alpha.git.2.be7d11e.fc24.noarch
python-pulp-repoauth-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch
python-pulp-rpm-common-2.12.1-0.1.alpha.git.23.f90340d.fc24.noarch
python-pulp-streamer-2.12.1-0.1.alpha.git.67.22eb720.fc24.noarch

Files

pulp smash output.txt (4.47 KB) pulp smash output.txt Ichimonji10, 02/14/2017 09:26 PM

Related issues

Has duplicate Pulp - Issue #3140: pulp_streamer doesnt set HTTP response headers, causing squid cache is not usedCLOSED - DUPLICATEdkliban@redhat.comActions
Actions #1

Updated by semyers about 7 years ago

The version numbers are for a different build than the 2.12.1 beta, but these failures show up in the builds done against the 2.12.1 beta and do apply to that release.

Actions #2

Updated by mhrivnak about 7 years ago

Is this behavior the same in 2.12.0? Or does it appear to be a regression only in 2.12.1?

Actions #3

Updated by mhrivnak about 7 years ago

Seeing the system log and apache request logs from during this test case's run would be helpful.

Actions #4

Updated by mhrivnak about 7 years ago

Wildly speculating, here is our top suspect for introducing the regression:

https://github.com/pulp/pulp/pull/2935

but this is also worth considering:

https://github.com/pulp/pulp/pull/2932

Actions #5

Updated by mhrivnak about 7 years ago

  • Status changed from NEW to POST
  • Assignee set to jortel@redhat.com
  • Sprint/Milestone set to 33

Added by jortel@redhat.com about 7 years ago

Revision 0ced6570 | View on GitHub

Fix streamer http header management. closes #2587

Added by jortel@redhat.com about 7 years ago

Revision 0ced6570 | View on GitHub

Fix streamer http header management. closes #2587

Actions #7

Updated by jortel@redhat.com about 7 years ago

  • Project changed from RPM Support to Pulp
Actions #8

Updated by jortel@redhat.com about 7 years ago

  • Status changed from POST to MODIFIED
Actions #9

Updated by bizhang about 7 years ago

  • Platform Release set to 2.12.1
Actions #10

Updated by bizhang about 7 years ago

  • Status changed from MODIFIED to 5
Actions #11

Updated by bizhang about 7 years ago

  • Version set to 2.12.1
Actions #12

Updated by bizhang about 7 years ago

  • Version deleted (2.12.1)
Actions #13

Updated by Ichimonji10 about 7 years ago

This issue is fixed in the current 2.12.1 and 2.13.0 nightlies.

Actions #14

Updated by bizhang about 7 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #15

Updated by dkliban@redhat.com over 6 years ago

  • Has duplicate Issue #3140: pulp_streamer doesnt set HTTP response headers, causing squid cache is not used added
Actions #17

Updated by bmbouter about 6 years ago

  • Sprint set to Sprint 15
Actions #18

Updated by bmbouter about 6 years ago

  • Sprint/Milestone deleted (33)
Actions #19

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF