Project

Profile

Help

Issue #2798

closed

content can't be fetched for background or on-demand repositories

Added by Ichimonji10 almost 5 years ago. Updated about 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
4. Urgent
Version:
Master
Platform Release:
2.13.3
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 21
Quarter:

Description

By default, RPM repository importers are given a download policy of "immediate," meaning that when a sync occurs, all content in the repository referenced by the importer is immediately downloaded. However, an RPM repository importer may also be given a "background" or "on_demand" policy. Both of these download policies let content be synced asynchronously. Both of these download policies are broken in the current 2.14 nightlies. Here's a simple script with which to reproduce the error:

pulp-admin rpm repo create \
  --repo-id foo \
  --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-unsigned/ \
  --download-policy background
pulp-admin rpm repo sync run --repo-id foo

Numerous errors like the following are logged:

Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:INFO: Trying URL: https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-unsigned/frog-0.1-1.noarch.rpm
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456) An unexpected error occurred: /var/lib/pulp/content/units/rpm/75/1fede100010ba9eba49e22b5874849768861f70432645df62d8fafa454dc1b/frog-0.1-1.noarch.rpm
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456) Traceback (most recent call last):
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 158, in _handle_get
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456)     last_report = self._download(request, entry, responder)
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 215, in _download
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456)     downloader = self._get_downloader(request, entry)
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 256, in _get_downloader
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456)     downloader.session = self.session_cache.get(request.uri, downloader)
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456) TypeError: get() takes exactly 2 arguments (3 given)
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: [-] 127.0.0.1 - - [06/Jun/2017:16:02:23 +0000] "GET /var/lib/pulp/content/units/rpm/75/1fede100010ba9eba49e22b5874849768861f70432645df62d8fafa454dc1b/frog-0.1-1.noarch.rpm HTTP/1.1" 500 - "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-514.21.1.el7.x86_64"

Writing a simple script with which to reproduce the on_demand error is a bit harder to do. The error looks the same, though. Here's an example of what shows up logs when an on_demand RPM is requested by a client:

Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:INFO: Trying URL: https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-signed/bear-4.1-1.noarch.rpm
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784) An unexpected error occurred: /var/lib/pulp/content/units/rpm/fe/a4f4a0861015d0cc487aa8b7358778964d6ee14ca865c4c891c514cbeaf9e1/bear-4.1-1.noarch.rpm
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784) Traceback (most recent call last):
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 158, in _handle_get
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784)     last_report = self._download(request, entry, responder)
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 215, in _download
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784)     downloader = self._get_downloader(request, entry)
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 256, in _get_downloader
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784)     downloader.session = self.session_cache.get(request.uri, downloader)
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784) TypeError: get() takes exactly 2 arguments (3 given)
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: [-] 127.0.0.1 - - [06/Jun/2017:17:18:13 +0000] "GET /var/lib/pulp/content/units/rpm/fe/a4f4a0861015d0cc487aa8b7358778964d6ee14ca865c4c891c514cbeaf9e1/bear-4.1-1.noarch.rpm HTTP/1.1" 500 - "-" "python-requests/2.17.3"

This error occurs for the 2.14 nightlies on F24, F25 and RHEL 7. Here's the relevant packages installed on my RHEL 7 test system:

[root@rhel-7-pulp-2-14 ~]# rpm -qa | grep -i 'pulp\|nectar' | sort
pulp-admin-client-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
pulp-docker-admin-extensions-2.5.0-0.1.alpha.git.275.8f02fb7.el7.noarch
pulp-docker-plugins-2.5.0-0.1.alpha.git.275.8f02fb7.el7.noarch
pulp-ostree-admin-extensions-1.3.0-0.1.alpha.git.198.db9f8a2.el7.noarch
pulp-ostree-plugins-1.3.0-0.1.alpha.git.198.db9f8a2.el7.noarch
pulp-puppet-admin-extensions-2.14.0-0.1.alpha.git.390.69e00a8.el7.noarch
pulp-puppet-plugins-2.14.0-0.1.alpha.git.390.69e00a8.el7.noarch
pulp-python-admin-extensions-2.1.0-0.1.alpha.git.171.b753bc2.el7.noarch
pulp-python-plugins-2.1.0-0.1.alpha.git.171.b753bc2.el7.noarch
pulp-rpm-admin-extensions-2.14.0-0.1.alpha.git.855.8e2a8c4.el7.noarch
pulp-rpm-plugins-2.14.0-0.1.alpha.git.855.8e2a8c4.el7.noarch
pulp-selinux-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
pulp-server-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
python-kombu-3.0.33-6.pulp.el7.noarch
python-nectar-1.5.4-1.el7.noarch
python-pulp-bindings-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
python-pulp-client-lib-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
python-pulp-common-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
python-pulp-docker-common-2.5.0-0.1.alpha.git.275.8f02fb7.el7.noarch
python-pulp-oid_validation-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
python-pulp-ostree-common-1.3.0-0.1.alpha.git.198.db9f8a2.el7.noarch
python-pulp-puppet-common-2.14.0-0.1.alpha.git.390.69e00a8.el7.noarch
python-pulp-python-common-2.1.0-0.1.alpha.git.171.b753bc2.el7.noarch
python-pulp-repoauth-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
python-pulp-rpm-common-2.14.0-0.1.alpha.git.855.8e2a8c4.el7.noarch
python-pulp-streamer-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
Actions #1

Updated by pthomas@redhat.com almost 5 years ago

This looks like a regression and a blocker. Please add it to the sprint during triage.

Actions #2

Updated by ttereshc almost 5 years ago

It seems like it's caused by this recent commit.

Actions #3

Updated by ttereshc almost 5 years ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 4. Urgent
  • Triaged changed from No to Yes
Actions #4

Updated by ttereshc almost 5 years ago

  • Sprint/Milestone set to 40

Added by jortel@redhat.com almost 5 years ago

Revision 8fceb2b9

Fix trace in get_downloader(). closes #2798

Actions #6

Updated by jortel@redhat.com almost 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to jortel@redhat.com
Actions #7

Updated by jortel@redhat.com almost 5 years ago

  • Status changed from ASSIGNED to POST
Actions #8

Updated by jortel@redhat.com almost 5 years ago

  • Status changed from POST to MODIFIED

Added by jortel@redhat.com almost 5 years ago

Revision 078ed1b1

Fix trace in get_downloader(). closes #2798

(cherry picked from commit 8fceb2b9e081de34c05ee4477b7074ef10e8fbd4)

Actions #11

Updated by pcreech almost 5 years ago

  • Platform Release set to 2.13.3
Actions #12

Updated by Ichimonji10 almost 5 years ago

The automated tests for this issue are passing.

Actions #13

Updated by pcreech almost 5 years ago

  • Status changed from MODIFIED to 5
Actions #14

Updated by pcreech almost 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #15

Updated by bmbouter about 4 years ago

  • Sprint set to Sprint 21
Actions #16

Updated by bmbouter about 4 years ago

  • Sprint/Milestone deleted (40)
Actions #17

Updated by bmbouter about 3 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF