RPM Support

By default, RPM repository importers are given a download policy of "immediate," meaning that when a sync occurs, all content in the repository referenced by the importer is immediately downloaded. However, an RPM repository importer may also be given a "background" or "on_demand" policy. Both of these download policies let content be synced asynchronously. Both of these download policies are broken in the current 2.14 nightlies. Here's a simple script with which to reproduce the error:

pulp-admin rpm repo create \
  --repo-id foo \
  --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-unsigned/ \
  --download-policy background
pulp-admin rpm repo sync run --repo-id foo

Numerous errors like the following are logged:

Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:INFO: Trying URL: https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-unsigned/frog-0.1-1.noarch.rpm
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456) An unexpected error occurred: /var/lib/pulp/content/units/rpm/75/1fede100010ba9eba49e22b5874849768861f70432645df62d8fafa454dc1b/frog-0.1-1.noarch.rpm
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456) Traceback (most recent call last):
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 158, in _handle_get
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456)     last_report = self._download(request, entry, responder)
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 215, in _download
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456)     downloader = self._get_downloader(request, entry)
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 256, in _get_downloader
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456)     downloader.session = self.session_cache.get(request.uri, downloader)
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: pulp.streamer.server:ERROR: (944-91456) TypeError: get() takes exactly 2 arguments (3 given)
Jun 06 12:02:24 rhel-7-pulp-2-14 pulp_streamer[944]: [-] 127.0.0.1 - - [06/Jun/2017:16:02:23 +0000] "GET /var/lib/pulp/content/units/rpm/75/1fede100010ba9eba49e22b5874849768861f70432645df62d8fafa454dc1b/frog-0.1-1.noarch.rpm HTTP/1.1" 500 - "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-514.21.1.el7.x86_64"

Writing a simple script with which to reproduce the on_demand error is a bit harder to do. The error looks the same, though. Here's an example of what shows up logs when an on_demand RPM is requested by a client:

Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:INFO: Trying URL: https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-signed/bear-4.1-1.noarch.rpm
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784) An unexpected error occurred: /var/lib/pulp/content/units/rpm/fe/a4f4a0861015d0cc487aa8b7358778964d6ee14ca865c4c891c514cbeaf9e1/bear-4.1-1.noarch.rpm
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784) Traceback (most recent call last):
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 158, in _handle_get
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784)     last_report = self._download(request, entry, responder)
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 215, in _download
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784)     downloader = self._get_downloader(request, entry)
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784)   File "/usr/lib/python2.7/site-packages/pulp/streamer/server.py", line 256, in _get_downloader
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784)     downloader.session = self.session_cache.get(request.uri, downloader)
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: pulp.streamer.server:ERROR: (946-06784) TypeError: get() takes exactly 2 arguments (3 given)
Jun 06 13:18:13 rhel-7-pulp-2-14 pulp_streamer[946]: [-] 127.0.0.1 - - [06/Jun/2017:17:18:13 +0000] "GET /var/lib/pulp/content/units/rpm/fe/a4f4a0861015d0cc487aa8b7358778964d6ee14ca865c4c891c514cbeaf9e1/bear-4.1-1.noarch.rpm HTTP/1.1" 500 - "-" "python-requests/2.17.3"

This error occurs for the 2.14 nightlies on F24, F25 and RHEL 7. Here's the relevant packages installed on my RHEL 7 test system:

[root@rhel-7-pulp-2-14 ~]# rpm -qa | grep -i 'pulp\|nectar' | sort
pulp-admin-client-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
pulp-docker-admin-extensions-2.5.0-0.1.alpha.git.275.8f02fb7.el7.noarch
pulp-docker-plugins-2.5.0-0.1.alpha.git.275.8f02fb7.el7.noarch
pulp-ostree-admin-extensions-1.3.0-0.1.alpha.git.198.db9f8a2.el7.noarch
pulp-ostree-plugins-1.3.0-0.1.alpha.git.198.db9f8a2.el7.noarch
pulp-puppet-admin-extensions-2.14.0-0.1.alpha.git.390.69e00a8.el7.noarch
pulp-puppet-plugins-2.14.0-0.1.alpha.git.390.69e00a8.el7.noarch
pulp-python-admin-extensions-2.1.0-0.1.alpha.git.171.b753bc2.el7.noarch
pulp-python-plugins-2.1.0-0.1.alpha.git.171.b753bc2.el7.noarch
pulp-rpm-admin-extensions-2.14.0-0.1.alpha.git.855.8e2a8c4.el7.noarch
pulp-rpm-plugins-2.14.0-0.1.alpha.git.855.8e2a8c4.el7.noarch
pulp-selinux-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
pulp-server-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
python-kombu-3.0.33-6.pulp.el7.noarch
python-nectar-1.5.4-1.el7.noarch
python-pulp-bindings-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
python-pulp-client-lib-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
python-pulp-common-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
python-pulp-docker-common-2.5.0-0.1.alpha.git.275.8f02fb7.el7.noarch
python-pulp-oid_validation-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
python-pulp-ostree-common-1.3.0-0.1.alpha.git.198.db9f8a2.el7.noarch
python-pulp-puppet-common-2.14.0-0.1.alpha.git.390.69e00a8.el7.noarch
python-pulp-python-common-2.1.0-0.1.alpha.git.171.b753bc2.el7.noarch
python-pulp-repoauth-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch
python-pulp-rpm-common-2.14.0-0.1.alpha.git.855.8e2a8c4.el7.noarch
python-pulp-streamer-2.14.0-0.1.alpha.git.18874.87c2a95.el7.noarch