Project

Profile

Help

Issue #1907

python-pulp-streamer is not pulled with groupinstall pulp-server-qpid

Added by lhorace over 4 years ago. Updated over 1 year ago.

Status:
CLOSED - WONTFIX
Priority:
High
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.8.2
Platform Release:
OS:
RHEL 7
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Documentation, Pulp 2
Sprint:
Quarter:

Description

Hello,

In regards to RHEL packages, I am not sure if I should be reporting this issue here or to the maintainer of https://repos.fedorapeople.org/repos/pulp/pulp/stable/2/.

Regardless, When I attempted to download content units for a repository, the task failed. Additionally, pulp did not log the reason of the failure.

[root@pulp ~]# pulp repo download --repo-id=icinga-x86_64-stable-release
+----------------------------------------------------------------------+
         Downloading Repository [icinga-x86_64-stable-release]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Download Repository Content
[==========================================        ] 85%
92 of 107 items
... failed

Task Succeeded

I came accross this issue https://pulp.plan.io/issues/1691 which was close because pulp_streamer was not running. In fact, in my case, although /usr/lib/systemd/system/pulp_streamer.service is provided by pulp-server-2.8.2-1.el7.noarch. There is no /usr/bin/pulp_streamer, which is provided by python-pulp-streamer-2.8.2-1.el7.noarch. Oddly enough, both pulp-server-2.8.2-1.el7.noarch and python-pulp-streamer-2.8.2-1.el7.noarch provide /usr/lib/systemd/system/pulp_streamer.service which yum did not see as a conflict. :/

[root@pulp ~]# rpm -qf /usr/lib/systemd/system/pulp_streamer.service 
pulp-server-2.8.2-1.el7.noarch
python-pulp-streamer-2.8.2-1.el7.noarch

Additionally, 'yum groupinstall pulp-server-qpid' from http://pulp.readthedocs.io/en/latest/user-guide/installation.html#prerequisites did not seem to pull python-pulp-streamer.

Apr 30 04:20:27 Installed: qpid-proton-c-0.12.1-1.el7.x86_64
Apr 30 04:20:29 Installed: qpid-proton-c-devel-0.12.1-1.el7.x86_64
Apr 30 04:20:51 Installed: python-qpid-proton-0.12.1-1.el7.x86_64
Apr 30 04:21:23 Installed: boost-system-1.53.0-25.el7.x86_64
Apr 30 04:21:25 Installed: boost-filesystem-1.53.0-25.el7.x86_64
Apr 30 04:21:26 Installed: boost-thread-1.53.0-25.el7.x86_64
Apr 30 04:21:28 Installed: 2:libunwind-1.1-5.el7_2.2.x86_64
Apr 30 04:21:30 Installed: gperftools-libs-2.4-7.0.1.el7.x86_64
Apr 30 04:21:32 Installed: yaml-cpp-0.5.1-6.el7.x86_64
Apr 30 04:21:34 Installed: libstemmer-0-2.585svn.el7.x86_64
Apr 30 04:21:36 Installed: 1:v8-3.14.5.10-17.el7.x86_64
Apr 30 04:21:38 Installed: snappy-1.1.0-3.el7.x86_64
Apr 30 04:21:39 Installed: boost-program-options-1.53.0-25.el7.x86_64
Apr 30 04:21:46 Installed: mongodb-server-2.6.11-1.el7.x86_64
Apr 30 04:32:52 Installed: boost-chrono-1.53.0-25.el7.x86_64
Apr 30 04:32:55 Installed: libicu-50.1.2-15.el7.x86_64
Apr 30 04:32:57 Installed: boost-regex-1.53.0-25.el7.x86_64
Apr 30 04:32:59 Installed: boost-date-time-1.53.0-25.el7.x86_64
Apr 30 04:33:01 Installed: libdb4-4.8.30-13.el7.x86_64
Apr 30 04:33:02 Installed: libdb4-cxx-4.8.30-13.el7.x86_64
Apr 30 04:33:06 Installed: boost-wave-1.53.0-25.el7.x86_64
Apr 30 04:33:07 Installed: boost-graph-1.53.0-25.el7.x86_64
Apr 30 04:33:10 Installed: boost-locale-1.53.0-25.el7.x86_64
Apr 30 04:33:11 Installed: boost-timer-1.53.0-25.el7.x86_64
Apr 30 04:33:13 Installed: boost-python-1.53.0-25.el7.x86_64
Apr 30 04:33:15 Installed: boost-serialization-1.53.0-25.el7.x86_64
Apr 30 04:33:17 Installed: boost-atomic-1.53.0-25.el7.x86_64
Apr 30 04:33:21 Installed: boost-context-1.53.0-25.el7.x86_64
Apr 30 04:33:28 Installed: cyrus-sasl-2.1.26-20.el7_2.x86_64
Apr 30 04:33:31 Installed: boost-random-1.53.0-25.el7.x86_64
Apr 30 04:33:34 Installed: boost-iostreams-1.53.0-25.el7.x86_64
Apr 30 04:33:37 Installed: libaio-0.3.109-13.el7.x86_64
Apr 30 04:33:40 Installed: boost-signals-1.53.0-25.el7.x86_64
Apr 30 04:33:44 Installed: boost-test-1.53.0-25.el7.x86_64
Apr 30 04:33:46 Installed: boost-math-1.53.0-25.el7.x86_64
Apr 30 04:33:48 Installed: boost-1.53.0-25.el7.x86_64
Apr 30 04:33:50 Installed: qpid-cpp-client-0.34-6.el7.x86_64
Apr 30 04:33:54 Installed: qpid-cpp-server-0.34-6.el7.x86_64
Apr 30 04:33:56 Installed: qpid-cpp-server-linearstore-0.34-6.el7.x86_64
Apr 30 07:34:48 Installed: python-dateutil-1.5-7.el7.noarch
Apr 30 07:34:51 Installed: m2crypto-0.21.1-17.el7.x86_64
Apr 30 07:34:53 Installed: python-rhsm-1.15.4-5.el7.x86_64
Apr 30 07:39:30 Installed: python-isodate-0.5.0-4.pulp.el7.noarch
Apr 30 07:39:32 Installed: python-pulp-common-2.8.2-1.el7.noarch
Apr 30 07:39:34 Installed: apr-1.4.8-3.el7.x86_64
Apr 30 07:39:36 Installed: apr-util-1.5.2-6.0.1.el7.x86_64
Apr 30 07:39:38 Installed: cronie-1.4.11-14.el7_2.1.x86_64
Apr 30 07:39:40 Installed: crontabs-1.11-6.20121102git.el7.noarch
Apr 30 07:39:42 Installed: cronie-anacron-1.4.11-14.el7_2.1.x86_64
Apr 30 07:39:44 Installed: python-blinker-1.3-2.el7.noarch
Apr 30 07:39:46 Installed: python-semantic_version-2.4.2-1.el7.noarch
Apr 30 07:39:47 Installed: deltarpm-3.6-3.el7.x86_64
Apr 30 07:39:50 Installed: python-httplib2-0.7.7-3.el7.noarch
Apr 30 07:39:51 Installed: python-anyjson-0.3.3-3.el7.noarch
Apr 30 07:39:53 Installed: python-amqp-1.4.9-1.el7.noarch
Apr 30 07:39:55 Installed: python-gofer-2.7.5-1.el7.noarch
Apr 30 07:39:57 Installed: 1:python-kombu-3.0.33-4.pulp.el7.noarch
Apr 30 07:39:59 Installed: python-deltarpm-3.6-3.el7.x86_64
Apr 30 07:40:01 Installed: httpd-tools-2.4.6-40.0.1.el7.x86_64
Apr 30 07:40:03 Installed: python-pulp-rpm-common-2.8.2-1.el7.noarch
Apr 30 07:40:05 Installed: python-django-bash-completion-1.6.11-5.el7.noarch
Apr 30 07:40:11 Installed: python-django-1.6.11-5.el7.noarch
Apr 30 07:40:13 Installed: libselinux-python-2.2.2-6.el7.x86_64
Apr 30 07:40:15 Installed: python-six-1.9.0-2.el7.noarch
Apr 30 07:40:17 Installed: checkpolicy-2.1.12-6.el7.x86_64
Apr 30 07:40:19 Installed: python-bson-3.2-1.el7.x86_64
Apr 30 07:40:23 Installed: python-pymongo-3.2-1.el7.x86_64
Apr 30 07:40:24 Installed: python-pymongo-gridfs-3.2-1.el7.x86_64
Apr 30 07:40:27 Installed: python-mongoengine-0.10.5-1.el7.noarch
Apr 30 07:40:29 Installed: pytz-2012d-5.el7.noarch
Apr 30 07:40:32 Installed: audit-libs-python-2.4.1-5.el7.x86_64
Apr 30 07:40:34 Installed: libusal-1.1.11-23.el7.x86_64
Apr 30 07:40:36 Installed: genisoimage-1.1.11-23.el7.x86_64
Apr 30 07:40:38 Installed: 1:python-billiard-3.3.0.17-2.el7.x86_64
Apr 30 07:40:39 Installed: python-backports-1.0-8.el7.x86_64
Apr 30 07:40:43 Installed: python-backports-ssl_match_hostname-3.4.0.2-4.el7.noarch
Apr 30 07:40:47 Installed: python-setuptools-0.9.8-4.el7.noarch
Apr 30 07:40:48 Installed: python-pulp-docker-common-2.0.0-1.el7.noarch
Apr 30 07:40:51 Installed: python-pulp-puppet-common-2.8.2-1.el7.noarch
Apr 30 07:40:52 Installed: python-urllib3-1.10.2-2.el7_1.noarch
Apr 30 07:40:55 Installed: libcgroup-0.41-8.el7.x86_64
Apr 30 07:40:59 Installed: libxml2-python-2.9.1-6.0.1.el7_2.2.x86_64
Apr 30 07:41:01 Installed: createrepo-0.9.9-25.el7_2.noarch
Apr 30 07:41:02 Installed: python-IPy-0.75-6.el7.noarch
Apr 30 07:41:04 Installed: saslwrapper-0.16-5.el7.x86_64
Apr 30 07:41:06 Installed: python-saslwrapper-0.16-5.el7.x86_64
Apr 30 07:41:08 Installed: python-simplejson-3.3.3-1.el7.x86_64
Apr 30 07:41:10 Installed: python-oauth2-1.5.211-8.el7.noarch
Apr 30 07:41:12 Installed: setools-libs-3.3.7-46.el7.x86_64
Apr 30 07:41:14 Installed: python-qpid-common-0.32-13.el7.noarch
Apr 30 07:41:16 Installed: python-qpid-0.32-13.el7.noarch
Apr 30 07:41:18 Installed: python-qpid-qmf-0.32-1.el7.x86_64
Apr 30 07:41:21 Installed: qpid-qmf-0.32-1.el7.x86_64
Apr 30 07:41:24 Installed: oracle-logos-70.0.3-4.0.7.el7.noarch
Apr 30 07:41:26 Installed: libsemanage-python-2.1.10-18.el7.x86_64
Apr 30 07:41:29 Installed: policycoreutils-python-2.2.5-20.0.1.el7.x86_64
Apr 30 07:41:31 Installed: pyparsing-1.5.6-9.el7.noarch
Apr 30 07:41:35 Installed: python-celery-3.1.11-1.el7.noarch
Apr 30 07:41:37 Installed: python-chardet-2.2.1-1.el7_1.noarch
Apr 30 07:41:39 Installed: python-requests-2.6.0-1.el7_1.noarch
Apr 30 07:41:41 Installed: python-nectar-1.5.1-1.el7.noarch
Apr 30 07:41:44 Installed: libxslt-1.1.28-5.0.1.el7.x86_64
Apr 30 07:41:46 Installed: python-lxml-3.2.1-4.el7.x86_64
Apr 30 07:41:48 Installed: python-ldap-2.4.15-2.el7.x86_64
Apr 30 07:41:50 Installed: mailcap-2.1.41-2.el7.noarch
Apr 30 07:41:54 Installed: httpd-2.4.6-40.0.1.el7.x86_64
Apr 30 07:41:58 Installed: 1:mod_ssl-2.4.6-40.0.1.el7.x86_64
Apr 30 07:41:59 Installed: mod_wsgi-3.4-12.el7_0.x86_64
Apr 30 07:42:01 Installed: python-pulp-repoauth-2.8.2-1.el7.noarch
Apr 30 07:42:04 Installed: python-pulp-oid_validation-2.8.2-1.el7.noarch
Apr 30 07:42:06 Installed: mod_xsendfile-0.12-10.el7.x86_64
Apr 30 07:42:13 Installed: pulp-server-2.8.2-1.el7.noarch
Apr 30 07:42:15 Installed: createrepo_c-libs-0.9.0-1.el7.x86_64
Apr 30 07:42:18 Installed: createrepo_c-0.9.0-1.el7.x86_64
Apr 30 07:42:21 Installed: pulp-rpm-plugins-2.8.2-1.el7.noarch
Apr 30 07:42:23 Installed: pulp-puppet-plugins-2.8.2-1.el7.noarch
Apr 30 07:42:24 Installed: pulp-docker-plugins-2.0.0-1.el7.noarch
Apr 30 07:42:28 Installed: pulp-selinux-2.8.2-1.el7.noarch
Apr 30 07:42:30 Installed: qpid-tools-0.32-9.el7.noarch
Apr 30 07:42:33 Installed: python-gofer-qpid-2.7.5-1.el7.noarch
Apr 30 08:09:51 Installed: htop-2.0.1-1.el7.x86_64
Apr 30 08:21:57 Installed: python-pulp-bindings-2.8.2-1.el7.noarch
Apr 30 08:21:59 Installed: python-okaara-1.0.35-1.el7.noarch
Apr 30 08:22:02 Installed: python-pulp-client-lib-2.8.2-1.el7.noarch
Apr 30 08:22:04 Installed: pulp-admin-client-2.8.2-1.el7.noarch
Apr 30 08:22:06 Installed: pulp-puppet-admin-extensions-2.8.2-1.el7.noarch
Apr 30 08:22:08 Installed: pulp-rpm-admin-extensions-2.8.2-1.el7.noarch
Apr 30 08:22:11 Installed: pulp-docker-admin-extensions-2.0.0-1.el7.noarch
Apr 30 08:38:59 Installed: less-458-9.el7.x86_64

Regards

History

#1 Updated by lhorace over 4 years ago

In regards to why the failure was not logged, seems like log_level of ERROR isn't high enough to report meaniful information. DEBUG did the job.

Regards

#2 Updated by lhorace over 4 years ago

I stumbled onto https://media.readthedocs.org/pdf/pulp/stable/pulp.pdf while I was trying to rectify another problem I was facing with pulp, which elobrates a bit more on download policies. Does immediate depends on python-pulp-streamer? Which explains perhaps why it wasn't listed as a dependency. Although, I have Apache, Varnish, python-pulp-streamer functioning, I still can't run:

[root@pulp log]# pulp repo download --repo-id=webmin-noarch-base                                    
+----------------------------------------------------------------------+
              Downloading Repository [webmin-noarch-base]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Download Repository Content
[================                                  ] 33%
1 of 3 items
... failed

Task Succeeded
May 11 08:00:03 pulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): pulp.lhprojects.int
May 11 08:00:03 pulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (2): pulp.lhprojects.int
May 11 08:00:03 pulp pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (3): pulp.lhprojects.int
May 11 08:00:04 pulp pulp: nectar.downloaders.threaded:WARNING: Skipping requests to pulp.lhprojects.int:443 due to repeated connection failures: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:765)
May 11 08:00:04 pulp pulp: pulp.server.controllers.repository:INFO: Download of /var/lib/pulp/content/units/rpm/ec/49f00d534bec06478085013b0d9bfee4c0c81acd2c1315c00cba7ea6ab4bed/usermin-1.701-1.noarch.rpm failed: A connection error occurred.
May 11 08:00:04 pulp pulp: nectar.downloaders.threaded:WARNING: Skipping requests to pulp.lhprojects.int:443 due to repeated connection failures: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:765)
May 11 08:00:04 pulp pulp: nectar.downloaders.threaded:WARNING: Skipping requests to pulp.lhprojects.int:443 due to repeated connection failures: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:765)
May 11 08:00:04 pulp pulp: pulp.server.controllers.repository:INFO: Download of /var/lib/pulp/content/units/rpm/e4/f1bc99febc315c784368c1c66b7a614d1bef8bd8434becec026c6b023f9023/webmin-1.791-1.noarch.rpm failed: A connection error occurred.
May 11 08:00:04 pulp pulp: pulp.server.controllers.repository:INFO: Download of /var/lib/pulp/content/units/rpm/91/8cad1b9b81a8c1fb5f85db579d7fac407cd733b24b4c436671edf4afd5e273/usermin-webmail-1.701-1.noarch.rpm failed: A connection error occurred.
May 11 08:00:04 pulp pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.download_repo[55c85f52-9b4c-4ab1-a32e-3e9596e8606b] succeeded in 21.5763582741s: None

Regards

#3 Updated by bmbouter over 4 years ago

Pulp is designed to have the streamer be an optional component which needs to be installed directly. Perhaps the docs should be more clear on that I could see purposing this bug for that, but the title and description would need to be edited.

Since it is an optional component, we should not include it in the pulp-server-qpid install which is reserved for required server-side bits when Pulp is run with the Qpid broker.

Immediate downloads are the default and should not require python-pulp-streamer which is what makes python-pulp-streamer optional.

Given ^ what do you want to do with this bug report?

#4 Updated by jcline@redhat.com over 4 years ago

With respect to your last comment about not being able to ``pulp repo download --repo-id=webmin-noarch-base``:

Pulp is making requests to the Apache reverse proxy and defaults to secure. This means it uses HTTPS and verifies the certificate that Apache offers. If you are running Pulp on the same host as the Apache reverse proxy/Varnish/Pulp streamer setup, you can safely use HTTP since all communication is happening locally. To do this, change the settings in the [lazy] section of ``/etc/pulp/server.conf``. Specifically, you can turn ``https_retrieval: true`` to ``https_retrieval: false``.

If, on the other hand, you want to use HTTPS, you'll need to configure the Pulp server to trust the CA that signed the SSL certificate the Apache reverse proxy is serving.

#5 Updated by lhorace over 4 years ago

bmbouter wrote:

Pulp is designed to have the streamer be an optional component which needs to be installed directly. Perhaps the docs should be more clear on that I could see purposing this bug for that, but the title and description would need to be edited.

Yes, this would be fanstatic to save cost in time having 'Optional Components' section in http://pulp.readthedocs.io/en/latest/user-guide/installation.html#prerequisites, with a link pointing to http://pulp.readthedocs.io/en/latest/user-guide/deferred-download.html.

Since it is an optional component, we should not include it in the pulp-server-qpid install which is reserved for required server-side bits when Pulp is run with the Qpid broker

This is not a bug? Wouldn't it be cleaner to have /usr/lib/systemd/system/pulp_streamer.service be provided by python-pulp-streamer-2.8.2-1.el7.noarch.

[root@pulp ~]# rpm -qf /usr/lib/systemd/system/pulp_streamer.service 
pulp-server-2.8.2-1.el7.noarch
python-pulp-streamer-2.8.2-1.el7.noarch

Immediate downloads are the default and should not require python-pulp-streamer which is what makes python-pulp-streamer optional.

Uh, when I switch download policy on my ecryptfs-x86_64 repo (which is a repository located on my internal lan) from background to immediate. Two new tasks are created however content units are not downloaded:

May 11 17:42:11 pulp pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
May 11 17:42:11 pulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[6ab08684-894f-4a33-a083-055792165a99]
May 11 17:42:11 pulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.download_deferred[c72eac06-6c0d-4993-a175-b3c34bcacb2a]
May 11 17:42:11 pulp pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.queue_download_deferred[6ab08684-894f-4a33-a083-055792165a99] succeeded in 0.0532712398563s: None
May 11 17:42:11 pulp pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.download_deferred[c72eac06-6c0d-4993-a175-b3c34bcacb2a] succeeded in 0.456320367055s: None
May 11 18:12:11 pulp pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
May 11 18:12:11 pulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[6f9dee02-2a89-406c-8529-95d57cc0b909]
May 11 18:12:11 pulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.download_deferred[d12ae7c8-1678-46cc-b495-a71d501b7fc4]
May 11 18:12:11 pulp pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.queue_download_deferred[6f9dee02-2a89-406c-8529-95d57cc0b909] succeeded in 0.0420156719629s: None
May 11 18:12:12 pulp pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.download_deferred[d12ae7c8-1678-46cc-b495-a71d501b7fc4] succeeded in 1.1721588918s: None
May 11 18:42:11 pulp pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
May 11 18:42:11 pulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[c8e94407-9c0d-4f1c-9f59-55c59a2e0d97]
May 11 18:42:11 pulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.download_deferred[23d6ec21-952a-47f6-9803-2d6d9e25048e]
May 11 18:42:11 pulp pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.queue_download_deferred[c8e94407-9c0d-4f1c-9f59-55c59a2e0d97] succeeded in 0.0333031900227s: None
May 11 18:42:12 pulp pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.download_deferred[23d6ec21-952a-47f6-9803-2d6d9e25048e] succeeded in 1.0592154311s: None
May 11 19:12:11 pulp pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
May 11 19:12:11 pulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[be6138e4-e20f-40bd-a08b-7db58c42226d]
May 11 19:12:11 pulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.download_deferred[f1976b9c-b149-4372-9a76-e93facc958c2]
May 11 19:12:11 pulp pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.queue_download_deferred[be6138e4-e20f-40bd-a08b-7db58c42226d] succeeded in 0.0498800999485s: None
May 11 19:12:12 pulp pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.download_deferred[f1976b9c-b149-4372-9a76-e93facc958c2] succeeded in 1.40341140889s: None
May 11 19:42:11 pulp pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
May 11 19:42:11 pulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[9416b977-daf1-4dfc-b1ea-33343f4534ad]
May 11 19:42:11 pulp pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.download_deferred[3d82ebb2-6cdb-4e85-b2b2-66c893cfefdf]
May 11 19:42:11 pulp pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.queue_download_deferred[9416b977-daf1-4dfc-b1ea-33343f4534ad] succeeded in 0.0565808371175s: None
May 11 19:42:12 pulp pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.download_deferred[3d82ebb2-6cdb-4e85-b2b2-66c893cfefdf] succeeded in 1.14704286307s: None
[root@pulp ~]# pulp -vv tasks details --task-id=3d82ebb2-6cdb-4e85-b2b2-66c893cfefdf
+----------------------------------------------------------------------+
                              Task Details
+----------------------------------------------------------------------+

2016-05-11 19:57:53,920 - DEBUG - sending GET request to /pulp/api/v2/tasks/3d82ebb2-6cdb-4e85-b2b2-66c893cfefdf/
2016-05-11 19:57:54,178 - INFO - GET request to /pulp/api/v2/tasks/3d82ebb2-6cdb-4e85-b2b2-66c893cfefdf/ with parameters None
2016-05-11 19:57:54,179 - INFO - Response status : 200 

2016-05-11 19:57:54,179 - INFO - Response body :
 {
  "exception": null, 
  "task_type": "pulp.server.controllers.repository.download_deferred", 
  "_href": "/pulp/api/v2/tasks/3d82ebb2-6cdb-4e85-b2b2-66c893cfefdf/", 
  "task_id": "3d82ebb2-6cdb-4e85-b2b2-66c893cfefdf", 
  "tags": [
    "pulp:action:deferred_download"
  ], 
  "finish_time": "2016-05-11T19:42:12Z", 
  "_ns": "task_status", 
  "start_time": "2016-05-11T19:42:11Z", 
  "traceback": null, 
  "spawned_tasks": [], 
  "progress_report": {
    "on_demand_download": [
      {
        "num_success": 0, 
        "description": "Download Cached On-Demand Content", 
        "step_type": "on_demand_download",
        "items_total": 0, 
        "state": "FINISHED", 
        "error_details": [], 
        "details": "", 
        "num_failures": 0, 
        "step_id": "72f09462-8b9a-4193-a8a9-f64194a9fd3b", 
        "num_processed": 0
      }
    ]
  }, 
  "queue": "celery.dq", 
  "state": "finished", 
  "worker_name": "celery", 
  "result": null, 
  "error": null, 
  "_id": {
    "$oid": "57338b137faeb6060a6f6d93"
  }, 
  "id": "57338b137faeb6060a6f6d93"
}

As far as I can see, no content units are downloaded:

[root@pulp log]# ll /var/lib/pulp/content
ls: cannot access /var/lib/pulp/content: No such file or directory

Given ^ what do you want to do with this bug report?

Perhaps changing the title to the following, would be a more fitting description as to what is going on, in this issue: Cannot download content units.

And tag documentation.

Regards

#6 Updated by lhorace over 4 years ago

wrote:

With respect to your last comment about not being able to ``pulp repo download --repo-id=webmin-noarch-base``:

Pulp is making requests to the Apache reverse proxy and defaults to secure. This means it uses HTTPS and verifies the certificate that Apache offers. If you are running Pulp on the same host as the Apache reverse proxy/Varnish/Pulp streamer setup, you can safely use HTTP since all communication is happening locally. To do this, change the settings in the [lazy] section of ``/etc/pulp/server.conf``. Specifically, you can turn ``https_retrieval: true`` to ``https_retrieval: false``.

If, on the other hand, you want to use HTTPS, you'll need to configure the Pulp server to trust the CA that signed the SSL certificate the Apache reverse proxy is serving.

Thanks a bunch, I spent 4 hours figuring out this issue but unfortunately, no-one, seemed to encounter this before. Hopefully this issue shows up in Google searchs.

Regards

#7 Updated by dkliban@redhat.com over 4 years ago

  • Triaged changed from No to Yes
  • Tags Documentation added

#8 Updated by dkliban@redhat.com over 4 years ago

  • Priority changed from Normal to High

#9 Updated by jcline@redhat.com over 4 years ago

lhorace wrote:

This is not a bug? Wouldn't it be cleaner to have /usr/lib/systemd/system/pulp_streamer.service be provided by python-pulp-streamer-2.8.2-1.el7.noarch.

Having both pulp-server and python-pulp-streamer provide /usr/lib/systemd/system/pulp_streamer.service is definitely a packaging bug. I've filed https://pulp.plan.io/issues/1924 against our packaging project.

Uh, when I switch download policy on my ecryptfs-x86_64 repo (which is a repository located on my internal lan) from background to immediate. Two new tasks are created however content units are not downloaded:

[...]

[...]

As far as I can see, no content units are downloaded:

[...]

This is expected. That task is dispatched on a schedule (you can change it in /etc/pulp/server.conf) and checks if there is any cached content. If there isn't any, it doesn't do anything.

#10 Updated by lhorace over 4 years ago

The problem, the Content Units are not downloadedd when I switch behavior (switch download policy)...Minor issue...

I have to pulp-admin repo download *

Consumer -> Pulp -> Apache -> Nginx (I have request store for about 30 minutes, then re-request) -> RPM Package

#11 Updated by jcline@redhat.com over 4 years ago

lhorace wrote:

The problem, the Content Units are not downloadedd when I switch behavior (switch download policy)...Minor issue...

I have to pulp-admin repo download *

Consumer -> Pulp -> Apache -> Nginx (I have request store for about 30 minutes, then re-request) -> RPM Package

Ah, I see. I believe you're experiencing https://pulp.plan.io/issues/1778 which appears to be in the upcoming 2.8.3 release.

#12 Updated by lhorace over 4 years ago

You guys can close this ticket and I will watch 1778

#13 Updated by bmbouter over 1 year ago

  • Status changed from NEW to CLOSED - WONTFIX

#14 Updated by bmbouter over 1 year ago

Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.

#15 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF