Story #2840
closedPublish ansible roles to install pulp3
100%
Description
Motivation¶
Pulp3 alpha needs to be fully installable, the easiest way of doing this is upload the necessary ansible roles onto ansible galaxy.
A lot of the roles from pulp/devel can be used as the basis for the galaxy roles including:
- db [0]
- systemd [1]
A new role would need to be written to install pulp from PyPI.
We can use jytr's roles for rabbitmq and qpidd [2]
All the roles needs to work in RHEL7 and Fedora 25 (preferably be platform agnostic)
This playbook should be published on ansible galaxy, and instructions should be published in the pulp3 installation document [3].
[0] https://github.com/pulp/devel/tree/de6a8351c2aba5a0ded3228ecce0448844101e4d/ansible/roles/db
[1] https://github.com/pulp/devel/tree/de6a8351c2aba5a0ded3228ecce0448844101e4d/ansible/roles/systemd
[2] https://galaxy.ansible.com/jtyr/
[3] https://github.com/pulp/pulp/blob/589f0e74f17f25442d6720b30e09e40b839a292e/docs/installation.rst
Updated by bizhang about 7 years ago
- Subject changed from Publish ansible playbook to install pulp3 to Publish ansible roles to install pulp3
- Description updated (diff)
Updated by bmbouter about 7 years ago
There are also Ansible roles (including Pulp2 installers) made by a user called jtyr on galaxy: https://galaxy.ansible.com/jtyr/ He has a rabbitMQ and Qpid role. Could we publish a playbook that calls his roles and then runs the to-be-created role that installs Pulp from PyPI.
Then the role published through this task would:
- install Pulp via pip
- Install the systemd unit files
- install the httpd or nginx configs
I'm not sure what provides the server.conf. I imagine it would probably be included via MANIFEST.in? If so then Ansible would get it when it pip installs Pulp from source via github.
Updated by bizhang about 7 years ago
- Description updated (diff)
server.conf should be included in the pulpcore python package.
Using jtyr's packages sounds good to me. I've updated the description to match.
Updated by bmbouter about 7 years ago
- Groomed changed from No to Yes
Thank you bizhang for writing this up!
Updated by mhrivnak about 7 years ago
This will also require httpd and a new config file for httpd. We should not re-use the httpd config from Pulp 2.
Updated by bmbouter about 7 years ago
Adding a checklist item to create the /var/lib/pulp directory as part of the Ansible playbook
Updated by bmbouter about 7 years ago
Adding a checklist item to add the 'pulp' user to the Ansible playbook.
Updated by amacdona@redhat.com about 7 years ago
- Blocks Task #2908: Build a plugin writer's Vagrant VM and publish on Vagrant cloud on a Pulp account added
Updated by ipanova@redhat.com about 7 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to ipanova@redhat.com
Updated by jortel@redhat.com almost 7 years ago
- Sprint/Milestone changed from 43 to 44
Added by ipanova@redhat.com almost 7 years ago
Updated by ipanova@redhat.com almost 7 years ago
- Status changed from ASSIGNED to MODIFIED
- % Done changed from 0 to 100
Applied in changeset devel|28599f4818c42b335594a63f8fe27a626a2861b6.
Added by ipanova@redhat.com almost 7 years ago
Revision 4b5a0542 | View on GitHub
Add ansible galaxy installation guide.
Added by ipanova@redhat.com almost 7 years ago
Revision 4b5a0542 | View on GitHub
Add ansible galaxy installation guide.
Updated by ipanova@redhat.com almost 7 years ago
As a result, here are the repos created for roles:
https://github.com/pulp/ansible-pulp3
https://github.com/pulp/ansible-pulp3_db
https://github.com/pulp/ansible-pulp3_systemd
new PRs created:
https://github.com/pulp/pulp/pull/3148
https://github.com/pulp/devel/pull/89
Updated by Ichimonji10 almost 7 years ago
These playbooks fail on RHEL 7:
$ ansible-playbook deploy-pulp3.yml -e pulp3_broker=rabbitmq -i rhel-7-test,
PLAY [all] ************************************************************************************************************
TASK [Gathering Facts] ************************************************************************************************
ok: [rhel-7-test]
TASK [Require minimal Python and Ansible versions] ********************************************************************
fatal: [rhel-7-test]: FAILED! => {
"assertion": "ansible_python_version|version_compare('3.5.0', operator='ge')",
"changed": false,
"evaluated_to": false,
"failed": true,
"msg": "This role and those that depend on it require at least Python 3.5 and Ansible 2.2.\n"
}
to retry, use: --limit @/home/ichimonji10/code/devel/ansible/deploy-pulp3.retry
PLAY RECAP ************************************************************************************************************
rhel-7-test : ok=1 changed=0 unreachable=0 failed=1
I believe the issue is that RHEL 7 provides Python 3.4:
$ yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
repo id repo name status
epel/x86_64 Extra Packages for Enterprise Linux 7 - x86_64 11,947
rhel-7-server-extras-rpms/x86_64 Red Hat Enterprise Linux 7 Server - Extras (RPMs) 639
rhel-7-server-optional-rpms/7Server/x86_64 Red Hat Enterprise Linux 7 Server - Optional (RPMs) 12,809
rhel-7-server-rpms/7Server/x86_64 Red Hat Enterprise Linux 7 Server (RPMs) 17,215
repolist: 42,610
Updated by Ichimonji10 almost 7 years ago
These playbooks don't function on Fedora 25. The failures don't become obvious until after 30+ minutes of runtime. Here's a summary of what eventually goes wrong:
[root@fedora-25-test ~]# systemctl --state failed
UNIT LOAD ACTIVE SUB DESCRIPTION
● pulp_resource_manager.service loaded failed failed Pulp Resource Manager
● pulp_worker@1.service loaded failed failed Pulp Celery Worker
● pulp_worker@2.service loaded failed failed Pulp Celery Worker
LOAD = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB = The low-level unit activation state, values depend on unit type.
3 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
The issue appears to be that pulp_resource_manager can't connect to RabbitMQ. I suspect using a third-party role to install and configure RabbitMQ is overkill.
# journalctl -u pulp_resource_manager
-- Logs begin at Tue 2017-09-19 11:54:18 EDT, end at Tue 2017-09-19 14:37:49 EDT. --
Sep 19 12:07:17 fedora-25-test systemd[1]: Started Pulp Resource Manager.
Sep 19 12:07:22 fedora-25-test celery[26720]: [2017-09-19 16:07:22,334: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
Sep 19 12:07:22 fedora-25-test celery[26720]: Trying again in 2.00 seconds...
Sep 19 12:07:24 fedora-25-test celery[26720]: [2017-09-19 16:07:24,371: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
Sep 19 12:07:24 fedora-25-test celery[26720]: Trying again in 4.00 seconds...
Sep 19 12:07:28 fedora-25-test celery[26720]: [2017-09-19 16:07:28,387: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
Sep 19 12:07:28 fedora-25-test celery[26720]: Trying again in 6.00 seconds...
Sep 19 12:07:34 fedora-25-test celery[26720]: [2017-09-19 16:07:34,424: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
Sep 19 12:07:34 fedora-25-test celery[26720]: Trying again in 8.00 seconds...
Sep 19 12:07:42 fedora-25-test celery[26720]: [2017-09-19 16:07:42,451: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
Sep 19 12:07:42 fedora-25-test celery[26720]: Trying again in 10.00 seconds...
Sep 19 12:07:52 fedora-25-test celery[26720]: [2017-09-19 16:07:52,482: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
Sep 19 12:07:52 fedora-25-test celery[26720]: Trying again in 12.00 seconds...
Sep 19 12:08:04 fedora-25-test celery[26720]: [2017-09-19 16:08:04,505: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
Sep 19 12:08:04 fedora-25-test celery[26720]: Trying again in 14.00 seconds...
Sep 19 12:08:18 fedora-25-test celery[26720]: [2017-09-19 16:08:18,530: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
(snip!)
Sep 19 12:56:46 fedora-25-test celery[26720]: [2017-09-19 16:56:46,885: CRITICAL/MainProcess] Unrecoverable error: OperationalError('[Errno 111] Connection refused',)
Sep 19 12:56:46 fedora-25-test celery[26720]: Traceback (most recent call last):
Sep 19 12:56:46 fedora-25-test celery[26720]: File "/home/pulp/pulpvenv/lib/python3.5/site-packages/kombu/connection.py", line 414, in _reraise_as_library_errors
Sep 19 12:56:46 fedora-25-test celery[26720]: yield
Sep 19 12:56:46 fedora-25-test celery[26720]: File "/home/pulp/pulpvenv/lib/python3.5/site-packages/kombu/connection.py", line 405, in ensure_connection
Sep 19 12:56:46 fedora-25-test celery[26720]: callback)
Sep 19 12:56:46 fedora-25-test celery[26720]: File "/home/pulp/pulpvenv/lib/python3.5/site-packages/kombu/utils/functional.py", line 333, in retry_over_time
Sep 19 12:56:46 fedora-25-test celery[26720]: return fun(*args, **kwargs)
Sep 19 12:56:46 fedora-25-test celery[26720]: File "/home/pulp/pulpvenv/lib/python3.5/site-packages/kombu/connection.py", line 261, in connect
Sep 19 12:56:46 fedora-25-test celery[26720]: return self.connection
Sep 19 12:56:46 fedora-25-test celery[26720]: File "/home/pulp/pulpvenv/lib/python3.5/site-packages/kombu/connection.py", line 802, in connection
Sep 19 12:56:46 fedora-25-test celery[26720]: self._connection = self._establish_connection()
Sep 19 12:56:46 fedora-25-test celery[26720]: File "/home/pulp/pulpvenv/lib/python3.5/site-packages/kombu/connection.py", line 757, in _establish_connection
Sep 19 12:56:46 fedora-25-test celery[26720]: conn = self.transport.establish_connection()
Sep 19 12:56:46 fedora-25-test celery[26720]: File "/home/pulp/pulpvenv/lib/python3.5/site-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
Sep 19 12:56:46 fedora-25-test celery[26720]: conn.connect()
Sep 19 12:56:46 fedora-25-test celery[26720]: File "/home/pulp/pulpvenv/lib/python3.5/site-packages/amqp/connection.py", line 282, in connect
Sep 19 12:56:46 fedora-25-test celery[26720]: self.transport.connect()
Sep 19 12:56:46 fedora-25-test celery[26720]: File "/home/pulp/pulpvenv/lib/python3.5/site-packages/amqp/transport.py", line 109, in connect
Sep 19 12:56:46 fedora-25-test celery[26720]: self._connect(self.host, self.port, self.connect_timeout)
Sep 19 12:56:46 fedora-25-test celery[26720]: File "/home/pulp/pulpvenv/lib/python3.5/site-packages/amqp/transport.py", line 150, in _connect
Sep 19 12:56:46 fedora-25-test celery[26720]: self.sock.connect(sa)
Sep 19 12:56:46 fedora-25-test celery[26720]: ConnectionRefusedError: [Errno 111] Connection refused
Sep 19 12:56:46 fedora-25-test celery[26720]: During handling of the above exception, another exception occurred:
(snip!)
Updated by ipanova@redhat.com almost 7 years ago
on el7 python3.5 is available trough software collections, maybe worth explicitly mentioning that in docs.
w/r to the broker probably some config parameter needs to be tweaked, maybe the defaults ones do not satisfy our specific install.
Updated by ipanova@redhat.com almost 7 years ago
We decided for now not to include the el7 support.
The broker connection issue was fixed, thanks for testing and pointing this out
Updated by bmbouter over 6 years ago
- Tags deleted (
Pulp 3 Plugin Writer Alpha)
Cleaning up Redmine tags
Updated by dkliban@redhat.com over 6 years ago
- Blocks deleted (Task #2908: Build a plugin writer's Vagrant VM and publish on Vagrant cloud on a Pulp account)
Updated by bmbouter over 4 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Publish ansible galaxy playbook for pulp3 pypi installation.
closes #2840 https://pulp.plan.io/issues/2840