Project

Profile

Help

Issue #5817

closed

Story #5762: [Epic] CI Improvements

Issue #5769: ansible-pulp molecule CI seems slower than it should be

ansible-pulp CI is slower than it should be due to a lack of ansible pipelining

Added by mdepaulo@redhat.com over 4 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
CI/CD
Sprint:
Quarter:

Added by Mike DePaulo over 4 years ago

Revision 0c222b02 | View on GitHub

Molecule: Enable Ansible pipelining via group_vars

Variables are currently sufficient for enabling pipelining in molecule because the role we tried (gzm55.smart_ssh_pipelining) did not need to modify system configuration for all 3 containers; it only set the variable.

Fixes: #5817 ansible-pulp CI is slower than it should be due to a lack of ansible pipelining [noissue]

Added by Mike DePaulo over 4 years ago

Revision 0c222b02 | View on GitHub

Molecule: Enable Ansible pipelining via group_vars

Variables are currently sufficient for enabling pipelining in molecule because the role we tried (gzm55.smart_ssh_pipelining) did not need to modify system configuration for all 3 containers; it only set the variable.

Fixes: #5817 ansible-pulp CI is slower than it should be due to a lack of ansible pipelining [noissue]

Added by Mike DePaulo over 4 years ago

Revision b031c5fa | View on GitHub

Problem: provisioning is slower than it could be

Solution: Enable ansible pipelining. It's safe to do on all our current boxes. If it ever isn't safe to do for 1, we can use this role to conditionally enable it: https://galaxy.ansible.com/gzm55/smart_ssh_pipelining

On my quick test (file module loop to touch /tmp/[1-20], it reduced the time per loop item from .41s .21s. Tasks without a loop receive a similar reduction in overhead.

re: #5817 ansible-pulp CI is slower than it should be due to a lack of ansible pipelining https://pulp.plan.io/issues/5817

Actions #1

Updated by mdepaulo@redhat.com over 4 years ago

  • Status changed from NEW to POST

Submitted a new PR:
https://github.com/pulp/ansible-pulp/pull/196

Also submitted one for pulplift:
https://github.com/pulp/pulplift/pull/58

Also, I considered having ansible-pulp use this role all the time instead:
https://galaxy.ansible.com/gzm55/smart_ssh_pipelining
It would enable SSH pipelining when it determines that it is safe to do so. (if it isn't safe, tasks would fail.)

However, its logic for determining when it is safe to do so is not safe enough for all of our users, with all their possible managed system configurations. In the example of the "dzdo" become method, it blindly enables pipelining, even when dzdo has its own version of sudo's "requiretty" setting enabled.

Actions #2

Updated by mdepaulo@redhat.com over 4 years ago

Note that for ansible-pulp's CI, the total run is reduced from about 33 mins on average to 26 min on average.

Actions #3

Updated by Anonymous over 4 years ago

  • Status changed from POST to MODIFIED
Actions #4

Updated by fao89 over 4 years ago

  • Tags CI/CD added
Actions #5

Updated by bmbouter over 4 years ago

  • Sprint/Milestone set to 3.0.0
Actions #6

Updated by bmbouter over 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF