Project

Profile

Help

Issue #5817

Story #5762: [Epic] CI Improvements

Issue #5769: ansible-pulp molecule CI seems slower than it should be

ansible-pulp CI is slower than it should be due to a lack of ansible pipelining

Added by mdepaulo@redhat.com about 1 year ago. Updated 12 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
CI/CD
Sprint:
Quarter:

Associated revisions

Revision 0c222b02 View on GitHub
Added by Mike DePaulo about 1 year ago

Molecule: Enable Ansible pipelining via group_vars

Variables are currently sufficient for enabling pipelining in molecule because the role we tried (gzm55.smart_ssh_pipelining) did not need to modify system configuration for all 3 containers; it only set the variable.

Fixes: #5817 ansible-pulp CI is slower than it should be due to a lack of ansible pipelining [noissue]

Revision 0c222b02 View on GitHub
Added by Mike DePaulo about 1 year ago

Molecule: Enable Ansible pipelining via group_vars

Variables are currently sufficient for enabling pipelining in molecule because the role we tried (gzm55.smart_ssh_pipelining) did not need to modify system configuration for all 3 containers; it only set the variable.

Fixes: #5817 ansible-pulp CI is slower than it should be due to a lack of ansible pipelining [noissue]

Revision b031c5fa View on GitHub
Added by Mike DePaulo about 1 year ago

Problem: provisioning is slower than it could be

Solution: Enable ansible pipelining. It's safe to do on all our current boxes. If it ever isn't safe to do for 1, we can use this role to conditionally enable it: https://galaxy.ansible.com/gzm55/smart_ssh_pipelining

On my quick test (file module loop to touch /tmp/[1-20], it reduced the time per loop item from .41s .21s. Tasks without a loop receive a similar reduction in overhead.

re: #5817 ansible-pulp CI is slower than it should be due to a lack of ansible pipelining https://pulp.plan.io/issues/5817

History

#1 Updated by mdepaulo@redhat.com about 1 year ago

  • Status changed from NEW to POST

Submitted a new PR:
https://github.com/pulp/ansible-pulp/pull/196

Also submitted one for pulplift:
https://github.com/pulp/pulplift/pull/58

Also, I considered having ansible-pulp use this role all the time instead:
https://galaxy.ansible.com/gzm55/smart_ssh_pipelining
It would enable SSH pipelining when it determines that it is safe to do so. (if it isn't safe, tasks would fail.)

However, its logic for determining when it is safe to do so is not safe enough for all of our users, with all their possible managed system configurations. In the example of the "dzdo" become method, it blindly enables pipelining, even when dzdo has its own version of sudo's "requiretty" setting enabled.

#2 Updated by mdepaulo@redhat.com about 1 year ago

Note that for ansible-pulp's CI, the total run is reduced from about 33 mins on average to 26 min on average.

#3 Updated by Anonymous about 1 year ago

  • Status changed from POST to MODIFIED

#4 Updated by fao89 12 months ago

  • Tags CI/CD added

#5 Updated by bmbouter 12 months ago

  • Sprint/Milestone set to 3.0.0

#6 Updated by bmbouter 12 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF