Project

Profile

Help

Task #5768

closed

Story #5762: [Epic] CI Improvements

ansible-pulp CI needs a more time efficient test matrix

Added by mdepaulo@redhat.com over 4 years ago. Updated almost 4 years ago.

Status:
CLOSED - COMPLETE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
CI/CD
Sprint:
Sprint 65
Quarter:

Description

The current plan for ansible-pulp is for it to be rolling release. Thus, it will use different codepaths for 3.0 vs master (3.1) branches.

This is currently done by testing pulpcore via pip (3.0) and master (source_dir), as well as 0.1 and master (source_dir) for pulp)_file.

But soon 3.1 will be out and need testing also.

And now we have to test upgrade support.

Soon dynamic includes will be tested as well.

This means a larger set of tests for our CI based on tox / ansible molecule.

Currently both stable release and master branch are tested via the molecule config. Via a single test from Travis's perspective. Travis tests different versions of python on the ansible management node; which isn't quite so important.

We should consider a test matrix like this. I estimate that DYNAMIC-INSTALL will take less long, but it will minimize duplicate work in terms of the docker image being pulled / cached.
TOXENV=3.6 TESTS=ALL-UPGRADES (3.0.0 -> 3.0.x, 3.0.0 -> 3.1.x, 3.0.0 -> master) (all of these use the same pulled molecule docker image for 3.0.0) (Note: We do not want to test upgrades with py 2.7 because lots of tasks have no changes to make & thus run fewer python code, but py 2.7 is the most different.)
TOXENV=2.7 TESTS=DYNAMIC-INSTALL (all branches installed from scratch) (should use the same molecule docker image built at the beginning)
TOXENV=3.7 TESTS=STATIC-INSTALL (all branches installed from scratch) (should use the same molecule docker image built at the beginning)

Also available in: Atom PDF