Project

Profile

Help

Issue #4420

Random choices in the tests lead to variations in the coverage.

Added by mdellweg 8 months ago. Updated 6 months ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Severity:
2. Medium
Version:
Platform Release:
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

In the functional tests, there are a lot of occurrences of `random.choice`. This leads inevitably to unpredictable test/coverage results.

4420_IRC_Conversation.log (4.76 KB) Short conversation about the problem from IRC bherring, 02/18/2019 02:04 PM 4420_IRC_Conversation.log

History

#1 Updated by CodeHeeler 8 months ago

  • Triaged changed from No to Yes
  • Tags Pulp 3 added

#2 Updated by mdellweg 8 months ago

This is kind of hard to nail down, but PR 3876 [0] should really only add coverage. Yet it drops one line.

[0] https://github.com/pulp/pulp/pull/3876

#3 Updated by bherring 8 months ago

Notes

Adding a reference [0] that would be useful to discuss, depending on the background.

Note that any time lines of code are removed without adding new functionality:

It is not wrong, your code contains less lines, so each line that is not tested is a larger percentage.
You can change the threshold for the amount of decrease to result in a failed build, on the projects page on coveralls.io.

There are also some suggestions in [0], depending on case.

Adding the IRC conversation for history about this conversation for those who missed it .

[0] - https://github.com/lemurheavy/coveralls-public/issues/565

#5 Updated by bmbouter 6 months ago

  • Tags deleted (Pulp 3)

Please register to edit this issue

Also available in: Atom PDF