Project

Profile

Help

Task #2266

Figure out how to test Pulp 3 Platform and Plugins

Added by semyers about 3 years ago. Updated 7 months ago.

Status:
CLOSED - COMPLETE
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Platform Release:
Blocks Release:
Backwards Incompatible:
No
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

In trying to tackle #2194, I started to go down a little bit of a rabbit hole related to getting the "ptests" alias to work. Each plugin has its own "run-tests.py", which brings in test runner code from pulp.devel, and each plugin then does something a little bit different when it comes to running its tests, with some running pep257, and other minor inconsistencies.

I'm not sure how to deal with this, but have some ideas:

  • pulp.devel (currently devel in the pulp platform repo, packaged as pulp-devel RPM) should go away, and common dev-related lib code should live in the pulp_dev lib from the devel repo
  • Platform and plugins should register an executable, either as a pulp-dev subcommand or as a standalone executable, to handle running pulp's tests with uniform standards across plugins
  • Django introduces a new dimension to testing (python manage.py test, specifically), with the plugins exposed as apps that depend on the platform app that must be taken into account when testing.
  • Travis CI should run the same set of acceptance tests that developers do prior to committing

Crazy idea:
Assuming we officially adopt a less coverage-oriented testing scheme (probatio gratia probationis? I'm not sure we have an issue open about this yet), we could potentially opt to run every installed plugin's tests (plus platform) when testing PRs instead of having a different tester for every plugin. Even if we do have thorough unit-testing with a 100% coverage goal, the Django tests should be written to run quickly.

History

#1 Updated by semyers about 3 years ago

  • Description updated (diff)

#2 Updated by semyers about 3 years ago

Oh yeah, all the different tests we want to run shouldn't get mixed up into one big testing function like they are in pulp 2, so it's easy to enable/disable different tests. The three things I wish we could easily do before submitting a PR are run the test suite, run the linter, test building the docs.

#3 Updated by dalley about 1 year ago

  • Status changed from NEW to CLOSED - COMPLETE

#4 Updated by bmbouter 7 months ago

  • Tags deleted (Pulp 3)

Please register to edit this issue

Also available in: Atom PDF