Unittest builder for plugins fails if the minimum required platform version is unavailable
When plugin requires Pulp core version to be >= X.Y, builder tries to install version X.Y which may not exist for certain boxes.
OSTree plugin requires Pulp core to be >= 2.10, so builder tries to install Pulp 2.10 on F25 system but fails  because we do not have pulp-2.10 build for F25.
The PR builder job for plugins should be modified to:
1) Install the latest GA core RPMs and test against those. If they pass then pass the PR[UPDATE] This is already done; see the first associated commit.
2) If there are failures, retest using packages from the nightly build of Pulp. It's possible that there is new code that is required from platform that is not yet released. If this passes, then pass the PR
#7 Updated by bmbouter over 4 years ago
For the latest version of core that will get installed, I'm wondering if that is the GA release of core or the nightly. Yesterday when we were talking I suggested using the nightly, but now I'm thinking that the existing GA is probably the best. The reason you would want it to be the nightly is because of situations wituations where plugin code relies on core changes that are not part of the GA release yet. The reason not to use nightlies is that they could be unreliable. I'm favoring using the GA for two reasons:
1. plugin code changes on the pulp2 line that also require unreleased core changes doesn't happen a whole lot (but some).
2. It's probably a best practice to test one thing at a time. This is about testing the plugin code, so that means core should be stable. If we use the nightly for most failures we'll have to determine if platform OR the plugin code is broken. By always using the GA of core we know everytime that it's the plugin. It's also possible that the plugin needs not-yet-released code in core, but the plugin writer should know if they are in one of those cases at which point tests can be run locally to prove they work perhaps.
What do others think about ^?
#11 Updated by bmbouter over 4 years ago
- Status changed from MODIFIED to NEW
We probably can't use the GA and need to use the nightlies for the rest of Pulp2 at least. I wrote some about why here.
Since we need to make a URLs change I'm moving back to NEW.
Please register to edit this issue