Pulp

Starting with the 2.8.0 many database collections will have validation checks on them. It's likely that there are fields in existing Pulp databases which will not validate. This is true in large part because there was no validation in previous Pulp versions. Fixing the issues with a migration or mongodb queries is generally straightforward, but knowing what the database problems are is a challenge.

To identify problems, a script should be written. The script makes the following assumptions:

• This script is designed to run on a test system that has the 2.8 code or the in-development version of 2.8 code installed.
• This 2.8 code will have all plugins types regardless of what the user's normal database had. These plugins include: pulp_docker, pulp_ostree, pulp_puppet, pulp_python, and pulp_rpm
• This test system must have a mongodb already running with the user's pre-upgraded data loaded into the pulp_database inside of it.
• The test environment's server.conf will contain the config which defines how to connect to this test db just like a normal pulp setup would do.
• The script should be run as user apache, and the first thing it will do is run pulp-manage-db.
• This script will connect to the database using the existing pulp connection mechanisms.

After pulp-manage-db is run, for each of the following models, the script will import each mongoengine based model using mongoengine and with no modifications will save() the model again. Any ValidationErrors will be raised with save(). It's important that any ValidationError exceptions are caught and recorded by writing to a separate text file. The actual record contents also should be written to the text file using mongoengine's to_mongo() call. The text file named 'validation_errors_<timestamp>.txt' should have <timestamp> be the current datetime so that multiple runs will not collide. If any validation problems occur, the script should instruct users to look at the contents of the validation file just created and to create a representative sample of validation issues at pulp.plan.io.

Note: once a single model has reached 50 ValidationErrors, continue on to the next model. It's not productive to log a huge number of errors that are all the same.

Also, a release note should be added to the 2.8.0 release notes indicating that any users is strongly recommended to run the pre-upgrade validation script prior to upgrading to 2.8.0.