Pulp

Currently Pulp does not use the MongoDB replica set functionality correctly when the user correctly configures the replica_set setting in the [database] section of server.conf. By default replica_set is not set. If replica_set is set, then Pulp should have pyMongo use the MongoReplicaSetClient type. This will allow Pulp to rely on PyMongo to automatically negotiate who the primary is and failover in the event that the Primary changes. Currently we attempt to do this, but there is a bug caused by parameter name mismatch which causes PyMongo to ignore our setting of this option. That is the first thing to fix. Once fixed it is expected that PyMongo will know about and use the replica_set as it interacts with MongoDB.

One thing to know is that when a failover of the primary happens an AutoReconnect exception is raised. To effectively test this story you'll have to have pulp "auto retry" the operation so that the next operation will go to the newly connected to Primary. On 2.6.x this should be the default behavior, on 2.7.0 you'll need to wait for story #1 to be implemented and set unsafe_autoretry = True in server.conf

In the case where multiple seeds are specified as fixed in issue (#1012), all seeds should be iterated through until a valid MongoReplicaSetClient connection can be found. Either Pulp needs to find the primary or PyMongo needs to reconnect to the primary. Specifically test that in a multi-node mongoDB configuration with one primary, one arbiter, and one secondary with all participating in a replica_set that you can specify only the secondary as the value for seed and that the MongoReplicaSetClient will automatically discover the primary and connect to it even though its not contained in the seed listing.

If Pulp does not have replica_set configured (the default) then PyMongo needs to use the single-mongoDB style client named MongoClient.

If this is story is introduced in a version which does not use mongoengine and then merged forward you'll have to ensure that the newer version with mongoengine continues to work as expected.

This story also introduces a constant named WRITE_CONCERN which should be hard coded to 'majority'. This will be the write concern that the entire Pulp codebase will use as its sets write concern throughout bugfix #1065. You'll need to verify that this will work correctly even if replica_set is not in use and PyMongo is using the MongoClient connection style. You will also need to verify that "majority" is a valid write concern to be specified on mongoDB server 2.4.0+. Verifying that is part of this story.

If Pulp does not have replica_set configured (the default) AND there are more than one seed specified in the [database] section of server.conf then an error should be logged and Pulp should refuse to start. This is because having multiple seeds without using replica set functionality of MongoDB is likely not a correct configuration and should be fixed up by the user. They will either need to have a single seed listed or configure a replica_set.