Project

Profile

Help

Issue #1139

closed

Task #1014: Short Term Improvements for Pulp's use of MongoDB

Fix Pulp's use of replica sets replica sets

Added by bmbouter about 7 years ago. Updated over 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
2.7.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Currently Pulp does not use the MongoDB replica set functionality correctly when the user correctly configures the replica_set setting in the [database] section of server.conf. By default replica_set is not set. If replica_set is set, then Pulp should have pyMongo use the MongoReplicaSetClient type. This will allow Pulp to rely on PyMongo to automatically negotiate who the primary is and failover in the event that the Primary changes. Currently we attempt to do this, but there is a bug caused by parameter name mismatch which causes PyMongo to ignore our setting of this option. That is the first thing to fix. Once fixed it is expected that PyMongo will know about and use the replica_set as it interacts with MongoDB.

One thing to know is that when a failover of the primary happens an AutoReconnect exception is raised. To effectively test this story you'll have to have pulp "auto retry" the operation so that the next operation will go to the newly connected to Primary. On 2.6.x this should be the default behavior, on 2.7.0 you'll need to wait for story #1 to be implemented and set unsafe_autoretry = True in server.conf

In the case where multiple seeds are specified as fixed in issue (#1012), all seeds should be iterated through until a valid MongoReplicaSetClient connection can be found. Either Pulp needs to find the primary or PyMongo needs to reconnect to the primary. Specifically test that in a multi-node mongoDB configuration with one primary, one arbiter, and one secondary with all participating in a replica_set that you can specify only the secondary as the value for seed and that the MongoReplicaSetClient will automatically discover the primary and connect to it even though its not contained in the seed listing.

If Pulp does not have replica_set configured (the default) then PyMongo needs to use the single-mongoDB style client named MongoClient.

If this is story is introduced in a version which does not use mongoengine and then merged forward you'll have to ensure that the newer version with mongoengine continues to work as expected.

This story also introduces a constant named WRITE_CONCERN which should be hard coded to 'majority'. This will be the write concern that the entire Pulp codebase will use as its sets write concern throughout bugfix #1065. You'll need to verify that this will work correctly even if replica_set is not in use and PyMongo is using the MongoClient connection style. You will also need to verify that "majority" is a valid write concern to be specified on mongoDB server 2.4.0+. Verifying that is part of this story.

If Pulp does not have replica_set configured (the default) AND there are more than one seed specified in the [database] section of server.conf then an error should be logged and Pulp should refuse to start. This is because having multiple seeds without using replica set functionality of MongoDB is likely not a correct configuration and should be fixed up by the user. They will either need to have a single seed listed or configure a replica_set.


Related issues

Blocked by Pulp - Issue #1012: If the first database seed listed in server.conf is not the current Primary replica, Pulp will not be able to write to the databaseCLOSED - CURRENTRELEASEdkliban@redhat.comActions
Blocks Pulp - Issue #1065: DeprecationWarning on pulp-manage-dbCLOSED - CURRENTRELEASEamacdona@redhat.comActions
Actions #1

Updated by bmbouter about 7 years ago

  • Parent task set to #1014
Actions #2

Updated by bmbouter about 7 years ago

  • Blocked by Issue #1012: If the first database seed listed in server.conf is not the current Primary replica, Pulp will not be able to write to the database added
Actions #3

Updated by bmbouter about 7 years ago

  • Blocks Issue #1065: DeprecationWarning on pulp-manage-db added
Actions #4

Updated by bmbouter about 7 years ago

  • Platform Release deleted (2.7.0)
Actions #5

Updated by bmbouter about 7 years ago

  • Description updated (diff)
Actions #6

Updated by jortel@redhat.com about 7 years ago

  • Triaged changed from No to Yes

Added by dkliban@redhat.com almost 7 years ago

Revision 1ad36448

Adds support for write concern

Pulp now accepts a write concern for the MongoDB connection. All writes to the database are now acknowledged. For replica sets, 'majority' or 'all' members can be asked to acknowledge the write.

closes: #974 closes: #1139

Added by dkliban@redhat.com almost 7 years ago

Revision 1ad36448

Adds support for write concern

Pulp now accepts a write concern for the MongoDB connection. All writes to the database are now acknowledged. For replica sets, 'majority' or 'all' members can be asked to acknowledge the write.

closes: #974 closes: #1139

Actions #7

Updated by dkliban@redhat.com almost 7 years ago

  • Status changed from NEW to MODIFIED
  • % Done changed from 0 to 100
Actions #8

Updated by dkliban@redhat.com almost 7 years ago

  • Platform Release set to 2.7.0
Actions #9

Updated by amacdona@redhat.com almost 7 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #10

Updated by bmbouter over 3 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF