DOCS-14938 document warning (#194)

* DOCS-14938 document rollback warning

* Better ref link

* Typo

* Review feedback

* Review feedback

* Add scenarios

* DOCS-14938 document warning

* DOCS-14938 document warning

Author: Dave

source/core/replica-set-arbiter.txt

@@ -15,13 +15,14 @@ Replica Set Arbiter
 
 .. start-content
 
-In some circumstances (such as you have a primary and a secondary but
-cost constraints prohibit adding another secondary), you may choose to
-add an arbiter to your replica set. An arbiter does **not** have a copy
-of the data set and **cannot** become a primary. However, an arbiter
-participates in :ref:`elections for primary <replica-set-elections>`.
-By default an arbiter has priority ``0``. An arbiter has exactly ``1``
-election vote.
+In some circumstances (such as when you have a primary and a secondary,
+but cost constraints prohibit adding another secondary), you may choose
+to add an arbiter to your replica set. An arbiter participates in
+:ref:`elections for primary <replica-set-elections>` but an arbiter does
+**not** have a copy of the data set and **cannot** become a primary.
+
+An arbiter has exactly ``1`` election vote. By default an arbiter has
+priority ``0``.
 
 .. important::
 
@@ -37,19 +38,6 @@ Release Version Considerations
 
 .. include:: /includes/5.1/fact-do-not-use-with-quarterlies.rst
 
-Example
--------
-
-.. start-content-even-votes-example
-
-For example, in the following replica set with a 2 data bearing members
-(the primary and a secondary), an arbiter allows the set to have an odd
-number of votes to break a tie:
-
-.. include:: /images/replica-set-primary-with-secondary-and-arbiter.rst
-
-.. end-content-even-votes-example
-
 Performance Issues with PSA replica sets
 ----------------------------------------
 
@@ -60,6 +48,35 @@ Replica Set Protocol Version and Arbiter
 
 .. include:: /includes/extracts/arbiters-and-pvs-with-reference.rst
 
+.. _rollbacks-multi-arbiters:
+
+Concerns with Multiple Arbiters
+-------------------------------
+
+Use a single arbiter to avoid problems with data consistency. Multiple
+arbiters prevent the reliable use of the majority write concern.
+
+To ensure that a write will persist after the failure of a primary node,
+the majority write concern requires a majority of nodes to acknowledge
+a write operation. Arbiters do not store any data, but they do
+contribute to the number of nodes in a replica set. When a replica set
+has multiple arbiters it is less likely that a majority of data bearing
+nodes will be available after a node failure.
+
+For more information, see this `Knowledge Base
+<https://kb.corp.mongodb.com/article/000019126/>`__ article on multiple
+arbiters.
+
+.. warning::
+
+   If a secondary node falls behind the primary, and the cluster is
+   :method:`reconfigured <rs.reconfig()>`, votes from multiple arbiters
+   can elect the node that had fallen behind. The new primary will not
+   have the unreplicated writes even though the writes could have been
+   majority committed by the old configuration. The result is data
+   loss.
+
+   To avoid this scenario, use at most a single arbiter.
 
 Security
 --------
@@ -91,3 +108,16 @@ exchanges are not encrypted.
 As with all MongoDB components, run arbiters in trusted network
 environments.
 
+Example
+-------
+
+.. start-content-even-votes-example
+
+For example, in the following replica set with 2 data-bearing members
+(the primary and a secondary), an arbiter allows the set to have an odd
+number of votes to break a tie:
+
+.. include:: /images/replica-set-primary-with-secondary-and-arbiter.rst
+
+.. end-content-even-votes-example
+

source/includes/admonition-multiple-arbiters.rst

@@ -1,3 +1,6 @@
 .. warning:: 
 
-   In general, avoid deploying more than one arbiter per replica set.
+   Avoid deploying more than one arbiter per replica set.
+
+   See also: :ref:`rollbacks-multi-arbiters`
+