DOCSP-41999-mongosync-disaster-recovery (#377) (#414)

* DOCSP-41999-mongosync-dr

* add redirect + build errors

* build error

* rebase

* add 1.8 to redirects

* remove output.txt

* MvK feedback

* *

* *

* move considerations section

* typo

* MvK feedback (2/2)

* more external review

* faq edits

* *

* MvK feedback (2/2)

* more feedback

* *

* add table

* *

* formatting

* *

* *

* *

* wording

* *

* title

* *

* nit

* change uptodate to consisten

Author: ajhuh-mdb

config/redirects

@@ -1,6 +1,6 @@
 define: prefix docs/cluster-to-cluster-sync
 define: base https://www.mongodb.com/docs/cluster-to-cluster-sync
-define: versions v0.9 master
+define: versions v0.9 v1.7 v1.8 master
 
 # raw: <source file> -> ${base}/<destination>
 
@@ -51,3 +51,6 @@ raw: ${prefix}/current/installation/install-on-windows/ -> ${base}/current/
 [v0.9]: ${prefix}/${version}/release-notes/1.0 -> ${base}/${version}/release-notes/
 [v0.9]: ${prefix}/${version}/release-notes/1.1 -> ${base}/${version}/release-notes/
 [v0.9]: ${prefix}/${version}/release-notes/1.2 -> ${base}/${version}/release-notes/
+
+# DOCSP-41999 redirect disaster recovery to mongosync behavior page
+[*]: ${prefix}/${version}/reference/disaster-recovery -> ${base}/${version}/about-mongosync/#considerations-for-continuous-sync

source/faq.txt

@@ -20,20 +20,18 @@ Support.
 Can I perform reads or writes to my destination cluster while ``mongosync`` is syncing?
 ---------------------------------------------------------------------------------------
 
-You can perform reads during synchronization if ``canCommit`` is ``true``. 
-However, the data that you read is 
-:term:`eventually consistent <eventual consistency>`, meaning that you might not 
-always read the latest writes.
+You can perform reads during synchronization at any time. However, the data 
+that you read is :term:`eventually consistent <eventual consistency>`. For 
+consistent reads, wait for the migration to commit. To learn more, see 
+:ref:`mongosync-considerations`.
 
-During an ongoing sync, you can write to any non-synced namespaces in the 
-destination cluster as long as the source cluster doesn’t include a namespace 
-with the same name. If you write to a synced namespace before issuing a 
-:ref:`commit <c2c-api-commit>` and while ``canWrite`` is ``false``, the behavior 
-is undefined. To avoid this undefined behavior, you can enable 
-:ref:`write blocking <c2c-dr-write-blocking>`.
+If you write to a synced namespace before issuing a :ref:`commit 
+<c2c-api-commit>` and while ``canWrite`` is ``false``, the behavior is 
+undefined. To ensure that you don't write to any synced namespaces,  denable 
+:ref:`write blocking <c2c-write-blocking>`.
 
-To check the value of ``canCommit`` or ``canWrite``, call the :ref:`progress 
-<c2c-api-progress>` API endpoint.
+To check the value of ``canWrite``, call the :ref:`progress <c2c-api-progress>` 
+API endpoint.
 
 Can ``mongosync`` run on its own hardware? 
 ------------------------------------------
@@ -86,10 +84,10 @@ Which security and authentication options are supported?
 authentication options, see :ref:`authentication`.
 
 
-Can I configure ``mongosync`` for high availability?
-----------------------------------------------------
+Does ``mongosync`` automatically restart on error?
+--------------------------------------------------
 
-There is no automatic failover built into ``mongosync``. However you
+``mongosync`` does not automatically restart on error. However, you
 can write a script or use your operating system's process managers,
 ``systemd`` for example, to restart the ``mongosync`` process. 
 

source/reference.txt

@@ -22,10 +22,6 @@ Reference
 :ref:`c2c-oplog-sizing`
    Increase the size of the oplog on the source cluster for large data
    sets. 
-
-:ref:`c2c-failure-recovery`
-   Prepare for disaster recovery using :ref:`connections
-   <c2c-connecting>` between clusters.
 
 :ref:`c2c-limitations`
    Limitations of ``mongosync``. 
@@ -57,7 +53,6 @@ Reference
    mongosync States </reference/mongosync-states>
    Filtered Sync </reference/collection-level-filtering>
    oplog Sizing </reference/oplog-sizing>
-   Disaster Recovery </reference/disaster-recovery>
    Limitations </reference/limitations>
    Logging </reference/logging>
    User Permissions </reference/permissions>

source/reference/api/commit.txt

@@ -153,30 +153,29 @@ characteristics that ``mongosync`` alters during the synchronization process.
 
    * - Collection Characteristic
      - Impact of ``commit`` 
-   * - :ref:`Unique Indexes <c2c-dr-unique>`
+   * - Unique Indexes
      - ``commit`` resets unique indexes that ``mongosync`` replicated as
        non-unique on the destination cluster.
 
-   * - :ref:`TTL Indexes <c2c-dr-ttl>`
+   * - TTL Indexes
      - ``commit`` resets ``expireAfterSeconds`` which ``mongosync`` set
        to the value of ``MAX_INT`` on the destination cluster.
 
-   * - :ref:`Hidden Indexes <c2c-dr-hidden>`
+   * - Hidden Indexes
      - ``commit`` resets hidden indexes that ``mongosync``
        replicated as non-hidden on the destination cluster.
 
-   * - :ref:`Write Blocking <c2c-dr-write-blocking>`
+   * - Write Blocking
      - If you enable write-blocking, ``mongosync`` blocks writes: 
 
        - On the destination cluster during sync.
        - On the source cluster when ``commit`` is received.
 
        .. seealso:: 
 
-          - :ref:`c2c-failure-recovery`
-          - :ref:`c2c-write-blocking`
+          :ref:`c2c-write-blocking`
 
-   * - :ref:`Capped Collections <c2c-dr-capped-collections>`
+   * - Capped Collections
      - ``commit`` resets the required maximum size of capped collections
        which ``mongosync`` set to the maximum allowable size on the
        destination cluster.

source/reference/disaster-recovery.txt

@@ -133,23 +133,92 @@ Read Preference
 connecting to the source and destination clusters. For more information,
 see :ref:`connections-read-preference`. 
 
+.. _mongosync-considerations:
+
+Considerations for Continuous Sync
+----------------------------------
+
+For any continuous synchronization use cases with ``mongosync``, ensure that 
+``mongosync`` commits before cutting over from the source to the 
+destination.
+
+If the source cluster shuts down before ``mongosync`` can commit, such as in 
+a disaster scenario, the destination cluster might not have a consistent 
+snapshot of the source data. To learn more, see :ref:`c2c-behavior-consistency`.
+
+.. note:: 
+  
+   After commit, you can't resume continuous sync between two clusters since 
+   ``mongosync`` can only sync into empty destination clusters. If you need to 
+   use the same two clusters after cutover, you can call the 
+   :ref:`c2c-api-reverse` endpoint to keep the clusters in sync.
+   Otherwise, start a new continuous sync operation by using a new empty 
+   destination cluster. 
+
+Sharded Clusters 
+~~~~~~~~~~~~~~~~
+
+When running continuous sync with sharded clusters, you must stop 
+:ref:`balancers <sharding-balancing>` on both the source and the destination 
+for the lifetime of the sync until commit.
+
+Temporary Changes to Collection Characteristics
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+``mongosync`` temporarily alters the following collection characteristics during 
+synchronization. The original values are restored during the commit process.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Change
+     - Description
+  
+   * - Unique Indexes
+     - Unique indexes on the source cluster are synced as non-unique indexes 
+       on the destination cluster.
+
+   * - TTL Indexes
+     - Synchronization sets ``expireAfterSeconds`` to the value of ``MAX_INT``
+       on the destination cluster.
+
+   * - Hidden Indexes
+     - Synchronization replicates hidden indexes as non-hidden.
+
+   * - Write Blocking
+     - If you enable write-blocking, ``mongosync`` blocks writes: 
+
+       - On the destination cluster during sync.
+       - On the source cluster when ``commit`` is received.
+      
+       To learn more, see :ref:`c2c-write-blocking`.
+
+   * - Capped Collections
+     - Synchronization sets capped collections to the maximum allowable
+       size. 
+
+   * - Dummy Indexes 
+     - In some cases, synchronization may create dummy indexes on the 
+       destination to support writes on sharded or collated collections. 
+
 Destination Clusters
 --------------------
 
+.. _c2c-behavior-consistency:
+
 Consistency
 ~~~~~~~~~~~
 
 ``mongosync`` supports :term:`eventual consistency` on the destination
-cluster. Read Consistency is not guaranteed on the destination cluster,
-the source and destination clusters may differ at a given point in
-time. However, if new writes are halted on the source cluster, the
-source and destination clusters will eventually hold the same data.
+cluster. Read consistency is not guaranteed on the destination cluster until 
+commit. Before committing, the source and destination clusters may differ at a 
+given point in time. To learn more, see :ref:`mongosync-considerations`.
 
-Individual writes may be reordered or combined as they are relayed from
-source to destination. For a given document, the total number of writes
-may differ between source and destination.
+While ``mongosync`` is syncing, ``mongosync`` may reorder or combine writes 
+as it relays them from source to destination. For a given document, the total 
+number of writes may differ between source and destination.
 
-Transactions may appear non-atomically on the destination cluster.
+Transactions might not appear atomically on the destination cluster.
 Retryable writes may not be retryable on the destination cluster.
 
 Profiling