Skip to content

Commit a7c8758

Browse files
biniona-mongodbterakilobyte
authored andcommitted
DOCSP-15183 explain heartbeat feature (#99)
Co-authored-by: Nathan <[email protected]>
1 parent 5ac35bd commit a7c8758

File tree

1 file changed

+19
-3
lines changed

1 file changed

+19
-3
lines changed

source/kafka-source.txt

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,8 @@ for more information.
2929

3030
.. note::
3131

32-
Change streams require a replicaSet or a sharded cluster using replicaSets.
32+
Change streams require a replicaSet or a sharded cluster using replicaSets. A
33+
standalone MongoDB instance cannot produce a change stream.
3334

3435
Message Delivery Guarantee
3536
--------------------------
@@ -488,7 +489,10 @@ an example source connector configuration file, see
488489
change stream. The connector sends heartbeat messages when
489490
source records are not published in the specified interval.
490491
This improves the resumability of the connector for low volume
491-
namespaces. Use ``0`` to disable.
492+
namespaces. See the
493+
:ref:`Invalid Resume Token section <source-invalid-resume-token>`
494+
of this page for more information on the problem this feature can help
495+
solve. Use ``0`` to disable.
492496
| **Default**: ``0``
493497
| **Accepted Values**: An integer
494498

@@ -716,11 +720,13 @@ The configuration shown above matches and copies existing documents that
716720
contain a field named ``totalUniqueViews`` with a value of ``5000`` or
717721
greater.
718722

723+
.. _source-invalid-resume-token:
724+
719725
How to Recover from An Invalid Resume Token
720726
-------------------------------------------
721727

722728
In the event your Connector pauses or shuts down long enough for the Change
723-
Stream resume token to expire from the
729+
Stream resume token (also referred to as the ``postBatchResumeToken``) to expire from the
724730
:manual:`MongoDB Oplog </core/replica-set-oplog/>`, you may encounter an error
725731
that prevents you from starting up the Connector. If you encounter this
726732
condition, you must delete the topic data referenced by
@@ -729,3 +735,13 @@ file referenced by the ``offset.storage.file.filename`` setting (e.g.
729735
``/tmp/connect.offsets``) if you are using the Standalone mode. After you
730736
delete the appropriate data, you should be able to start your Connector
731737
workers and listen to the change stream.
738+
739+
.. tip:: Consider Enabling Heartbeats
740+
741+
If you encounter an invalid resume token error due to your source
742+
MongoDB namespace being infrequently updated, consider enabling the heartbeat
743+
feature. This feature periodically updates the Change stream resume token to the current entry in
744+
your MongoDB Oplog if no updates occurred in your source MongoDB namespace.
745+
To learn how to enable heartbeats, see the
746+
:ref:`Source Connector Configuration Properties section <source-connector-configuration-properties>`.
747+

0 commit comments

Comments
 (0)