44Copy Existing Data
55==================
66
7- This usage example demonstrates how to copy data from a MongoDB collection to an
7+ .. facet::
8+ :name: genre
9+ :values: reference
10+
11+ .. meta::
12+ :keywords: code example, copy, filter, configuration setting
13+
14+ .. contents:: On this page
15+ :local:
16+ :backlinks: none
17+ :depth: 2
18+ :class: singlecol
19+
20+ These usage examples demonstrate how to copy data from MongoDB to an
821{+kafka+} topic using the {+source-connector+}.
922
10- Example
11- -------
23+ Examples
24+ --------
25+
26+ The following examples show how to configure your source connector to
27+ copy existing data from a single collection or from multiple collections.
28+
29+ Copy and Filter Collection Data
30+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1231
1332Suppose you want to copy a MongoDB collection to {+kafka+} and filter some data.
1433
@@ -51,7 +70,7 @@ The ``customers`` collection contains the following documents:
5170.. _source-usage-example-copy-existing-data-copy-data:
5271
5372Copy Data
54- ~~~~~~~~~
73+ `````````
5574
5675Copy the contents of the ``customers`` collection of the ``shopping`` database by
5776specifying the following configuration options in your source connector:
@@ -78,7 +97,7 @@ To learn more about the ``startup.mode`` option, see
7897.. _source-usage-example-copy-existing-data-mask-data:
7998
8099Filter Data
81- ~~~~~~~~~~~
100+ ```````````
82101
83102You can filter data by specifying an aggregation pipeline in the
84103``startup.mode.copy.existing.pipeline`` option of your source connector configuration. The
@@ -97,9 +116,10 @@ To learn more about aggregation pipelines, see the following resources:
97116- :ref:`<source-usage-example-custom-pipeline>` Usage Example
98117- :manual:`Aggregation </aggregation>` in the MongoDB manual.
99118
119+ .. _source-usage-example-copy-existing-data-config:
100120
101121Specify the Configuration
102- ~~~~~~~~~~~~~~~~~~~~~~~~~
122+ `````````````````````````
103123
104124Your final source connector configuration to copy the ``customers`` collection should
105125look like this:
@@ -116,10 +136,47 @@ in the ``shopping.customers`` {+kafka+} topic:
116136.. literalinclude:: /includes/usage-examples/copy/payload.json
117137 :language: json
118138 :copyable: false
119- :emphasize-lines: 6,7
120139
121140.. note:: Write the Data in your Topic into a Collection
122141
123142 Use a change data capture handler to convert change event documents in an
124143 {+kafka+} topic into MongoDB write operations. To learn more, see the
125144 :ref:`Change Data Capture Handlers <sink-fundamentals-cdc-handler>` guide.
145+
146+ Copy Data From Multiple Sources
147+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
148+
149+ Suppose you want to copy data from another collection in the ``shopping``
150+ database named ``products``, which contains the following document:
151+
152+ .. code-block:: json
153+ :copyable: false
154+
155+ {
156+ "_id": 1,
157+ "item_name": "lipstick",
158+ "department": "cosmetics",
159+ "quantity": 45
160+ }
161+
162+ You can copy from both the ``customers`` and ``products`` collections
163+ by using the ``startup.mode.copy.existing.namespace.regex`` configuration
164+ setting, as shown in the following code:
165+
166+ .. literalinclude:: /includes/usage-examples/copy/multisource.properties
167+ :language: properties
168+ :emphasize-lines: 5
169+
170+ In addition to the change event document in the ``shopping.customers`` {+kafka+} topic,
171+ described in the :ref:`preceding section <source-usage-example-copy-existing-data-config>`,
172+ you can see the following document in the ``shopping.products`` topic:
173+
174+ .. literalinclude:: /includes/usage-examples/copy/multisource.json
175+ :language: json
176+ :copyable: false
177+
178+ .. tip::
179+
180+ To learn more about the ``startup.mode.copy.existing.namespace.regex`` setting,
181+ see the :ref:`Settings table <source-configuration-startup-table-start>` in
182+ the Startup Properties guide.
0 commit comments