Skip to content

Commit 52b00b9

Browse files
mungitoperritojeff-allen-mongo
mungitoperrito
authored and
jeff-allen-mongo
committed
DOCSP-15956 reintroduce dotDollar validation
1 parent ebd4945 commit 52b00b9

File tree

11 files changed

+300
-24
lines changed

11 files changed

+300
-24
lines changed

‎source/core/crud.txt‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ Query Plan, Performance, and Analysis
2626

2727
Miscellaneous
2828
- :doc:`/core/tailable-cursors`
29+
- :doc:`/core/dot-dollar-considerations`
2930

3031
.. seealso::
3132

@@ -34,10 +35,11 @@ Miscellaneous
3435
.. toctree::
3536
:titlesonly:
3637

38+
/tutorial/analyze-query-plan
3739
/core/write-operations-atomicity
38-
/core/read-isolation-consistency-recency
3940
/core/distributed-queries
40-
/core/query-plans
41+
/core/dot-dollar-considerations
42+
/core/read-isolation-consistency-recency
4143
/core/query-optimization
42-
/tutorial/analyze-query-plan
44+
/core/query-plans
4345
/core/tailable-cursors

‎source/core/document.txt‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -75,9 +75,7 @@ Field names are strings.
7575
:doc:`Documents </core/document>` have the following restrictions on field
7676
names:
7777

78-
- The field name ``_id`` is reserved for use as a primary key; its
79-
value must be unique in the collection, is immutable, and may be of
80-
any type other than an array.
78+
- .. include:: /includes/fact-id-field-name-rules.rst
8179

8280
.. include:: /includes/fact-document-field-name-restrictions.rst
8381

@@ -95,7 +93,6 @@ existing user document.
9593
Field Value Limit
9694
~~~~~~~~~~~~~~~~~
9795

98-
9996
MongoDB 2.6 through MongoDB versions with :ref:`featureCompatibilityVersion <view-fcv>` (fCV) set to ``"4.0"`` or earlier
10097
For :doc:`indexed collections </indexes>`, the values for the
10198
indexed fields have a :limit:`Maximum Index Key Length <Index Key Limit>`. See
@@ -142,7 +139,7 @@ For examples querying arrays, see:
142139
.. seealso::
143140

144141
- :update:`$[]` all positional operator for update operations,
145-
142+
146143
- :update:`$[\<identifier\>]` filtered positional operator for update operations,
147144

148145
- :update:`$` positional operator for update operations,
@@ -188,6 +185,7 @@ For examples querying embedded documents, see:
188185

189186
- :doc:`/tutorial/query-array-of-documents/`
190187

188+
191189
Document Limitations
192190
--------------------
193191

@@ -220,6 +218,9 @@ The ``_id`` field has the following behavior and constraints:
220218
server receives a document that does not have the ``_id`` field
221219
first, then the server will move the field to the beginning.
222220

221+
_ If the ``_id`` contains subfields, the subfield names cannot begin
222+
with a (``$``) symbol.
223+
223224
- The ``_id`` field may contain values of any :doc:`BSON data type
224225
</reference/bson-types>`, other than an array, regex, or undefined.
225226

@@ -286,7 +287,7 @@ For examples, see:
286287
- :doc:`/tutorial/query-documents`
287288

288289
- :doc:`/tutorial/query-embedded-documents`
289-
290+
290291
- :doc:`/tutorial/query-arrays`
291292

292293
- :doc:`/tutorial/query-array-of-documents/`
@@ -327,7 +328,7 @@ type:
327328
Further Reading
328329
---------------
329330

330-
For more information on the MongoDB document model, download the
331+
For more information on the MongoDB document model, download the
331332
`MongoDB Application Modernization Guide
332333
<https://www.mongodb.com/modernize?tck=docs_server>`_.
333334

‎source/core/dot-dollar-considerations.txt‎

Lines changed: 228 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
====================================
2+
Field Names with (``.``) and (``$``)
3+
====================================
4+
5+
.. default-domain:: mongodb
6+
7+
.. contents:: On this page
8+
:local:
9+
:backlinks: none
10+
:depth: 1
11+
:class: singlecol
12+
13+
.. _crud-concepts-dot-dollar-considerations:
14+
15+
Overview
16+
--------
17+
18+
MongoDB 5.0 adds improved support for field names that are (``$``)
19+
prefixed or that contain (``.``) characters. The validation rules for
20+
storing data have been updated to make it easier to work with data
21+
sources that use these characters.
22+
23+
In most cases data that has been stored using field names like these
24+
is not directly accessible. You need to use helper methods like
25+
:expression:`$getField`, :expression:`$setField`, and
26+
:expression:`$literal` in queries that access those fields.
27+
28+
The field name validation rules are not the same for all types of
29+
storage operations. This page summarizes how different insert and
30+
update operations handle (``$``) prefixed field names.
31+
32+
Insert operations
33+
-----------------
34+
35+
(``$``) prefixed fields are permitted as top level and nested field
36+
names for inserts.
37+
38+
.. code-block:: javascript
39+
:emphasize-lines: 3
40+
41+
db.sales.insertOne( {
42+
"$price": 50.00,
43+
"quantity": 30
44+
} )
45+
46+
(``$``) prefixed fields are permitted on inserts using otherwise
47+
reserved words. Operator names like :update:`$inc` can be used as
48+
field names as well as words like ``id``, ``db``, and ``ref``.
49+
50+
.. code-block:: javascript
51+
:emphasize-lines: 2, 4-6
52+
53+
db.books.insertOne( {
54+
"$id": "h1961-01",
55+
"location": {
56+
"$db": "novels",
57+
"$ref": "2007042768",
58+
"$inc": true
59+
} } )
60+
61+
An update which creates a new document during an :term:`upsert` is
62+
treated as an ``insert`` rather than an ``update`` for field name
63+
validation. :term:`Upserts <upsert>` can accept (``$``) prefixed
64+
fields. However, :term:`upserts <upsert>` are a special case and
65+
similar update operations may cause an error if the ``match`` portion
66+
of the update selects an existing document.
67+
68+
This code sample has ``upsert: true`` so it will insert a new document
69+
if the collection doesn't already contain a document that matches the
70+
query term, ``{ "date": "2021-07-07" }``. If this sample code matches
71+
an existing document, the update will fail since ``$hotel`` is (``$``)
72+
prefixed.
73+
74+
.. code-block:: javascript
75+
:emphasize-lines: 5
76+
77+
db.expenses.updateOne(
78+
{ "date": "2021-07-07" },
79+
{ $set: {
80+
"phone": 25.17,
81+
"$hotel": 320.10
82+
} },
83+
{ upsert: true }
84+
)
85+
86+
Document Replacing Updates
87+
--------------------------
88+
89+
Update operators either replace existing fields with new documents
90+
or else modify those fields. In cases where the update performs a
91+
replacement, (``$``) prefixed fields are not permitted as top level
92+
field names.
93+
94+
Consider a document like
95+
96+
.. code-block:: javascript::
97+
98+
{
99+
"_id": "E123",
100+
"address": {
101+
"$number": 123,
102+
"$street": "Elm Road"
103+
},
104+
"$rooms": {
105+
"br": 2,
106+
"bath": 1
107+
}
108+
}
109+
110+
You could use an update operator that replaces an existing document to
111+
modify the ``address.$street`` field but you could not update the
112+
``$rooms`` field that way.
113+
114+
.. code-block::
115+
116+
db.housing.updateOne(
117+
{ "_id": "E123" },
118+
{ $set: { "address.$street": "Elm Ave" } }
119+
)
120+
121+
Use :expression:`$setField` as part of an aggregation pipeline to
122+
:ref:`update top level <dotDollar-aggregate-update>` (``$``) prefixed
123+
fields like ``$rooms``.
124+
125+
Document Modifying Updates
126+
--------------------------
127+
128+
When an update modifies, rather than replaces, existing document
129+
fields, (``$``) prefixed fields can be top level field names.
130+
Subfields can be accessed directly, but you need a helper method to
131+
access the top level fields.
132+
133+
.. seealso::
134+
135+
:expression:`$getField`, :expression:`$setField`,
136+
:expression:`$literal`, :pipeline:`$replaceWith`
137+
138+
Consider a collection with documents like this inventory record:
139+
140+
.. code-block::
141+
:copyable: false
142+
143+
{
144+
_id: ObjectId("610023ad7d58ecda39b8d161"),
145+
"part": "AB305",
146+
"$bin": 200,
147+
"quantity": 100,
148+
"pricing": { sale: true, "$discount": 60 }
149+
}
150+
151+
The ``pricing.$discount`` subfield can be queried directly.
152+
153+
.. code-block::
154+
155+
db.inventory.findAndModify( {
156+
query: { "part": { $eq: "AB305" } },
157+
update: { $inc: { "pricing.$discount": 10 } }
158+
} )
159+
160+
161+
Use :expression:`$getField` and :expression:`$literal` to access the
162+
value of the top level ``$bin`` field.
163+
164+
.. code-block::
165+
:emphasize-lines: 3
166+
167+
db.inventory.findAndModify( {
168+
query: { $expr: {
169+
$eq: [ { $getField: { $literal: "$bin" } }, 200 ]
170+
} },
171+
update: { $inc: { "quantity": 10 } }
172+
} )
173+
174+
.. _dotDollar-aggregate-update:
175+
176+
Updates Using Aggregation Pipelines
177+
-----------------------------------
178+
179+
Use :expression:`$setField`, :expression:`$getField`, and
180+
:expression:`$literal` in the :pipeline:`$replaceWith` stage to modify
181+
(``$``) prefixed fields in an aggregation :term:`pipeline`.
182+
183+
Consider a collection of school records like:
184+
185+
.. code-block:: javascript
186+
:copyable: false
187+
188+
{
189+
"_id": 100001,
190+
"$term": "fall",
191+
"registered": true,
192+
"grade": 4
193+
}
194+
195+
Create a new collection for the spring semester using a
196+
:term:`pipeline` to update the (``$``) prefixed ``$term`` field.
197+
198+
.. code-block:: javascript
199+
:emphasize-lines: 3-5
200+
201+
db.school.aggregate( [
202+
{ $match: { "registered": true } },
203+
{ $replaceWith: {
204+
$setField: {
205+
field: { $literal: "$term" },
206+
input: "$$ROOT",
207+
value: "spring"
208+
} } },
209+
{ $out: "spring2022" }
210+
] )
211+
212+
General Restrictions
213+
--------------------
214+
215+
In addition to the storage validation rules above, there are some
216+
general restrictions on using (``$``) prefixed field names. These
217+
fields cannot:
218+
219+
- Be indexed
220+
- Be used as part of a shard key
221+
- Be validated using :query:`$jsonSchema`
222+
- Be be modified with an escape sequence
223+
- Be used with
224+
:driver:`Field Level Encryption </security/client-side-field-level-encryption-guide>`
225+
- Be used as a subfield in an ``_id`` document
226+
227+
.. include:: /includes/warning-possible-data-loss.rst
228+

‎source/includes/fact-document-field-name-restrictions.rst‎

Lines changed: 5 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,9 @@
11
- Field names **cannot** contain the ``null`` character.
22

3-
- Top-level field names **cannot** start with the dollar sign (``$``) character.
3+
- The server permits storage of field names that contain dots (``.``)
4+
and dollar signs (``$``).
45

5-
Otherwise, starting in MongoDB 3.6, the server permits storage of
6-
field names that contain dots (i.e. ``.``) and dollar signs (i.e.
7-
``$``).
6+
- MongodB 5.0 adds improved support for the use of (``$``) and (``.``)
7+
in field names. There are some restrictions. See
8+
:ref:`Field Name Considerations <crud-concepts-dot-dollar-considerations>` for more details.
89

9-
.. important::
10-
11-
The MongoDB Query Language cannot always meaningfully express
12-
queries over documents whose field names contain these characters
13-
(see :issue:`SERVER-30575`).
14-
15-
Until support is added in the query language, the use of ``$`` and
16-
``.`` in field names is not recommended and is not supported by
17-
the official MongoDB drivers.

‎source/includes/fact-id-field-name-rules.rst‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
The field name ``_id`` is reserved for use as a primary key; its value
2+
must be unique in the collection, is immutable, and may be of any type
3+
other than an array. If the ``_id`` contains subfields, the subfield
4+
names cannot begin with a (``$``) symbol.
5+

‎source/includes/warning-possible-data-loss.rst‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
.. warning::
2+
3+
There is a small chance of data loss when using (``$``) prefixed
4+
field names or field names with (``.``) characters when this type of
5+
field name is used in conjunction with unacknowledged writes
6+
(:doc:`write concern </reference/write-concern>` ``w=0``) on servers
7+
that are older than MongoDB 5.0.
8+
9+
When running :doc:`insert </tutorial/insert-documents>`,
10+
:doc:`update </tutorial/insert-documents>`, and
11+
:doc:`findAndModify </reference/method/db.collection.findAndModify>`
12+
commands, drivers that are 5.0 compatible remove restrictions on
13+
using documents with field names that are (``$``) prefixed or that
14+
contain (``.``) characters. These field names generated a
15+
client-side error in earlier driver versions.
16+
17+
The restrictions are removed regardless of the server version the
18+
driver is connected to. If a 5.0 driver sends a document to an older
19+
server, the document will be rejected without sending an error.
20+

‎source/reference/command/findAndModify.txt‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -457,6 +457,10 @@ Comparisons with the ``update`` Method
457457

458458
.. include:: /includes/fact-findAndModify-update-comparison.rst
459459

460+
.. seealso ::
461+
462+
:ref:`Considerations for field names <crud-concepts-dot-dollar-considerations>`
463+
460464
Transactions
461465
~~~~~~~~~~~~
462466

@@ -474,7 +478,6 @@ Write Concerns and Transactions
474478

475479
.. include:: /includes/extracts/transactions-operations-write-concern.rst
476480

477-
478481
Examples
479482
--------
480483

‎source/reference/limits.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,8 +93,14 @@ Naming Restrictions
9393

9494
.. include:: /includes/fact-document-field-name-restrictions.rst
9595

96+
.. include:: /includes/warning-possible-data-loss.rst
97+
9698
.. include:: /includes/warning-document-duplicate-key-names.rst
9799

100+
.. limit:: Restrictions on ``_id``
101+
102+
.. include:: /includes/fact-id-field-name-rules.rst
103+
98104
.. _faq-dev-namespace:
99105

100106
Namespaces

0 commit comments

Comments
 (0)