(DOCSP-24007) Improve sbe performance (#3245)

jeff-allen-mongo · RuoxinXu · web-flow · commit 5f1eb5abb826 · 2023-05-30T20:02:10.000-04:00
* (DOCSP-29528): Improve SBE release notes (#3068) (#3083) * (DOCSP-29528): Improve SBE release notes * wording tweak * wording * change section title * (DOCSP-29529): Update log messages for SBE 7.0 changes (#3224) * (DOCSP-29529): Update log messages for SBE 7.0 changes * line spacing * copy review edits * wording * update release version * DOCSP-29530 Adjust $planCacheStats output for SBE (#3201) * formatting tabs * fix build error * formatting tab title * add more sbe content * fill out SBE table * try to fix linting * edit examples * remove version added for SBE * add version changed * fix formatting for code block * single sourcing * fix ref errors * wording * remove 'i.e.' * format field names in table * fix spacing for example comments * address review comments * Update source/includes/planCacheStats/indexFilterSet.rst Co-authored-by: Ruoxin Xu <395794845@qq.com> * fix cachedPlan link * wording --------- Co-authored-by: Ruoxin Xu <395794845@qq.com> --------- Co-authored-by: Ruoxin Xu <395794845@qq.com>
diff --git a/snooty.toml b/snooty.toml
@@ -247,6 +247,9 @@ windows-sha256 = "C777DF7816BB8C9A760FDEA782113949408B6F39D72BE29A2551FA51E2FE04
 source-available = ":github:`source available and free to use <mongodb/mongo>`"
 pcre-abbr = ":abbr:`PCRE (Perl Compatible Regular Expressions)`"
 pcre = "Perl Compatible Regular Expressions"
+sbe = "slot-based query execution engine"
+sbe-short = "slot-based engine"
+sbe-title = "Slot-Based Query Execution Engine"
 fcv = ":abbr:`fCV (feature compatibility version)`"
 fts = "Atlas Search"
 # In Use Encryption Constants
diff --git a/source/includes/planCacheStats/cachedPlan.rst b/source/includes/planCacheStats/cachedPlan.rst
@@ -0,0 +1,4 @@
+The details of the cached plan. The fields included in the
+``cachedPlan`` vary based on whether the query was completed using the
+classic engine or the {+sbe+}. For more information on query plans, see
+:data:`explain.queryPlanner`.
diff --git a/source/includes/planCacheStats/estimatedSizeBytes.rst b/source/includes/planCacheStats/estimatedSizeBytes.rst
@@ -0,0 +1 @@
+The estimated size in bytes of a plan cache entry.
diff --git a/source/includes/planCacheStats/host.rst b/source/includes/planCacheStats/host.rst
@@ -0,0 +1,8 @@
+The hostname and port of the :binary:`~bin.mongod` instance from which
+the plan cache information was returned.
+
+When run on a sharded cluster, the operation returns plan cache entry
+information from a single member in each shard replica set. This member
+is identified with the :ref:`shard <plancachestats-shard>` and
+:ref:`host <plancachestats-host>` fields. See also
+:ref:`plancachestats-read-pref`.
diff --git a/source/includes/planCacheStats/indexFilterSet.rst b/source/includes/planCacheStats/indexFilterSet.rst
@@ -0,0 +1,2 @@
+A boolean that indicates whether an :ref:`index filter
+<index-filters>` exists for the query shape.
diff --git a/source/includes/planCacheStats/isActive.rst b/source/includes/planCacheStats/isActive.rst
@@ -0,0 +1,9 @@
+A boolean that indicates whether the entry is active or inactive.
+          
+- If active, the query planner is currently using the entry to generate
+  query plans.
+
+- If inactive, the query planner is not currently using the entry to
+  generate query plans.
+  
+See :ref:`cache-entry-state`.
diff --git a/source/includes/planCacheStats/planCacheKey.rst b/source/includes/planCacheStats/planCacheKey.rst
@@ -0,0 +1,4 @@
+A hexadecimal string that represents the hash of the key used to find
+the plan cache entry associated with this query. The plan cache key is a
+function of both the query shape and the currently available indexes for
+that shape. See :data:`explain.queryPlanner.planCacheKey`.
diff --git a/source/includes/planCacheStats/queryHash.rst b/source/includes/planCacheStats/queryHash.rst
@@ -0,0 +1,2 @@
+A hexadecimal string that represents the hash of the :term:`query
+shape`. See :data:`explain.queryPlanner.queryHash`.
diff --git a/source/includes/planCacheStats/version.rst b/source/includes/planCacheStats/version.rst
@@ -0,0 +1,5 @@
+A number that indicates the query engine used to complete the query.
+
+- ``1`` indicates that the classic engine was used.
+
+- ``2`` indicates that the :ref:`{+sbe+} <5.1-rel-notes-sbe>` was used.
diff --git a/source/includes/planCacheStats/works.rst b/source/includes/planCacheStats/works.rst
@@ -0,0 +1,4 @@
+The number of "work units" performed by the query execution plan during
+the trial period when the query planner evaluates candidate plans. For
+more information, see
+:data:`explain.executionStats.executionStages.works`.
diff --git a/source/reference/command/planCacheClear.txt b/source/reference/command/planCacheClear.txt
@@ -76,7 +76,7 @@ The command takes the following optional fields:
  
  
 To see the query shapes for which cached query plans exist, see
-:ref:`planCacheStats-list-query-shapes`.
+:ref:`planCacheStats-examples`.
 
 Required Access
 ---------------
diff --git a/source/reference/log-messages.txt b/source/reference/log-messages.txt
@@ -1009,23 +1009,24 @@ Starting in MongoDB 5.0, :ref:`slow operation
 <database-profiling-specify-slowms-threshold>` log messages include a 
 ``remote`` field specifying client IP address.
 
+Starting in MongoDB 6.2, slow operation log messages include a
+``queryFramework`` field that indicates which query engine completed the
+query:
+
+- ``queryFramework: "classic"`` indicates that the classic engine
+  completed the query.
+
+- ``queryFramework: "sbe"`` indicates that the {+sbe+} completed the
+  query.
+
 Starting in MongoDB 6.3, :ref:`slow operation
 <database-profiling-specify-slowms-threshold>` log messages and :ref:`database 
 profiler <profiler>` entries include a ``cpuNanos`` field that specifies the 
 total CPU time spent by a query operation in nanoseconds. The ``cpuNanos`` field 
 is only available on Linux systems.
 
-The following example output includes information about
-a slow :doc:`aggregation </aggregation>` operation:
-
-.. code-block:: javascript
-   :copyable: false
-
-   {"t":{"$date":"2020-05-20T20:10:08.731+00:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn281","msg":"Slow query","attr":{"type":"command","ns":"stocks.trades","appName":"MongoDB Shell","command":{"aggregate":"trades","pipeline":[{"$project":{"ticker":1.0,"price":1.0,"priceGTE110":{"$gte":["$price",110.0]},"_id":0.0}},{"$sort":{"price":-1.0}}],"allowDiskUse":true,"cursor":{},"lsid":{"id":{"$uuid":"fa658f9e-9cd6-42d4-b1c8-c9160fabf2a2"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1590005405,"i":1}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"$db":"test"},"planSummary":"COLLSCAN","cursorid":1912190691485054730,"keysExamined":0,"docsExamined":1000001,"hasSortStage":true,"usedDisk":true,"numYields":1002,"nreturned":101,"reslen":17738,"locks":{"ReplicationStateTransition":{"acquireCount":{"w":1119}},"Global":{"acquireCount":{"r":1119}},"Database":{"acquireCount":{"r":1119}},"Collection":{"acquireCount":{"r":1119}},"Mutex":{"acquireCount":{"r":117}}},"storage":{"data":{"bytesRead":232899899,"timeReadingMicros":186017},"timeWaitingMicros":{"cache":849}},"cpuNanos":22426151000,"remote": "192.168.14.15:37666","protocol":"op_msg","durationMillis":22427}}
-
-See the :ref:`examples section <log-message-json-examples>` for a
-:ref:`pretty-printed <log-message-pretty-printing>` version of this log
-entry.
+For a :ref:`pretty-printed <log-message-pretty-printing>` example of a
+slow operation log entry, see :ref:`log-message-json-examples`.
 
 .. _log-messages-remoteOpWaitMillis:
 
@@ -1363,116 +1364,59 @@ This example shows a :ref:`slow operation message
 
    {
      "t": {
-       "$date": "2020-05-20T20:10:08.731+00:00"
+       "$date": "2023-05-24T13:46:27.853+00:00"
      },
      "s": "I",
      "c": "COMMAND",
      "id": 51803,
-     "ctx": "conn281",
+     "ctx": "conn3",
      "msg": "Slow query",
      "attr": {
        "type": "command",
-       "ns": "stocks.trades",
+       "ns": "db.coll",
        "appName": "MongoDB Shell",
        "command": {
-         "aggregate": "trades",
-         "pipeline": [
-           {
-             "$project": {
-               "ticker": 1,
-               "price": 1,
-               "priceGTE110": {
-                 "$gte": [
-                   "$price",
-                   110
-                 ]
-               },
-               "_id": 0
-             }
-           },
-           {
-             "$sort": {
-               "price": -1
-             }
-           }
-         ],
-         "allowDiskUse": true,
-         "cursor": {},
+         "find": "coll",
+         "filter": {
+           "b": -1
+         },
          "lsid": {
            "id": {
-             "$uuid": "fa658f9e-9cd6-42d4-b1c8-c9160fabf2a2"
-           }
-         },
-         "$clusterTime": {
-           "clusterTime": {
-             "$timestamp": {
-               "t": 1590005405,
-               "i": 1
-             }
-           },
-           "signature": {
-             "hash": {
-               "$binary": {
-                 "base64": "AAAAAAAAAAAAAAAAAAAAAAAAAAA=",
-                 "subType": "0"
-               }
-             },
-             "keyId": 0
+             "$uuid": "5d50b19c-8559-420a-a122-8834e012274a"
            }
          },
-         "$db": "test"
+         "$db": "db"
        },
        "planSummary": "COLLSCAN",
-       "cursorid": 1912190691485054700,
+       "planningTimeMicros": 228,
        "keysExamined": 0,
-       "docsExamined": 1000001,
-       "hasSortStage": true,
-       "usedDisk": true,
-       "mirrored": false,
-       "numYields": 1002,
-       "nreturned": 101,
-       "queryFramework": "classic"
-       "reslen": 17738,
+       "docsExamined": 232922,
+       "fromPlanCache": true,
+       "nBatches": 1,
+       "cursorExhausted": true,
+       "numYields": 232,
+       "nreturned": 0,
+       "queryHash": "9C05019A",
+       "planCacheKey": "C41063D6",
+       "queryFramework": "sbe",
+       "reslen": 96,
        "locks": {
-         "ReplicationStateTransition": {
+         "FeatureCompatibilityVersion": {
            "acquireCount": {
-             "w": 1119
+             "r": 233
            }
          },
          "Global": {
            "acquireCount": {
-             "r": 1119
+             "r": 233
            }
-         },
-         "Database": {
-           "acquireCount": {
-             "r": 1119
-           }
-         },
-         "Collection": {
-           "acquireCount": {
-             "r": 1119
-           }
-         },
-         "Mutex": {
-           "acquireCount": {
-             "r": 117
-           }
-         }
-       },
-       "storage": {
-         "data": {
-           "bytesRead": 232899899,
-           "timeReadingMicros": 186017
-         },
-         "timeWaitingMicros": {
-           "cache": 849
          }
        },
-       "cpuNanos": 22426151000,
-       "remote": "192.168.14.15:37666",
+       "storage": {},
+       "cpuNanos": 111410254,
+       "remote": "127.0.0.1:47150",
        "protocol": "op_msg",
-       "durationMillis": 22427
+       "durationMillis": 111
      }
    }
 
diff --git a/source/reference/method/PlanCache.clearPlansByQuery.txt b/source/reference/method/PlanCache.clearPlansByQuery.txt
@@ -71,7 +71,7 @@ Definition
 
 
    To see the query shapes for which cached query plans exist, see
-   :ref:`planCacheStats-list-query-shapes`.
+   :ref:`planCacheStats-examples`.
 
 Required Access
 ---------------
diff --git a/source/reference/operator/aggregation/planCacheStats.txt b/source/reference/operator/aggregation/planCacheStats.txt
diff --git a/source/release-notes/4.2-compatibility.txt b/source/release-notes/4.2-compatibility.txt
diff --git a/source/release-notes/7.0.txt b/source/release-notes/7.0.txt

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1 @@`
	`1`	`+The estimated size in bytes of a plan cache entry.`
Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,2 @@`
	`1`	+A boolean that indicates whether an :ref:`index filter
	`2`	+<index-filters>` exists for the query shape.
Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,2 @@`
	`1`	+A hexadecimal string that represents the hash of the :term:`query
	`2`	+shape`. See :data:`explain.queryPlanner.queryHash`.