(DOCSP-5011): Filtering updates for numbers and strings (#168)

* (DOCSP-5011): Filtering updates for numbers and strings

* Adding blurb for all other strings

* Updates per Kay's feedback

* Updates per Tom's feedback

* Updates per Tom's feedback

Author: jeff-allen-mongo

source/filter-documents.txt

@@ -9,181 +9,241 @@ Filter Documents in the Visualization
 .. contents:: On this page
    :local:
    :backlinks: none
-   :depth: 1
+   :depth: 2
    :class: singlecol
 
-To display a subset of results in your data that match a given
-criteria, use the :guilabel:`Filters` input bar above the chart
-display. Input a filter document employing the same syntax used in the
-query portion of the :manual:`db.collection.find()
-</tutorial/query-documents/>` method. After entering a filter document,
-click :guilabel:`Apply` to see the filter reflected in your visualization.
+Filters display a subset of results that match a given criteria.
+|charts| provides two ways to filter your data. You can either use:
 
-.. note::
+- The :ref:`Filter Tab <filter-tab>` in the Chart Builter, or
 
-   Filters on large collections may have performance issues if there
-   are not appropriate :manual:`indexes <indexes>` on the collection.
+- The :ref:`Query Bar <query-bar>` above the chart display.
 
-Basic Filter Example
---------------------
+.. _filter-tab:
 
-The following chart shows the average runtime of movies by genre.
-The filter of ``{runtime: {$gte: 120 }}`` means that we are only
-taking into account movies which have a runtime greater than or
-equal to 120 minutes.
+Filter Tab
+----------
 
-.. figure:: /images/charts/charts-dashboard-filter-2.png
-   :figwidth: 720px
-   :alt: Charts dashboard filter
+The chart builder contains a filter tab where you can drag and drop
+fields to specify filters based on those fields. To filter data using
+the filter tab:
 
-.. _regex-filter:
+1. Click the center tab in the chart builder:
 
-Regular Expression (RegEx) Filter
----------------------------------
+   .. figure:: /images/charts/filter-tab.png
+      :scale: 60%
+      :alt: Image showing how to access filter tab
 
-.. _`regular expression`: https://en.wikipedia.org/wiki/Regular_expression?oldid=858335070
+   |
 
+#. Drag a field from the :guilabel:`Fields` on the left to the
+   :guilabel:`Chart Filters` section of the tab.
+   
+The data type of the selected field dictates the available filtering
+options. You can filter fields with the following data types:
 
-RegEx filters allow you to apply a `regular expression`_ as the match
-criteria to restrict the data |charts| displays. The expression
-uses the following syntax:
+- :ref:`Numeric <filter-numeric>`
 
-.. code-block:: sh
+- :ref:`String <filter-string>`
 
-   { <field>: { $regex: "pattern", $options: "<options>" } }
+.. note::
 
-The ``$options`` are optional and are the same as the
-:query:`$regex options <$options>` in the MongoDB shell.
+   You cannot use the same field for multiple filters.
 
-.. example::
+.. _filter-numeric:
 
-   **Filter data to document fields that start with a specific letter**
+Filter Numeric Fields
+~~~~~~~~~~~~~~~~~~~~~
 
-   To find all documents where the ``jobs`` field begins with the
-   letter ``A``, you would write the following in the
-   :guilabel:`Filter` bar:
+When you drag a numeric field to the filter panel, you
+can filter based on minimum and/or maximum values for that field.
 
-   .. code-block:: javascript
+.. list-table::
+   :header-rows: 1
 
-      { "job" : { $regex : "^A" } }
+   * - To specify a minimum value:
+     - To specify a maximum value:
 
-   **Filter data to document fields that start with a specific letter
-   ignoring case**
+   * - 1. Toggle :guilabel:`Min` to :guilabel:`On`.
 
-   To find all documents where the ``jobs`` field begins with the
-   letter ``A`` or ``a``, you would write the following in the
-   :guilabel:`Filter` bar:
+       #. Specify the desired minimum value.
+       
+       #. Select whether this is an inclusive minimum value.
 
-   .. code-block:: javascript
+     - 1. Toggle :guilabel:`Max` to :guilabel:`On`.
 
-      { "job" : { $regex : "^A", $options : "i" } }
+       #. Specify the desired maximum value.
+       
+       #. Select whether this is an inclusive maximum value.
 
-.. note::
+.. example::
 
-   The quotation marks around the regular expression are required. You
-   may not use forward slashes to delineate the regex value as you may
-   in the MongoDB shell.
+   If you have a minimum value of ``5`` with the :guilabel:`Inclusive`
+   setting on, |charts| shows documents where the field is greater than
+   or equal to ``5``.
 
-.. _date-filter:
+   Alternatively, if :guilabel:`Inclusive` is off, |charts| shows
+   documents where the field is greater than ``5``.
 
-Relative Date Filter
---------------------
+.. _filter-string:
 
-Relative Date Filters allow you to specify a date from which |charts|
-restricts the data displayed. For example, you can set a Relative Date
-Filter to display data from only the last month or last year. To
-create a filter spanning from the specified date to the current date,
-specify the number of milliseconds since the Unix epoch of January 1,
-1970. Use this date in conjunction with a
-:manual:`comparison query operator </reference/operator/query-comparison/>`
-to set the inclusive or exclusive time range of the data displayed.
+Filter String Fields
+~~~~~~~~~~~~~~~~~~~~
 
-.. example::
+When you drag a string field to the filter panel, |charts| displays a
+list of up to 20 distinct field values. If more than 20 distinct values
+exist, |charts| displays 20 randomly selected values.
 
-   The following Relative Date Filter returns documents that have a
-   ``timestamp`` field which resolves to a date less than 30 days from
-   the current date:
+The list also includes:
 
-   .. code-block:: javascript
+- :guilabel:`NULL / MISSING` for documents with ``null`` values for
+  the field or are missing the field.
 
-      { timestamp: {  $gt: new Date(new Date() - 30 * 24 * 60 * 60 * 1000 ) } }
+- :guilabel:`Empty String` for documents with ``""`` values for the
+  field.
 
-   First, the inner ``new Date()`` constructor generates the current
-   date in milliseconds since the Unix epoch of January 1, 1970. The
-   mathematical series ``30 * 24 * 60 * 60 * 1000`` results in the
-   number of milliseconds elapsed in 30 days. The filter takes the
-   current date in milliseconds since the Unix epoch and subtracts the
-   number of milliseconds in 30 days. This results in a new millisecond
-   value which the filter passes to an outer ``new Date()`` constructor
-   and resolves to the date 30 days prior to the time the user executes
-   the filter.
+To include a value not displayed in the list, add a specific value by
+clicking :guilabel:`Add Value`.
 
-   Using a mathematical series as shown here allows the filter to
-   always span a relative timeframe of 30 days prior to the time
-   the user executes the filter.
+Select the string values to display in the chart. By default, all
+values are selected.
 
-For a more complete example of a relative date filter, see the
-:ref:`Relative Date Filter Example <relative-date-example>`.
+.. tip::
 
-.. admonition:: ISO-8601 Dates
-   :class: note
+   If all strings are selected, you can click :guilabel:`Deselect All` 
+   at the top of the list to hide all strings.
+   
+   If not all strings are selected, you can click :guilabel:`Select All`
+   to return to the default state of having all strings displayed.
 
-   The date functions utilized in |charts| filters are consistent and
-   compatible with the date functions used in the
-   :manual:`mongo shell </mongo>`. As a result, you can also use the
-   ``ISODate()`` constructor in your |charts-short| query filters.
-   Specifying an ``ISODate()`` constructor with no parameters exhibits
-   the same behavior as specifying a ``new Date()`` constructor with no
-   paramters, both returning the current date in their respective
-   formats.
+Display Strings Not in the List
+```````````````````````````````
+
+You can check :guilabel:`All other values` to
+display all string values not included in the list.
+
+- If :guilabel:`All other values` is checked, |charts| filters out
+  any unchecked list items by using a :query:`$nin` query.
+  
+- If :guilabel:`All other values` is unchecked, |charts| only
+  includes the checked list items by using an :query:`$in` query.
+
+.. _query-bar:
 
-   .. example::
+Query Bar
+---------
+  
+To filter data using the :guilabel:`Query` bar:
 
-      The following filter returns documents that have a
-      ``timestamp`` field between January 1, 2017 and December 31, 2017
-      inclusively:
+1. In the :guilabel:`Query` bar, input a filter document. Use the
+   same syntax used in the query portion of the
+   :manual:`db.collection.find()
+   </reference/method/db.collection.find/>` method.
+   
+#. Click :guilabel:`Apply`. 
 
-      .. code-block:: javascript
+.. note::
 
-         {$and: [{timestamp: {$gte: ISODate("2017-01-01")}},
-         {timestamp: {$lte: ISODate("2017-12-31")}}]}
+   Filters on large collections may have performance issues if there
+   are not appropriate :manual:`indexes <indexes>` on the collection.
 
-.. _relative-date-example:
+Examples
+~~~~~~~~
 
-Relative Date Filter Example
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Comparison Query Filter
+```````````````````````
 
-The following chart visualizes workout data. Each document in the
-collection represents an individual workout activity, which contains
-information such as the type of workout and various exercise
-statistics. This line chart shows the distance per month covered
-across all workouts over the past year (365 days):
+The following chart shows the average runtime of movies by
+genre. The filter of ``{runtime: {$gte: 120 }}`` means that we only
+include movies which have a runtime greater than or equal to 120
+minutes.
 
-.. figure:: /images/charts/relative-date-example.png
+.. figure:: /images/charts/charts-dashboard-filter-2.png
    :figwidth: 720px
    :alt: Charts dashboard filter
 
-The chart utilizes the following filter:
+.. _regex-filter:
+
+Regular Expression (RegEx) Query
+````````````````````````````````
+
+.. _`regular expression`: :manual:`
+
+
+Use the :query:`$regex` query operator to filter using a
+regular expression:
+
+.. code-block:: javascript
+
+   { <field>: { $regex: "pattern", $options: "<options>" } }
+
+.. note::
+
+   You must enclose the pattern in quotes.  You cannot use the ``/pattern/`` syntax.
+
+For example, to find all documents where the ``jobs`` field begins with
+the letter ``A``, you would write the following in the
+:guilabel:`Query` bar:
+
+.. code-block:: javascript
+
+   { "job" : { $regex : "^A" } }
+
+To find all documents where the ``jobs`` field begins with the
+letter ``A`` or ``a``, you would write the following in the
+:guilabel:`Query` bar:
 
 .. code-block:: javascript
 
-   { "Workout Date (As Date)": {$gte: new Date(new Date() - 365 * 24 * 60 * 60 * 1000 ) }}
+   { "job" : { $regex : "^A", $options : "i" } }
+
+Date Range Query
+````````````````
+
+The following query returns documents that have a
+``timestamp`` field between January 1, 2017 and December 31, 2017
+inclusively:
+
+.. code-block:: javascript
 
-This filter returns documents where the ``Workout Date (As Date)``
-field is within one year prior to the time |charts-short| executes the
-query. The ``(new Date() - 365 * 24 * 60 * 60 * 1000 )`` parameter
-results in the date one year prior to the current date as expressed in
-milliseconds since the Unix epoch. |charts-short| returns documents
-with a value greater than or equal to this date, as signified by the
-:manual:`$gte </reference/operator/query/gte/>` operator.
+   {timestamp: {$gte: new Date("2017-01-01"), $lte: new Date("2017-12-31")}}
 
-.. admonition:: Date() is not supported
+.. admonition:: ISO-8601 Dates
    :class: note
 
-   The ``Date()`` function (as opposed to the ``new Date()``
-   constructor) returns the current date as a string, so it cannot be
-   used for filtering dates in |charts-short|. Use:
+   The date functions utilized in |charts| queries are consistent and
+   compatible with the date functions used in the
+   :manual:`mongo shell </mongo>`. As a result, you can use:
 
    - ``new Date()``,
    - ``ISODate()``, or
-   - ``new ISODate()``
+   - ``new ISODate()``.
+
+   The ``Date()`` function (as opposed to the ``new Date()``
+   constructor) returns the current date as a string, so it cannot be
+   used for querying dates in |charts-short|. 
+
+.. _date-filter:
+
+Relative Date Query
+```````````````````
+
+You can include a mathematical expression in the ``new Date()``
+constructor to get a relative date, such as 1 month ago or 1 year
+ago.
+
+For example, the following query returns documents that
+have a ``timestamp`` field greater than 30 days
+(30 * 24 * 60 * 60 * 1000 milliseconds) from the current date and time:
+
+.. code-block:: javascript
+
+   { timestamp: {  $gt: new Date(new Date() - 30 * 24 * 60 * 60 * 1000 ) } }
+
+Alternatively, you can specify the date components. For example, the
+following query returns documents that have a ``timestamp`` field
+greater than 1 month ago:
+
+.. code-block:: javascript
+
+    { timestamp: { $gt: new Date(new Date().getFullYear(), new Date().getMonth() -1 , new Date().getDate() ) } }