diff --git a/docs/guide.md b/docs/guide.md index 2ed451b08c3..9c2766c311b 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -559,6 +559,7 @@ Valid options: * [methods](#methods) * [query](#query-helpers) * [autoSearchIndex](#autoSearchIndex) +* [readConcern](#readConcern)

option: autoIndex

@@ -1473,6 +1474,24 @@ schema.searchIndex({ const Test = mongoose.model('Test', schema); ``` +

+ + option: readConcern + +

+ +[Read concerns](https://www.mongodb.com/docs/manual/reference/read-concern/) are similar to [`writeConcern`](#writeConcern), but for read operations like `find()` and `findOne()`. +To set a default `readConcern`, pass the `readConcern` option to the schema constructor as follows. + +```javascript +const eventSchema = new mongoose.Schema( + { name: String }, + { + readConcern: { level: 'available' } // <-- set default readConcern for all queries + } +); +``` +

With ES6 Classes

Schemas have a [`loadClass()` method](api/schema.html#schema_Schema-loadClass) diff --git a/docs/transactions.md b/docs/transactions.md index 901282dac44..4251cd5d017 100644 --- a/docs/transactions.md +++ b/docs/transactions.md @@ -1,8 +1,6 @@ # Transactions in Mongoose -[Transactions](https://www.mongodb.com/transactions) are new in MongoDB -4.0 and Mongoose 5.2.0. Transactions let you execute multiple operations -in isolation and potentially undo all the operations if one of them fails. +[Transactions](https://www.mongodb.com/transactions) let you execute multiple operations in isolation and potentially undo all the operations if one of them fails. This guide will get you started using transactions with Mongoose.

Getting Started with Transactions

@@ -86,6 +84,33 @@ Below is an example of executing an aggregation within a transaction. [require:transactions.*aggregate] ``` +

Using AsyncLocalStorage

+ +One major pain point with transactions in Mongoose is that you need to remember to set the `session` option on every operation. +If you don't, your operation will execute outside of the transaction. +Mongoose 8.4 is able to set the `session` operation on all operations within a `Connection.prototype.transaction()` executor function using Node's [AsyncLocalStorage API](https://nodejs.org/api/async_context.html#class-asynclocalstorage). +Set the `transactionAsyncLocalStorage` option using `mongoose.set('transactionAsyncLocalStorage', true)` to enable this feature. + +```javascript +mongoose.set('transactionAsyncLocalStorage', true); + +const Test = mongoose.model('Test', mongoose.Schema({ name: String })); + +const doc = new Test({ name: 'test' }); + +// Save a new doc in a transaction that aborts +await connection.transaction(async() => { + await doc.save(); // Notice no session here + throw new Error('Oops'); +}).catch(() => {}); + +// false, `save()` was rolled back +await Test.exists({ _id: doc._id }); +``` + +With `transactionAsyncLocalStorage`, you no longer need to pass sessions to every operation. +Mongoose will add the session by default under the hood. +

Advanced Usage

Advanced users who want more fine-grained control over when they commit or abort transactions diff --git a/docs/typescript/schemas.md b/docs/typescript/schemas.md index e4127ce1621..8dfa5310556 100644 --- a/docs/typescript/schemas.md +++ b/docs/typescript/schemas.md @@ -9,7 +9,7 @@ Mongoose can automatically infer the document type from your schema definition a We recommend relying on automatic type inference when defining schemas and models. ```typescript -import { Schema } from 'mongoose'; +import { Schema, model } from 'mongoose'; // Schema const schema = new Schema({ name: { type: String, required: true }, @@ -32,6 +32,31 @@ There are a few caveats for using automatic type inference: 2. You need to define your schema in the `new Schema()` call. Don't assign your schema definition to a temporary variable. Doing something like `const schemaDefinition = { name: String }; const schema = new Schema(schemaDefinition);` will not work. 3. Mongoose adds `createdAt` and `updatedAt` to your schema if you specify the `timestamps` option in your schema, *except* if you also specify `methods`, `virtuals`, or `statics`. There is a [known issue](https://github.com/Automattic/mongoose/issues/12807) with type inference with timestamps and methods/virtuals/statics options. If you use methods, virtuals, and statics, you're responsible for adding `createdAt` and `updatedAt` to your schema definition. +If you need to explicitly get the raw document type (the value returned from `doc.toObject()`, `await Model.findOne().lean()`, etc.) from your schema definition, you can use Mongoose's `inferRawDocType` helper as follows: + +```ts +import { Schema, InferRawDocType, model } from 'mongoose'; + +const schemaDefinition = { + name: { type: String, required: true }, + email: { type: String, required: true }, + avatar: String +} as const; +const schema = new Schema(schemaDefinition); + +const UserModel = model('User', schema); +const doc = new UserModel({ name: 'test', email: 'test' }); + +type RawUserDocument = InferRawDocType; + +useRawDoc(doc.toObject()); + +function useRawDoc(doc: RawUserDocument) { + // ... +} + +``` + If automatic type inference doesn't work for you, you can always fall back to document interface definitions. ## Separate document interface definition diff --git a/lib/aggregate.js b/lib/aggregate.js index 827f1642a60..35c32c480a9 100644 --- a/lib/aggregate.js +++ b/lib/aggregate.js @@ -1022,6 +1022,11 @@ Aggregate.prototype.exec = async function exec() { applyGlobalMaxTimeMS(this.options, model.db.options, model.base.options); applyGlobalDiskUse(this.options, model.db.options, model.base.options); + const asyncLocalStorage = this.model()?.db?.base.transactionAsyncLocalStorage?.getStore(); + if (!this.options.hasOwnProperty('session') && asyncLocalStorage?.session != null) { + this.options.session = asyncLocalStorage.session; + } + if (this.options && this.options.cursor) { return new AggregationCursor(this); } diff --git a/lib/connection.js b/lib/connection.js index 05ff52461b0..8c02b686baa 100644 --- a/lib/connection.js +++ b/lib/connection.js @@ -398,11 +398,7 @@ Connection.prototype.createCollection = async function createCollection(collecti throw new MongooseError('Connection.prototype.createCollection() no longer accepts a callback'); } - if ((this.readyState === STATES.connecting || this.readyState === STATES.disconnected) && this._shouldBufferCommands()) { - await new Promise(resolve => { - this._queue.push({ fn: resolve }); - }); - } + await this._waitForConnect(); return this.db.createCollection(collection, options); }; @@ -494,11 +490,7 @@ Connection.prototype.startSession = async function startSession(options) { throw new MongooseError('Connection.prototype.startSession() no longer accepts a callback'); } - if ((this.readyState === STATES.connecting || this.readyState === STATES.disconnected) && this._shouldBufferCommands()) { - await new Promise(resolve => { - this._queue.push({ fn: resolve }); - }); - } + await this._waitForConnect(); const session = this.client.startSession(options); return session; @@ -539,7 +531,7 @@ Connection.prototype.startSession = async function startSession(options) { Connection.prototype.transaction = function transaction(fn, options) { return this.startSession().then(session => { session[sessionNewDocuments] = new Map(); - return session.withTransaction(() => _wrapUserTransaction(fn, session), options). + return session.withTransaction(() => _wrapUserTransaction(fn, session, this.base), options). then(res => { delete session[sessionNewDocuments]; return res; @@ -558,9 +550,16 @@ Connection.prototype.transaction = function transaction(fn, options) { * Reset document state in between transaction retries re: gh-13698 */ -async function _wrapUserTransaction(fn, session) { +async function _wrapUserTransaction(fn, session, mongoose) { try { - const res = await fn(session); + const res = mongoose.transactionAsyncLocalStorage == null + ? await fn(session) + : await new Promise(resolve => { + mongoose.transactionAsyncLocalStorage.run( + { session }, + () => resolve(fn(session)) + ); + }); return res; } catch (err) { _resetSessionDocuments(session); @@ -618,13 +617,24 @@ Connection.prototype.dropCollection = async function dropCollection(collection) throw new MongooseError('Connection.prototype.dropCollection() no longer accepts a callback'); } + await this._waitForConnect(); + + return this.db.dropCollection(collection); +}; + +/** + * Waits for connection to be established, so the connection has a `client` + * + * @return Promise + * @api private + */ + +Connection.prototype._waitForConnect = async function _waitForConnect() { if ((this.readyState === STATES.connecting || this.readyState === STATES.disconnected) && this._shouldBufferCommands()) { await new Promise(resolve => { this._queue.push({ fn: resolve }); }); } - - return this.db.dropCollection(collection); }; /** @@ -637,16 +647,31 @@ Connection.prototype.dropCollection = async function dropCollection(collection) */ Connection.prototype.listCollections = async function listCollections() { - if ((this.readyState === STATES.connecting || this.readyState === STATES.disconnected) && this._shouldBufferCommands()) { - await new Promise(resolve => { - this._queue.push({ fn: resolve }); - }); - } + await this._waitForConnect(); const cursor = this.db.listCollections(); return await cursor.toArray(); }; +/** + * Helper for MongoDB Node driver's `listDatabases()`. + * Returns an object with a `databases` property that contains an + * array of database objects. + * + * #### Example: + * const { databases } = await mongoose.connection.listDatabases(); + * databases; // [{ name: 'mongoose_test', sizeOnDisk: 0, empty: false }] + * + * @method listCollections + * @return {Promise<{ databases: Array<{ name: string }> }>} + * @api public + */ + +Connection.prototype.listDatabases = async function listDatabases() { + // Implemented in `lib/drivers/node-mongodb-native/connection.js` + throw new MongooseError('listDatabases() not implemented by driver'); +}; + /** * Helper for `dropDatabase()`. Deletes the given database, including all * collections, documents, and indexes. @@ -667,11 +692,7 @@ Connection.prototype.dropDatabase = async function dropDatabase() { throw new MongooseError('Connection.prototype.dropDatabase() no longer accepts a callback'); } - if ((this.readyState === STATES.connecting || this.readyState === STATES.disconnected) && this._shouldBufferCommands()) { - await new Promise(resolve => { - this._queue.push({ fn: resolve }); - }); - } + await this._waitForConnect(); // If `dropDatabase()` is called, this model's collection will not be // init-ed. It is sufficiently common to call `dropDatabase()` after diff --git a/lib/drivers/node-mongodb-native/connection.js b/lib/drivers/node-mongodb-native/connection.js index c0f3261595f..3c64ff2216f 100644 --- a/lib/drivers/node-mongodb-native/connection.js +++ b/lib/drivers/node-mongodb-native/connection.js @@ -197,6 +197,19 @@ NativeConnection.prototype.doClose = async function doClose(force) { return this; }; +/** + * Implementation of `listDatabases()` for MongoDB driver + * + * @return Promise + * @api public + */ + +NativeConnection.prototype.listDatabases = async function listDatabases() { + await this._waitForConnect(); + + return await this.db.admin().listDatabases(); +}; + /*! * ignore */ diff --git a/lib/error/browserMissingSchema.js b/lib/error/browserMissingSchema.js index 3f271499d4d..608cfd983e4 100644 --- a/lib/error/browserMissingSchema.js +++ b/lib/error/browserMissingSchema.js @@ -4,7 +4,7 @@ 'use strict'; -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); class MissingSchemaError extends MongooseError { diff --git a/lib/error/cast.js b/lib/error/cast.js index f7df49b8c7e..115927117f7 100644 --- a/lib/error/cast.js +++ b/lib/error/cast.js @@ -75,7 +75,6 @@ class CastError extends MongooseError { * ignore */ setModel(model) { - this.model = model; this.message = formatMessage(model, this.kind, this.value, this.path, this.messageFormat, this.valueType); } diff --git a/lib/error/divergentArray.js b/lib/error/divergentArray.js index 6bb527d0205..f266dbde449 100644 --- a/lib/error/divergentArray.js +++ b/lib/error/divergentArray.js @@ -5,7 +5,7 @@ 'use strict'; -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); class DivergentArrayError extends MongooseError { /** diff --git a/lib/error/eachAsyncMultiError.js b/lib/error/eachAsyncMultiError.js index 9c04020312b..b14156a09a3 100644 --- a/lib/error/eachAsyncMultiError.js +++ b/lib/error/eachAsyncMultiError.js @@ -4,7 +4,7 @@ 'use strict'; -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); /** diff --git a/lib/error/invalidSchemaOption.js b/lib/error/invalidSchemaOption.js index 2ab1aa9497e..089dc6a03ef 100644 --- a/lib/error/invalidSchemaOption.js +++ b/lib/error/invalidSchemaOption.js @@ -5,7 +5,7 @@ 'use strict'; -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); class InvalidSchemaOptionError extends MongooseError { /** diff --git a/lib/error/missingSchema.js b/lib/error/missingSchema.js index 50c81054a90..2b3bf242526 100644 --- a/lib/error/missingSchema.js +++ b/lib/error/missingSchema.js @@ -5,7 +5,7 @@ 'use strict'; -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); class MissingSchemaError extends MongooseError { /** diff --git a/lib/error/notFound.js b/lib/error/notFound.js index e1064bb89d9..19a22f3a101 100644 --- a/lib/error/notFound.js +++ b/lib/error/notFound.js @@ -4,7 +4,7 @@ * Module dependencies. */ -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); const util = require('util'); class DocumentNotFoundError extends MongooseError { diff --git a/lib/error/objectExpected.js b/lib/error/objectExpected.js index 6506f60656a..9f7a8116618 100644 --- a/lib/error/objectExpected.js +++ b/lib/error/objectExpected.js @@ -4,7 +4,7 @@ 'use strict'; -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); class ObjectExpectedError extends MongooseError { diff --git a/lib/error/objectParameter.js b/lib/error/objectParameter.js index 295582e2484..b3f5b80849d 100644 --- a/lib/error/objectParameter.js +++ b/lib/error/objectParameter.js @@ -4,7 +4,7 @@ 'use strict'; -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); class ObjectParameterError extends MongooseError { /** diff --git a/lib/error/overwriteModel.js b/lib/error/overwriteModel.js index 1ff180b0498..8904e4e74b3 100644 --- a/lib/error/overwriteModel.js +++ b/lib/error/overwriteModel.js @@ -5,7 +5,7 @@ 'use strict'; -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); class OverwriteModelError extends MongooseError { diff --git a/lib/error/parallelSave.js b/lib/error/parallelSave.js index e0628576de7..57ac458238d 100644 --- a/lib/error/parallelSave.js +++ b/lib/error/parallelSave.js @@ -4,7 +4,7 @@ * Module dependencies. */ -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); class ParallelSaveError extends MongooseError { /** diff --git a/lib/error/strict.js b/lib/error/strict.js index 393ca6e1fc7..6cf4cf91141 100644 --- a/lib/error/strict.js +++ b/lib/error/strict.js @@ -4,7 +4,7 @@ 'use strict'; -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); class StrictModeError extends MongooseError { diff --git a/lib/error/strictPopulate.js b/lib/error/strictPopulate.js index f7addfa5287..288799897bc 100644 --- a/lib/error/strictPopulate.js +++ b/lib/error/strictPopulate.js @@ -4,7 +4,7 @@ 'use strict'; -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); class StrictPopulateError extends MongooseError { /** diff --git a/lib/error/validator.js b/lib/error/validator.js index 4ca7316d7bf..f7ee2ef4761 100644 --- a/lib/error/validator.js +++ b/lib/error/validator.js @@ -4,7 +4,7 @@ 'use strict'; -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); class ValidatorError extends MongooseError { diff --git a/lib/error/version.js b/lib/error/version.js index b357fb16ca3..85f2921a517 100644 --- a/lib/error/version.js +++ b/lib/error/version.js @@ -4,7 +4,7 @@ * Module dependencies. */ -const MongooseError = require('./'); +const MongooseError = require('./mongooseError'); class VersionError extends MongooseError { /** diff --git a/lib/helpers/omitUndefined.js b/lib/helpers/omitUndefined.js new file mode 100644 index 00000000000..5c9eb88564a --- /dev/null +++ b/lib/helpers/omitUndefined.js @@ -0,0 +1,20 @@ +'use strict'; + +module.exports = function omitUndefined(val) { + if (val == null || typeof val !== 'object') { + return val; + } + if (Array.isArray(val)) { + for (let i = val.length - 1; i >= 0; --i) { + if (val[i] === undefined) { + val.splice(i, 1); + } + } + } + for (const key of Object.keys(val)) { + if (val[key] === void 0) { + delete val[key]; + } + } + return val; +}; diff --git a/lib/helpers/query/cast$expr.js b/lib/helpers/query/cast$expr.js index a13190b1c41..9889d47ada1 100644 --- a/lib/helpers/query/cast$expr.js +++ b/lib/helpers/query/cast$expr.js @@ -3,6 +3,7 @@ const CastError = require('../../error/cast'); const StrictModeError = require('../../error/strict'); const castNumber = require('../../cast/number'); +const omitUndefined = require('../omitUndefined'); const booleanComparison = new Set(['$and', '$or']); const comparisonOperator = new Set(['$cmp', '$eq', '$lt', '$lte', '$gt', '$gte']); @@ -125,18 +126,11 @@ function _castExpression(val, schema, strictQuery) { val.$round = $round.map(v => castNumberOperator(v, schema, strictQuery)); } - _omitUndefined(val); + omitUndefined(val); return val; } -function _omitUndefined(val) { - const keys = Object.keys(val); - for (let i = 0, len = keys.length; i < len; ++i) { - (val[keys[i]] === void 0) && delete val[keys[i]]; - } -} - // { $op: } function castNumberOperator(val) { if (!isLiteral(val)) { diff --git a/lib/helpers/schema/applyReadConcern.js b/lib/helpers/schema/applyReadConcern.js new file mode 100644 index 00000000000..80d4da6eb20 --- /dev/null +++ b/lib/helpers/schema/applyReadConcern.js @@ -0,0 +1,22 @@ +'use strict'; + +const get = require('../get'); + +module.exports = function applyReadConcern(schema, options) { + if (options.readConcern !== undefined) { + return; + } + + // Don't apply default read concern to operations in transactions, + // because you shouldn't set read concern on individual operations + // within a transaction. + // See: https://www.mongodb.com/docs/manual/reference/read-concern/ + if (options && options.session && options.session.transaction) { + return; + } + + const level = get(schema, 'options.readConcern.level', null); + if (level != null) { + options.readConcern = { level }; + } +}; diff --git a/lib/index.js b/lib/index.js index 6247d6a4d5c..67534e9e793 100644 --- a/lib/index.js +++ b/lib/index.js @@ -4,10 +4,14 @@ * Module dependencies. */ -require('./driver').set(require('./drivers/node-mongodb-native')); +const mongodbDriver = require('./drivers/node-mongodb-native'); + +require('./driver').set(mongodbDriver); const mongoose = require('./mongoose'); +mongoose.setDriver(mongodbDriver); + mongoose.Mongoose.prototype.mongo = require('mongodb'); module.exports = mongoose; diff --git a/lib/model.js b/lib/model.js index 13a0a877a8d..27af8745207 100644 --- a/lib/model.js +++ b/lib/model.js @@ -27,6 +27,7 @@ const applyEmbeddedDiscriminators = require('./helpers/discriminator/applyEmbedd const applyHooks = require('./helpers/model/applyHooks'); const applyMethods = require('./helpers/model/applyMethods'); const applyProjection = require('./helpers/projection/applyProjection'); +const applyReadConcern = require('./helpers/schema/applyReadConcern'); const applySchemaCollation = require('./helpers/indexes/applySchemaCollation'); const applyStaticHooks = require('./helpers/model/applyStaticHooks'); const applyStatics = require('./helpers/model/applyStatics'); @@ -296,8 +297,11 @@ Model.prototype.$__handleSave = function(options, callback) { } const session = this.$session(); + const asyncLocalStorage = this[modelDbSymbol].base.transactionAsyncLocalStorage?.getStore(); if (!saveOptions.hasOwnProperty('session') && session != null) { saveOptions.session = session; + } else if (asyncLocalStorage?.session != null) { + saveOptions.session = asyncLocalStorage.session; } if (this.$isNew) { // send entire doc @@ -417,6 +421,8 @@ Model.prototype.$__handleSave = function(options, callback) { where[key] = val; } } + + applyReadConcern(this.$__schema, optionsWithCustomValues); this.constructor.collection.findOne(where, optionsWithCustomValues) .then(documentExists => { const matchedCount = !documentExists ? 0 : 1; @@ -1655,6 +1661,31 @@ Model.dropSearchIndex = async function dropSearchIndex(name) { return await this.$__collection.dropSearchIndex(name); }; +/** + * List all [Atlas search indexes](https://www.mongodb.com/docs/atlas/atlas-search/create-index/) on this model's collection. + * This function only works when connected to MongoDB Atlas. + * + * #### Example: + * + * const schema = new Schema({ name: { type: String, unique: true } }); + * const Customer = mongoose.model('Customer', schema); + * + * await Customer.createSearchIndex({ name: 'test', definition: { mappings: { dynamic: true } } }); + * const res = await Customer.listSearchIndexes(); // Includes `[{ name: 'test' }]` + * + * @param {Object} [options] + * @return {Promise} + * @api public + */ + +Model.listSearchIndexes = async function listSearchIndexes(options) { + _checkContext(this, 'listSearchIndexes'); + + const cursor = await this.$__collection.listSearchIndexes(options); + + return await cursor.toArray(); +}; + /** * Does a dry-run of `Model.syncIndexes()`, returning the indexes that `syncIndexes()` would drop and create if you were to run `syncIndexes()`. * @@ -3541,6 +3572,10 @@ Model.bulkWrite = async function bulkWrite(ops, options) { } const validations = ops.map(op => castBulkWrite(this, op, options)); + const asyncLocalStorage = this.db.base.transactionAsyncLocalStorage?.getStore(); + if ((!options || !options.hasOwnProperty('session')) && asyncLocalStorage?.session != null) { + options = { ...options, session: asyncLocalStorage.session }; + } let res = null; if (ordered) { diff --git a/lib/mongoose.js b/lib/mongoose.js index 915720b59f7..cfac2331ced 100644 --- a/lib/mongoose.js +++ b/lib/mongoose.js @@ -38,6 +38,8 @@ require('./helpers/printJestWarning'); const objectIdHexRegexp = /^[0-9A-Fa-f]{24}$/; +const { AsyncLocalStorage } = require('node:async_hooks'); + /** * Mongoose constructor. * @@ -68,8 +70,8 @@ function Mongoose(options) { autoCreate: true, autoSearchIndex: false }, options); - const createInitialConnection = utils.getOption('createInitialConnection', this.options); - if (createInitialConnection == null || createInitialConnection) { + const createInitialConnection = utils.getOption('createInitialConnection', this.options) ?? true; + if (createInitialConnection && this.__driver != null) { const conn = this.createConnection(); // default connection conn.models = this.models; } @@ -101,6 +103,10 @@ function Mongoose(options) { } this.Schema.prototype.base = this; + if (options?.transactionAsyncLocalStorage) { + this.transactionAsyncLocalStorage = new AsyncLocalStorage(); + } + Object.defineProperty(this, 'plugins', { configurable: false, enumerable: true, @@ -169,6 +175,9 @@ Mongoose.prototype.setDriver = function setDriver(driver) { const oldDefaultConnection = _mongoose.connections[0]; _mongoose.connections = [new Connection(_mongoose)]; _mongoose.connections[0].models = _mongoose.models; + if (oldDefaultConnection == null) { + return _mongoose; + } // Update all models that pointed to the old default connection to // the new default connection, including collections @@ -267,7 +276,7 @@ Mongoose.prototype.set = function(key, value) { if (optionKey === 'objectIdGetter') { if (optionValue) { - Object.defineProperty(mongoose.Types.ObjectId.prototype, '_id', { + Object.defineProperty(_mongoose.Types.ObjectId.prototype, '_id', { enumerable: false, configurable: true, get: function() { @@ -275,7 +284,13 @@ Mongoose.prototype.set = function(key, value) { } }); } else { - delete mongoose.Types.ObjectId.prototype._id; + delete _mongoose.Types.ObjectId.prototype._id; + } + } else if (optionKey === 'transactionAsyncLocalStorage') { + if (optionValue && !_mongoose.transactionAsyncLocalStorage) { + _mongoose.transactionAsyncLocalStorage = new AsyncLocalStorage(); + } else if (!optionValue && _mongoose.transactionAsyncLocalStorage) { + delete _mongoose.transactionAsyncLocalStorage; } } } @@ -1278,6 +1293,28 @@ Mongoose.prototype.skipMiddlewareFunction = Kareem.skipWrappedFunction; Mongoose.prototype.overwriteMiddlewareResult = Kareem.overwriteResult; +/** + * Takes in an object and deletes any keys from the object whose values + * are strictly equal to `undefined`. + * This function is useful for query filters because Mongoose treats + * `TestModel.find({ name: undefined })` as `TestModel.find({ name: null })`. + * + * #### Example: + * + * const filter = { name: 'John', age: undefined, status: 'active' }; + * mongoose.omitUndefined(filter); // { name: 'John', status: 'active' } + * filter; // { name: 'John', status: 'active' } + * + * await UserModel.findOne(mongoose.omitUndefined(filter)); + * + * @method omitUndefined + * @param {Object} [val] the object to remove undefined keys from + * @returns {Object} the object passed in + * @api public + */ + +Mongoose.prototype.omitUndefined = require('./helpers/omitUndefined'); + /** * The exports object is an instance of Mongoose. * diff --git a/lib/query.js b/lib/query.js index 34af09d552b..5bb3ee9a611 100644 --- a/lib/query.js +++ b/lib/query.js @@ -13,6 +13,7 @@ const QueryCursor = require('./cursor/queryCursor'); const ValidationError = require('./error/validation'); const { applyGlobalMaxTimeMS, applyGlobalDiskUse } = require('./helpers/query/applyGlobalOption'); const handleReadPreferenceAliases = require('./helpers/query/handleReadPreferenceAliases'); +const applyReadConcern = require('./helpers/schema/applyReadConcern'); const applyWriteConcern = require('./helpers/schema/applyWriteConcern'); const cast = require('./cast'); const castArrayFilters = require('./helpers/update/castArrayFilters'); @@ -1944,9 +1945,15 @@ Query.prototype._optionsForExec = function(model) { if (!model) { return options; } + applyReadConcern(model.schema, options); // Apply schema-level `writeConcern` option applyWriteConcern(model.schema, options); + const asyncLocalStorage = this.model?.db?.base.transactionAsyncLocalStorage?.getStore(); + if (!this.options.hasOwnProperty('session') && asyncLocalStorage?.session != null) { + options.session = asyncLocalStorage.session; + } + const readPreference = model && model.schema && model.schema.options && diff --git a/lib/schema.js b/lib/schema.js index 04c631eb799..97c64a38e1c 100644 --- a/lib/schema.js +++ b/lib/schema.js @@ -66,6 +66,7 @@ const numberRE = /^\d+$/; * - [_id](https://mongoosejs.com/docs/guide.html#_id): bool - defaults to true * - [minimize](https://mongoosejs.com/docs/guide.html#minimize): bool - controls [document#toObject](https://mongoosejs.com/docs/api/document.html#Document.prototype.toObject()) behavior when called manually - defaults to true * - [read](https://mongoosejs.com/docs/guide.html#read): string + * - [readConcern](https://mongoosejs.com/docs/guide.html#readConcern): object - defaults to null, use to set a default [read concern](https://www.mongodb.com/docs/manual/reference/read-concern/) for all queries. * - [writeConcern](https://mongoosejs.com/docs/guide.html#writeConcern): object - defaults to null, use to override [the MongoDB server's default write concern settings](https://www.mongodb.com/docs/manual/reference/write-concern/) * - [shardKey](https://mongoosejs.com/docs/guide.html#shardKey): object - defaults to `null` * - [strict](https://mongoosejs.com/docs/guide.html#strict): bool - defaults to true diff --git a/lib/validOptions.js b/lib/validOptions.js index c9968237595..2654a7521ed 100644 --- a/lib/validOptions.js +++ b/lib/validOptions.js @@ -32,6 +32,7 @@ const VALID_OPTIONS = Object.freeze([ 'strictQuery', 'toJSON', 'toObject', + 'transactionAsyncLocalStorage', 'translateAliases' ]); diff --git a/package.json b/package.json index 13628081763..ef6ec8a6b85 100644 --- a/package.json +++ b/package.json @@ -19,9 +19,9 @@ ], "license": "MIT", "dependencies": { - "bson": "^6.5.0", + "bson": "^6.7.0", "kareem": "2.6.3", - "mongodb": "6.5.0", + "mongodb": "6.6.2", "mpath": "0.9.0", "mquery": "5.0.0", "ms": "2.1.3", diff --git a/scripts/tsc-diagnostics-check.js b/scripts/tsc-diagnostics-check.js index 2f74bf39b92..3e1f6c66282 100644 --- a/scripts/tsc-diagnostics-check.js +++ b/scripts/tsc-diagnostics-check.js @@ -3,7 +3,7 @@ const fs = require('fs'); const stdin = fs.readFileSync(0).toString('utf8'); -const maxInstantiations = isNaN(process.argv[2]) ? 125000 : parseInt(process.argv[2], 10); +const maxInstantiations = isNaN(process.argv[2]) ? 127500 : parseInt(process.argv[2], 10); console.log(stdin); diff --git a/test/connection.test.js b/test/connection.test.js index 3d1170838cf..77ec8ac64d1 100644 --- a/test/connection.test.js +++ b/test/connection.test.js @@ -1580,6 +1580,16 @@ describe('connections:', function() { }); assert.ok(session); }); + it('listDatabases() should return a list of database objects with a name property (gh-9048)', async function() { + const connection = await mongoose.createConnection(start.uri).asPromise(); + // If this test is running in isolation, then the `start.uri` db might not + // exist yet, so create this collection (and the associated db) just in case + await connection.createCollection('tests').catch(() => {}); + + const { databases } = await connection.listDatabases(); + assert.ok(connection.name); + assert.ok(databases.map(database => database.name).includes(connection.name)); + }); describe('createCollections()', function() { it('should create collections for all models on the connection with the createCollections() function (gh-13300)', async function() { const m = new mongoose.Mongoose(); diff --git a/test/docs/transactions.test.js b/test/docs/transactions.test.js index 8b883e3388c..e21639331b9 100644 --- a/test/docs/transactions.test.js +++ b/test/docs/transactions.test.js @@ -351,6 +351,55 @@ describe('transactions', function() { await session.endSession(); }); + describe('transactionAsyncLocalStorage option', function() { + let m; + before(async function() { + m = new mongoose.Mongoose(); + m.set('transactionAsyncLocalStorage', true); + + await m.connect(start.uri); + }); + + after(async function() { + await m.disconnect(); + }); + + it('transaction() sets `session` by default if transactionAsyncLocalStorage option is set', async function() { + const Test = m.model('Test', m.Schema({ name: String })); + + await Test.createCollection(); + await Test.deleteMany({}); + + const doc = new Test({ name: 'test_transactionAsyncLocalStorage' }); + await assert.rejects( + () => m.connection.transaction(async() => { + await doc.save(); + + await Test.updateOne({ name: 'foo' }, { name: 'foo' }, { upsert: true }); + + let docs = await Test.aggregate([{ $match: { _id: doc._id } }]); + assert.equal(docs.length, 1); + + docs = await Test.find({ _id: doc._id }); + assert.equal(docs.length, 1); + + docs = await async function test() { + return await Test.findOne({ _id: doc._id }); + }(); + assert.equal(doc.name, 'test_transactionAsyncLocalStorage'); + + throw new Error('Oops!'); + }), + /Oops!/ + ); + let exists = await Test.exists({ _id: doc._id }); + assert.ok(!exists); + + exists = await Test.exists({ name: 'foo' }); + assert.ok(!exists); + }); + }); + it('transaction() resets $isNew on error', async function() { db.deleteModel(/Test/); const Test = db.model('Test', Schema({ name: String })); diff --git a/test/model.findByIdAndUpdate.test.js b/test/model.findByIdAndUpdate.test.js index cbd953f5606..9db1b39d228 100644 --- a/test/model.findByIdAndUpdate.test.js +++ b/test/model.findByIdAndUpdate.test.js @@ -53,9 +53,6 @@ describe('model: findByIdAndUpdate:', function() { 'shape.side': 4, 'shape.color': 'white' }, { new: true }); - console.log('doc'); - console.log(doc); - console.log('doc'); assert.equal(doc.shape.kind, 'gh8378_Square'); assert.equal(doc.shape.name, 'after'); diff --git a/test/schema.test.js b/test/schema.test.js index 0092a44a4ee..8cd58ba7b9f 100644 --- a/test/schema.test.js +++ b/test/schema.test.js @@ -3237,4 +3237,25 @@ describe('schema', function() { assert.equal(doc.element, '#hero'); assert.ok(doc instanceof ClickedModel); }); + + it('supports schema-level readConcern (gh-14511)', async function() { + const eventSchema = new mongoose.Schema({ + name: String + }, { readConcern: { level: 'available' } }); + const Event = db.model('Test', eventSchema); + + let q = Event.find(); + let options = q._optionsForExec(); + assert.deepStrictEqual(options.readConcern, { level: 'available' }); + + q = Event.find().setOptions({ readConcern: { level: 'local' } }); + options = q._optionsForExec(); + assert.deepStrictEqual(options.readConcern, { level: 'local' }); + + q = Event.find().setOptions({ readConcern: null }); + options = q._optionsForExec(); + assert.deepStrictEqual(options.readConcern, null); + + await q; + }); }); diff --git a/test/types/base.test.ts b/test/types/base.test.ts index fba2acf37b0..9d25cfe30f3 100644 --- a/test/types/base.test.ts +++ b/test/types/base.test.ts @@ -69,3 +69,5 @@ function setAsObject() { expectError(mongoose.set({ invalid: true })); } + +const x: { name: string } = mongoose.omitUndefined({ name: 'foo' }); diff --git a/test/types/connection.test.ts b/test/types/connection.test.ts index 93c3fc6c0d8..c5979663a20 100644 --- a/test/types/connection.test.ts +++ b/test/types/connection.test.ts @@ -78,6 +78,10 @@ expectType>( conn.listCollections().then(collections => collections.map(coll => coll.name)) ); +expectType>( + conn.listDatabases().then(dbs => dbs.databases.map(db => db.name)) +); + export function autoTypedModelConnection() { const AutoTypedSchema = autoTypedSchema(); const AutoTypedModel = connection.model('AutoTypeModelConnection', AutoTypedSchema); diff --git a/test/types/schema.test.ts b/test/types/schema.test.ts index 8deb72a6eda..bbbcdd5d71f 100644 --- a/test/types/schema.test.ts +++ b/test/types/schema.test.ts @@ -6,6 +6,7 @@ import { HydratedDocument, IndexDefinition, IndexOptions, + InferRawDocType, InferSchemaType, InsertManyOptions, ObtainDocumentType, @@ -1494,3 +1495,19 @@ function gh14573() { const doc = new UserModel({ names: { _id: '0'.repeat(24), firstName: 'foo' } }); doc.names?.ownerDocument(); } + +function gh13772() { + const schemaDefinition = { + name: String, + docArr: [{ name: String }] + }; + const schema = new Schema(schemaDefinition); + type RawDocType = InferRawDocType; + expectAssignable< + { name?: string | null, docArr?: Array<{ name?: string | null }> } + >({} as RawDocType); + + const TestModel = model('User', schema); + const doc = new TestModel(); + expectAssignable(doc.toObject()); +} diff --git a/types/connection.d.ts b/types/connection.d.ts index b2812d01cf6..2f47bdc84e5 100644 --- a/types/connection.d.ts +++ b/types/connection.d.ts @@ -132,6 +132,12 @@ declare module 'mongoose' { */ listCollections(): Promise[]>; + /** + * Helper for MongoDB Node driver's `listDatabases()`. + * Returns an array of database names. + */ + listDatabases(): Promise; + /** * A [POJO](https://masteringjs.io/tutorials/fundamentals/pojo) containing * a map from model names to models. Contains all models that have been diff --git a/types/document.d.ts b/types/document.d.ts index 2bb82e3c677..c0723e883bf 100644 --- a/types/document.d.ts +++ b/types/document.d.ts @@ -16,7 +16,7 @@ declare module 'mongoose' { * * TQueryHelpers - Object with any helpers that should be mixed into the Query type * * DocType - the type of the actual Document created */ - class Document { + class Document { constructor(doc?: any); /** This documents _id. */ diff --git a/types/error.d.ts b/types/error.d.ts index 226fad31931..3fec7c41399 100644 --- a/types/error.d.ts +++ b/types/error.d.ts @@ -28,7 +28,6 @@ declare module 'mongoose' { value: any; path: string; reason?: NativeError | null; - model?: any; constructor(type: string, value: any, path: string, reason?: NativeError, schemaType?: SchemaType); } diff --git a/types/index.d.ts b/types/index.d.ts index cd5695b35f2..2e2a5fbc6a7 100644 --- a/types/index.d.ts +++ b/types/index.d.ts @@ -20,6 +20,7 @@ /// /// /// +/// /// /// /// @@ -68,6 +69,8 @@ declare module 'mongoose' { /** Gets mongoose options */ export function get(key: K): MongooseOptions[K]; + export function omitUndefined>(val: T): T; + /* ! ignore */ export type CompileModelOptions = { overwriteModels?: boolean, diff --git a/types/inferrawdoctype.d.ts b/types/inferrawdoctype.d.ts new file mode 100644 index 00000000000..95a912ff240 --- /dev/null +++ b/types/inferrawdoctype.d.ts @@ -0,0 +1,116 @@ +import { + IsSchemaTypeFromBuiltinClass, + RequiredPaths, + OptionalPaths, + PathWithTypePropertyBaseType, + PathEnumOrString +} from './inferschematype'; + +declare module 'mongoose' { + export type InferRawDocType< + DocDefinition, + TSchemaOptions extends Record = DefaultSchemaOptions + > = { + [ + K in keyof (RequiredPaths & + OptionalPaths) + ]: ObtainRawDocumentPathType; + }; + + /** + * @summary Obtains schema Path type. + * @description Obtains Path type by separating path type from other options and calling {@link ResolvePathType} + * @param {PathValueType} PathValueType Document definition path type. + * @param {TypeKey} TypeKey A generic refers to document definition. + */ + type ObtainRawDocumentPathType< + PathValueType, + TypeKey extends string = DefaultTypeKey + > = ResolveRawPathType< + PathValueType extends PathWithTypePropertyBaseType ? PathValueType[TypeKey] : PathValueType, + PathValueType extends PathWithTypePropertyBaseType ? Omit : {}, + TypeKey + >; + + /** + * Same as inferSchemaType, except: + * + * 1. Replace `Types.DocumentArray` and `Types.Array` with vanilla `Array` + * 2. Replace `ObtainDocumentPathType` with `ObtainRawDocumentPathType` + * 3. Replace `ResolvePathType` with `ResolveRawPathType` + * + * @summary Resolve path type by returning the corresponding type. + * @param {PathValueType} PathValueType Document definition path type. + * @param {Options} Options Document definition path options except path type. + * @param {TypeKey} TypeKey A generic of literal string type."Refers to the property used for path type definition". + * @returns Number, "Number" or "number" will be resolved to number type. + */ + type ResolveRawPathType = {}, TypeKey extends string = DefaultSchemaOptions['typeKey']> = + PathValueType extends Schema ? + InferSchemaType : + PathValueType extends (infer Item)[] ? + IfEquals> : + Item extends Record ? + Item[TypeKey] extends Function | String ? + // If Item has a type key that's a string or a callable, it must be a scalar, + // so we can directly obtain its path type. + ObtainRawDocumentPathType[] : + // If the type key isn't callable, then this is an array of objects, in which case + // we need to call ObtainDocumentType to correctly infer its type. + Array> : + IsSchemaTypeFromBuiltinClass extends true ? + ObtainRawDocumentPathType[] : + IsItRecordAndNotAny extends true ? + Item extends Record ? + ObtainRawDocumentPathType[] : + Array> : + ObtainRawDocumentPathType[] + >: + PathValueType extends ReadonlyArray ? + IfEquals> : + Item extends Record ? + Item[TypeKey] extends Function | String ? + ObtainRawDocumentPathType[] : + ObtainDocumentType[]: + IsSchemaTypeFromBuiltinClass extends true ? + ObtainRawDocumentPathType[] : + IsItRecordAndNotAny extends true ? + Item extends Record ? + ObtainRawDocumentPathType[] : + Array> : + ObtainRawDocumentPathType[] + >: + PathValueType extends StringSchemaDefinition ? PathEnumOrString : + IfEquals extends true ? PathEnumOrString : + IfEquals extends true ? PathEnumOrString : + PathValueType extends NumberSchemaDefinition ? Options['enum'] extends ReadonlyArray ? Options['enum'][number] : number : + IfEquals extends true ? number : + PathValueType extends DateSchemaDefinition ? Date : + IfEquals extends true ? Date : + PathValueType extends typeof Buffer | 'buffer' | 'Buffer' | typeof Schema.Types.Buffer ? Buffer : + PathValueType extends BooleanSchemaDefinition ? boolean : + IfEquals extends true ? boolean : + PathValueType extends ObjectIdSchemaDefinition ? Types.ObjectId : + IfEquals extends true ? Types.ObjectId : + IfEquals extends true ? Types.ObjectId : + PathValueType extends 'decimal128' | 'Decimal128' | typeof Schema.Types.Decimal128 ? Types.Decimal128 : + IfEquals extends true ? Types.Decimal128 : + IfEquals extends true ? Types.Decimal128 : + IfEquals extends true ? bigint : + IfEquals extends true ? bigint : + PathValueType extends 'bigint' | 'BigInt' | typeof Schema.Types.BigInt | typeof BigInt ? bigint : + PathValueType extends 'uuid' | 'UUID' | typeof Schema.Types.UUID ? Buffer : + IfEquals extends true ? Buffer : + PathValueType extends MapConstructor | 'Map' ? Map> : + IfEquals extends true ? Map> : + PathValueType extends ArrayConstructor ? any[] : + PathValueType extends typeof Schema.Types.Mixed ? any: + IfEquals extends true ? any: + IfEquals extends true ? any: + PathValueType extends typeof SchemaType ? PathValueType['prototype'] : + PathValueType extends Record ? ObtainDocumentType : + unknown; +} diff --git a/types/models.d.ts b/types/models.d.ts index 45c0d6bf93d..34f938c07eb 100644 --- a/types/models.d.ts +++ b/types/models.d.ts @@ -569,6 +569,12 @@ declare module 'mongoose' { Array>> >; + /** + * List all [Atlas search indexes](https://www.mongodb.com/docs/atlas/atlas-search/create-index/) on this model's collection. + * This function only works when connected to MongoDB Atlas. + */ + listSearchIndexes(options?: mongodb.ListSearchIndexesOptions): Promise>; + /** The name of the model */ modelName: string; diff --git a/types/mongooseoptions.d.ts b/types/mongooseoptions.d.ts index 7fec10b208f..9c35ab8222b 100644 --- a/types/mongooseoptions.d.ts +++ b/types/mongooseoptions.d.ts @@ -203,6 +203,13 @@ declare module 'mongoose' { */ toObject?: ToObjectOptions; + /** + * Set to true to make Mongoose use Node.js' built-in AsyncLocalStorage (Node >= 16.0.0) + * to set `session` option on all operations within a `connection.transaction(fn)` call + * by default. Defaults to false. + */ + transactionAsyncLocalStorage?: boolean; + /** * If `true`, convert any aliases in filter, projection, update, and distinct * to their database property names. Defaults to false. diff --git a/types/schemaoptions.d.ts b/types/schemaoptions.d.ts index 31795187cf0..4df87a806ea 100644 --- a/types/schemaoptions.d.ts +++ b/types/schemaoptions.d.ts @@ -120,6 +120,10 @@ declare module 'mongoose' { * to all queries derived from a model. */ read?: string; + /** + * Set a default readConcern for all queries at the schema level + */ + readConcern?: { level: 'local' | 'available' | 'majority' | 'snapshot' | 'linearizable' } /** Allows setting write concern at the schema level. */ writeConcern?: WriteConcern; /** defaults to true. */ diff --git a/types/types.d.ts b/types/types.d.ts index f63b1934907..08f90c6184c 100644 --- a/types/types.d.ts +++ b/types/types.d.ts @@ -83,7 +83,7 @@ declare module 'mongoose' { class ObjectId extends mongodb.ObjectId { } - class Subdocument extends Document { + class Subdocument extends Document { $isSingleNested: true; /** Returns the top level document of this sub-document. */