Skip to content

Conversation

@vkarpov15
Copy link
Collaborator

Re: #14394

Summary

#14394 points out that toObject() gets slow when there's a lot of virtuals. Part of that is we always take the "slow path" when applying virtuals by assuming the virtual is nested.

Consider the following script, 200 documents with 800 virtuals each. The below example uses replica set, but not strictly necessary for this example.

'use strict';

const mongoose = require('mongoose');

void async function main() {
  await mongoose.connect('mongodb://127.0.0.1:27017,127.0.0.1:27018/mongoose_test');

  const ChildSchema = new mongoose.Schema({ name: String, parentId: 'ObjectId' });
  const ChildModel = mongoose.model('Child', ChildSchema);

  const ParentSchema = new mongoose.Schema({
    name: String
  });
  ParentSchema.virtual('child1', { ref: 'Child', localField: '_id', foreignField: 'parentId' });
  ParentSchema.virtual('child2', { ref: 'Child', localField: '_id', foreignField: 'parentId' });
  ParentSchema.virtual('child3', { ref: 'Child', localField: '_id', foreignField: 'parentId' });
  ParentSchema.virtual('child4', { ref: 'Child', localField: '_id', foreignField: 'parentId' });
  const ParentModel = mongoose.model('Parent', ParentSchema);
  /*await ParentModel.deleteMany({});
  await ChildModel.deleteMany({});

  const parents = [];
  for (let i = 0; i < 200; ++i) {
    const parent = await ParentModel.create({ name: 'test parent ' + i });
    const children = [];
    console.log(`${i} / 200`);
    for (let j = 0; j < 200; ++j) {
      children.push({ name: 'test child ' + i + '_' + j, parentId: parent._id });
    }
    await ChildModel.create(children);
    parents.push(parent);
  }

  await new Promise(resolve => setTimeout(resolve, 3000));*/

    const docs = await ParentModel.find().populate(['child1', 'child2', 'child3', 'child4']);
  console.log('Num docs', docs.length, docs[0]?.child1?.length);
  for (const doc of docs) {
    doc.name = 'test parent';
  }
  const start = Date.now();
  docs.forEach(doc => doc.toObject({ virtuals: true }));
  console.log('toObject ms:', Date.now() - start);

  console.log('Done');
  process.exit(0);
}();

Without this change:

$ node gh-14394.js 
Num docs 200 200
BulkSave and toObject ms: 2380
Done

With this change:

$ node gh-14394.js 
Num docs 200 200
BulkSave and toObject ms: 1215
Done

Nearly 2x faster. We can take a look and see if we can do better, but this PR seems like low hanging fruit.

Examples

@vkarpov15 vkarpov15 added this to the 8.3.3 milestone Apr 27, 2024
@vkarpov15 vkarpov15 merged commit 286c7d1 into master Apr 29, 2024
@hasezoey hasezoey deleted the vkarpov15/gh-14394 branch April 30, 2024 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants