Hydrate from JSON not hydrating populated virtual objects

[ilovepixelart]

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Mongoose version

8.2.4

Node.js version

20.12.1

MongoDB server version

7.0.7

Typescript version (if applicable)

5.4.3

Description

Consider such example

Story.ts

import { Schema, model } from 'mongoose'

import type IStory from '../interfaces/IStory'

const StorySchema = new Schema<IStory>({
  userId: {
    type: Schema.Types.ObjectId,
    ref: 'User',
    required: true,
    index: true,
  },
  title: {
    type: String,
    required: true,
  },
}, { timestamps: true })

export default model('Story', StorySchema)

User.ts

import { Schema, model } from 'mongoose'

import type IUser from '../interfaces/IUser'

const UserSchema = new Schema<IUser>({
  name: {
    type: String,
    required: true,
  },
  role: {
    type: String,
    required: true,
  },
  age: {
    type: Number,
  },
}, { timestamps: true })

UserSchema.virtual('stories', {
  ref: 'Story',
  localField: '_id',
  foreignField: 'userId',
})

export default model('User', UserSchema)

hydrate-populate-from-json.test.ts

const user = await User.create({ name: 'Alex', role: 'user' })
const story1 = await Story.create({ title: 'Ticket 1', userId: user._id })
const story2 = await Story.create({ title: 'Ticket 2', userId: user._id })

const populated = await User.findOne({ name: 'Alex' }).populate('stories').lean().exec()

expect(populated).not.toBeNull()
expect(populated?._id instanceof mongoose.Types.ObjectId).toBeTruthy()
expect(populated?.name).toBe('Alex')
expect(populated?.stories).not.toBeNull()
expect(populated?.stories?.length).toBe(2)

// This is what I expect also after hydration see failed comment bellow

expect(populated?.stories?.[0]._id).toEqual(story1._id)
expect(populated?.stories?.[0]._id instanceof mongoose.Types.ObjectId).toBeTruthy()
expect(typeof populated?.stories?.[0]._id).toBe('object')
expect(populated?.stories?.[0].createdAt instanceof Date).toBeTruthy()
expect(typeof populated?.stories?.[0].createdAt).toBe('object')

expect(populated?.stories?.[1]._id).toEqual(story2._id)
expect(populated?.stories?.[1]._id instanceof mongoose.Types.ObjectId).toBeTruthy()
expect(typeof populated?.stories?.[1]._id).toBe('object')
expect(populated?.stories?.[1].createdAt instanceof Date).toBeTruthy()
expect(typeof populated?.stories?.[1].createdAt).toBe('object')

const populatedJson = JSON.stringify(populated)
expect(populatedJson).not.toBeNull()

const hydrated = User.hydrate(JSON.parse(populatedJson))

// Root object was hydrated properly

expect(hydrated).not.toBeNull()
expect(hydrated?._id instanceof mongoose.Types.ObjectId).toBeTruthy()
expect(hydrated?.name).toBe('Alex')
expect(hydrated?.stories).not.toBeNull()
expect(hydrated?.stories?.length).toBe(2)

// Will fail here populated stories array is not hydrated _id and createdAt updatedAt are strings

expect(hydrated?.stories?.[0]._id).toEqual(story1._id)
expect(typeof hydrated?.stories?.[0]._id).toBe('object')
expect(hydrated?.stories?.[0]._id instanceof mongoose.Types.ObjectId).toBeTruthy()
expect(typeof hydrated?.stories?.[0].createdAt).toBe('object')
expect(hydrated?.stories?.[0].createdAt instanceof Date).toBeTruthy()

expect(hydrated?.stories?.[1]._id).toEqual(story2._id)
expect(typeof hydrated?.stories?.[1]._id).toBe('object')
expect(hydrated?.stories?.[1]._id instanceof mongoose.Types.ObjectId).toBeTruthy()
expect(typeof hydrated?.stories?.[1].createdAt).toBe('object')
expect(hydrated?.stories?.[1].createdAt instanceof Date).toBeTruthy()

Weird thing, when I use:

const hydrated = User.hydrate(JSON.parse(populatedJson), undefined, { hydratedPopulatedDocs: true })

Then even the root object now is not hydrated (_id and dates will be strings even on root level)

Steps to Reproduce

Check the description models and test provided

Expected Behavior

After hydrating json I expect to receive list of populated virtuals also in hydrated form

[IslandRhythms]

import { Schema, model, connect, Types, connection } from 'mongoose'
import * as assert from 'assert';

interface IStory {
  _id: Schema.Types.ObjectId,
  userId: Schema.Types.ObjectId,
  title: string,
  createdAt: Date
}

interface IUser {
  name: string,
  role: string,
  age: number,
  stories: [IStory]
}

const StorySchema = new Schema<IStory>({
  userId: {
    type: Schema.Types.ObjectId,
    ref: 'User',
    required: true,
    index: true,
  },
  title: {
    type: String,
    required: true,
  },
}, { timestamps: true })

const UserSchema = new Schema<IUser>({
  name: {
    type: String,
    required: true,
  },
  role: {
    type: String,
    required: true,
  },
  age: {
    type: Number,
  },
}, { timestamps: true })

UserSchema.virtual('stories', {
  ref: 'Story',
  localField: '_id',
  foreignField: 'userId',
});

const User = model('User', UserSchema);
const Story = model('Story', StorySchema);

async function run() {

  await connect('mongodb://localhost:27017');
  await connection.dropDatabase();
  const user = await User.create({ name: 'Alex', role: 'user' })
  const story1 = await Story.create({ title: 'Ticket 1', userId: user._id })
  const story2 = await Story.create({ title: 'Ticket 2', userId: user._id })

  const populated = await User.findOne({ name: 'Alex' }).populate('stories').lean();
  console.log('what is populated', populated);

  assert(populated);
  assert(populated?._id instanceof Types.ObjectId)
  assert.equal(populated.name, 'Alex');
  assert(populated.stories);
  assert.equal(populated.stories.length, 2)


  const populatedJson = JSON.stringify(populated)
  assert(populatedJson)

  const hydrated = User.hydrate(JSON.parse(populatedJson))
  console.log('what is hydrated', hydrated)


  // Will fail here populated stories array is not hydrated _id and createdAt updatedAt are strings

  assert.equal(hydrated.stories[0]._id.toString(), story1._id.toString());
  console.log(typeof hydrated.stories[0]._id)
  assert(typeof hydrated.stories[0]._id == 'object');
  assert(hydrated?.stories?.[0]._id instanceof Types.ObjectId)
  assert(typeof hydrated?.stories?.[0].createdAt == 'object');
  assert(hydrated?.stories?.[0].createdAt instanceof Date)
/*
  assert.equal(hydrated?.stories?.[1]._id.toString(), story2._id.toString())
  assert(typeof hydrated?.stories?.[1]._id == 'object')

  assert(hydrated?.stories?.[1]._id instanceof Types.ObjectId);
  assert(typeof hydrated?.stories?.[1].createdAt == 'object')
  assert(hydrated?.stories?.[1].createdAt instanceof Date)
  */
}

run();

[vkarpov15]

This issue comes down to the fact that hydrate() doesn't recursively hydrate() docs in populated virtuals. I'm working on a fix.

[vkarpov15]

#14533 has a fix. You will need to call hydrate with hydratedPopulatedDocs option: User.hydrate(JSON.parse(populatedJson), null, { hydratedPopulatedDocs: true }).