Skip to content

Conversation

@lawmicha
Copy link
Contributor

@lawmicha lawmicha commented Aug 18, 2022

Issue #, if available:

Table of Contents:

  1. feat(datastore): LazyModel Implementation #2174 (You are here)
  2. test(datastore): Lazy loading integration tests #2448
  3. test(api): Lazy loading integration tests #2449

Description of changes:
This PR is the first of a series of PRs for lazy loading, and consists of the LazyModel implementation for DataStore and API.

Codegen

The first hand written swift model types is included here using the following bi-directional has-many belongs-to schema from the V2 Transformer tests:

# 11 Explicit Bi-Directional Belongs to Relationship

type Post4V2 @model @auth(rules: [{allow: public}]) {
  id: ID!
  title: String!
  comments: [Comment4V2] @hasMany(indexName: "byPost4", fields: ["id"])
}

type Comment4V2 @model @auth(rules: [{allow: public}]) {
  id: ID!
  postID: ID! @index(name: "byPost4", sortKeyFields: ["content"])
  content: String!
  post: Post4V2 @belongsTo(fields: ["postID"])
}

DataStore LazyModel

When querying for a child model, like Comment, the FK value of Post is retrieve as part of the SQL query for Comment. This FK value of the Post is stored in the LazyModel.

  • Statement from SQL will be converted to a json object to run the decoder
  • the decoder will call into the LazyModel's decoder
  • Lazy Model's decoder will attempt to decode using the DataStoreModelDecoder
  • DataStoreModelDecoder will return true if it detects "DataStoreModelIdentifierMetadata"
  • DataStoreModelDecoder will create a DatastoreModelProvider with the metadata
  • The LazyModel is instanitated with the DataStoreModelProvider that can perform lazy loading

API LazyModel

When querying for a child model, like Comment, the selection set created will be nested one level, with the nested level containing only the identifiers of the model, in this case just the identifiers of the Post.

  • Selection set is generated containing the nested level, only identifier fields.
  • The GraphQL response is received containing a data object that has the comment values and post identifier values.
  • The data is decoded into the Comment with the LazyModel
  • LazyModel store the identifiers extract from the GraphQL response payload.

Lazy Load will use the identifiers and construct a customized GraphQL request with the variables containing the map of identifiers.

Lazy Loading composite keys

Loading composite key for both the DataStore and API 's LazyModel ModelProvider has some challenges. For DataStore, the identifier stored will be "@@PrimaryKey" --> "[FK value]" which is in the format of "[identifier1]#[identifier2]". Later when this is loaded, the value gets used as a predicate into DataStore, specifically the predicate construction will look like this:
fields("@@primaryKey").eq("[identifier1]#[identifier2]")

For API, the LazyModel is decoded from the GraphQL data so the identifiers map extends to all scenarios:

  • when single identifiers, the identifiers is just "id" -> "[identifier value]"
  • for renamed identifiers then it's "[renamedIdKey]" -> "[identifier value]"
  • for composite keys, the map is populated with more than one key value pair.
    When the data is loaded, the map is translated to the input object, and the document contains references to the input object's values and their type, looking like this:
// the lazy model's loading functionality will create a GraphQL request
identifiers = ["id": "postId123", "index": "4"]
documentBuilder.append(ModelIdDecorator(identifiers: identifiers, schema: schema))
let request = documentBuilder.build()

// request.document
queryPost(id: ID!, index: String!)

// request.variables
{
  "id: "postId123",
  "index": "4"
}

Eager Loading a LazyModel instance

The decoders require to have an ordering among them, we can technically always put the API model decoder and DataSore model decoder first so that when it attempts to use the decoders, and both are registered on the model decoder registry, then if it is successfully in decoding, it would have attempted DataStore before API. Both decoders are successful in decoding eager loaded models, so either providers can be created. However, if the payload can be decoded to the notLoaded LazyModel then we may have a problem where the lazy load will call into the incorrectly decoded model provider

Check points: (check or cross out if not relevant)

  • Added new tests to cover change, if needed
  • Build succeeds with all target using Swift Package Manager
  • All unit tests pass
  • All integration tests pass
  • Security oriented best practices and standards are followed (e.g. using input sanitization, principle of least privilege, etc)
  • Documentation update for the change if required
  • PR title conforms to conventional commit style
  • If breaking change, documentation/changelog update with migration instructions

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@lawmicha lawmicha force-pushed the lawmicha.lazymodel2 branch from db36074 to 3ebefdd Compare August 19, 2022 13:40
Base automatically changed from lawmicha.list to dev-preview August 19, 2022 14:11
@lawmicha lawmicha force-pushed the lawmicha.lazymodel2 branch from 3ebefdd to 81df154 Compare August 19, 2022 14:48
@lawmicha lawmicha temporarily deployed to UnitTest August 19, 2022 14:49 Inactive
@lawmicha lawmicha temporarily deployed to UnitTest August 19, 2022 14:49 Inactive
@lawmicha lawmicha force-pushed the lawmicha.lazymodel2 branch from 81df154 to c17464d Compare August 20, 2022 00:36
@lawmicha lawmicha temporarily deployed to UnitTest August 20, 2022 00:36 Inactive
@lawmicha lawmicha temporarily deployed to UnitTest August 20, 2022 00:36 Inactive
@lawmicha lawmicha temporarily deployed to UnitTest August 20, 2022 00:36 Inactive
@lawmicha lawmicha force-pushed the lawmicha.lazymodel2 branch from c17464d to b28edbb Compare August 22, 2022 14:30
@lawmicha lawmicha temporarily deployed to UnitTest August 22, 2022 14:30 Inactive
@lawmicha lawmicha temporarily deployed to UnitTest August 22, 2022 14:30 Inactive
@lawmicha lawmicha force-pushed the lawmicha.lazymodel2 branch from b28edbb to 2a25311 Compare August 22, 2022 16:08
@lawmicha lawmicha temporarily deployed to UnitTest August 22, 2022 16:08 Inactive
@lawmicha lawmicha temporarily deployed to UnitTest August 22, 2022 16:08 Inactive
@lawmicha lawmicha force-pushed the lawmicha.lazymodel2 branch 2 times, most recently from f733791 to d621f0f Compare October 14, 2022 03:25
@lawmicha lawmicha changed the title feat: LazyModel for API and DataStore feat(datastore): LazyModel Implementation Oct 14, 2022
@lawmicha lawmicha force-pushed the lawmicha.lazymodel2 branch from d621f0f to 69b5b0b Compare October 17, 2022 09:17
@harsh62 harsh62 added the datastore Issues related to the DataStore category label Nov 2, 2022
@lawmicha lawmicha closed this Dec 22, 2022
@lawmicha lawmicha deleted the lawmicha.lazymodel2 branch January 26, 2023 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

datastore Issues related to the DataStore category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants