[Enhancement]: Custom Metadata for Entities and Fields.

[JerryNixon]

It is not possible to include descriptions for entities and fields to enhance both OpenAPI and the GraphQL schema. This metadata can come from many sources: database extended properties, Purview and data dictionaries, or a local file. This is a recommendation with an eye towards future support for other sources.

Value & Goals

1. The OpenAPI output, which is used by customers to create custom documentation, would have more expressive information to ensure the resulting documentation is more understandable and less error-prone.
2. The GraphQL schema, which provides intellisense and can be used to enable exotic features deeper in our backlog, would have more expressive information, improving - at least - the developer experience.

To complete this feature, both OpenAPI and GraphQL Schemas should be updated to incorporate descriptions when present.

Implementation

Two ways, in the config or external.

Inline configuration option

Not all DAB users are gigantic. Some just have a simple database with a few tables to expose through endpoints and they like the self-contained nature of the configuration file. For these users, the inline option is supported.

{
  "$schema": "https://github.com/Azure/data-api-builder/releases/download/v1.1.7/dab.draft.schema.json",
  "data-source": { },
  "data-source-files": [ ],
  "metadata": {
    "entities": [ // new
      {
        "entity-name:" "<entity-name>",
        "fields": [
          {
            "field-name": "string-value", 
            "description": "string-value"
          }
        ]
      }
    ]
  },
  "runtime": { },
  "entities": { }
}

External file option

Some DAB users are gigantic, with hundreds of entities. These scenarios can explode the size of the configuration file - even making it unusable in a practical sense. For these users, the external file option is supported.

{
  "$schema": "https://github.com/Azure/data-api-builder/releases/download/v1.1.7/dab.draft.schema.json",
  "data-source": { },
  "data-source-files": [ ],
  "metadata": {
    "file-paths": ["<string-array>"] // new
  },
  "runtime": { },
  "entities": { }
}

Metadata file

entity-name equates to the entity alias provided in the configuration, not the source value. field-name equates to the name of the column, unless it is mapped in the configuration file; in this case, field-name equates to the mapped name.

[
    {
      "entity-name:" "<entity-name>",
      "fields": [
        {
          "field-name": "string-value", 
          "description": "string-value"
        }
      ]
    }
]

It may be possible to extract this format from metadata providers in a future vers.

Command-line updates

This operation would create or update the value in the configuration file.

dab configure --metadata.file-paths "file1.json, file2.json"

This operation would scaffold a metadata file from configuration.

dab export --metadata --entities * --output metadata.json 
dab export --metadata --entities book,author,publisher --output metadata.json 

This operation add entities and field descriptions to the metadata.

dab configure --metadata.entity "<entity-name>" --field-description "'<field-name>':'description','<field-name>':'description'"

This operation would validate a metadata file against the configuration file.

1. Ensure all entities in metadata.json exist in dab-config.json. All entities in dab-config.json are not required to be in metadata.json
2. Ensure all fields in metadata.json exist in dab-config.json. All fields in dab-config.json are not required to be in metadata.json

dab validate --metadata file.json --configuration dab-config.json

This operation would validate the config's metadata section, if present.

dab validate