Marqeta C# Core API SDK

Our C# SDK for the Marqeta Core API, generated by Kiota.

Tip

If you're here because the sky is falling and you need to make a very quick fix to the SDK without modifying the script please see instructions here.

Documentation

Marqeta API

For complete reference API documentation, see the Marqeta Core API Reference.

Kiota tooling

For reference on Kiota, from tooling, to using the generated client and its concepts please visit the MS docs here, or the GitHub repository.

Dependencies

.NET 8.0 SDK

The tool will install a local tool for Kiota as defined in .config/dotnet-tools.json.

Test secrets

The SDK generation script also builds and tests the generated client. To run tests locally, user secrets need to be configured to use your developer public sandbox instance.

In the Marqeta.Core.Sdk.Tests directory, run the following commands:

dotnet user-secrets init
dotnet user-secrets set "Marqeta:BaseUrl" "https://sandbox-api.marqeta.com/v3/"
dotnet user-secrets set "Marqeta:UserName" "<Application token>"
dotnet user-secrets set "Marqeta:Password" "<Access Token>"

You can obtain a developer public sandbox to obtain the needed credentials by signing up to marqeta as per instructions here.

Generating the SDK

Execute the dotnet fsi GenerateSdkFromSourceUrl.fsx command in the root source directory. This will execute the F# script via F# interactive.

What does the script do?

1. Downloads the latest CoreAPI.yaml from Marqeta OpenAPI repository.
2. Parses it with OpenAPI.NET.
3. Saves the parsed file to disk as Marqeta.Core.SdkSourceCoreAPI.yaml, this allows us to keep formatting and ordering consistent for easier diffs.
4. Applies a variety of modifications to the OpenAPI specification.
5. Validates the modified specification.
6. Saves the modified specification to disk as Marqeta.Core.Sdk/CoreAPI.yaml.
7. Installs tools specified in .config/dotnet-tools.json (currently only Kiota).
8. Invokes Kiota to generate a C# client in Marqeta.Core.Sdk/Generated, if a client already exists (denoted by the presence of kiota-lock.json), it'll update the existing client.
9. Builds the solution and run all tests.

Updating dependencies

Packages in GenerateSdkFromSourceUrl.fsx script

Nuget packages needed for the script to run

1. Check for new versions of the packages referenced at the top of file prefaced with #r
2. Check for breaking changes
3. Update versions if appropriate
4. Re-run script
5. Commit and push

Kiota CLI

The Kiota dotnet CLI tool used to generate the output SDK from the OpenAPI specification file

1. Check for relevant breaking changes on the GitHub Release Page
2. Run the command dotnet tool update microsoft.openapi.kiota in the root directory of repository to update the tool
3. Run the GenerateSdkFromSourceUrl.fsx script
4. If build and tests succeed then commit and push

Marqeta SDK - Kiota nuget packages

Kiota nuget packages added as a dependency to the Marqeta SDK project

1. Check for relevant breaking changers on the GitHub Release Page
2. Update all Kiota nuget packages Microsoft.Kiota.*
3. Diff the following files with the same file of each found in Kiota dotnet GitHub
   • Marqeta.Core.Sdk/Serialization/Json/CustomJsonParseNode.cs
   • Marqeta.Core.Sdk/Serialization/Json/CustomJsonParseNodeFactory.cs
   • Marqeta.Core.Sdk/Serialization/Json/TypeConstants.cs
   • Marqeta.Core.Sdk/Serialization/Text/CustomTextParseNode.cs
   • Marqeta.Core.Sdk/Serialization/Text/CustomTextParseNodeFactory.cs
4. Pay attention to changes listed in serialization changes, as well as any comments starting with // Modified:
5. Bring our versions of these files inline where appropriate to make sure we're following Kiotas general standards for serialization and for bug/performance fixes
6. If build and tests succeed then commit and push

Marqeta SDK - Misc nuget packages

Miscellaneous nuget packages added as a depdendency to the Marqeta SDK project for things like IoC (also covers nuget packages for the Marqeta.Core.Sdk.Tests project)

1. Check for relevant breaking changes
2. If versions have updated, then update these where appropriate
3. If build and tests succeed then commit and push

Note

Updating Kiota

When updating Kiota it is best to update the CLI and packages together in the same release as the Kiota generation depends on its abstractions and the generated SDK may require changes not present in our versions of the Kiota nuget packages.

Making changes

1. In the GenerateSdkFromSourceUrl.fsx script, make changes in the OpenApiHelpers module, there are a lot of examples of modifications in there already, so if you're unsure follow an existing example.
   • There are a lot of functions for different sections, mostly following the hierarchical structure of the OpenAPI specification, with functions for specific schema models (e.g. #/components/schemas/transaction_model, #/components/schemas/card_holder_model).
   • Ensure the changes have relevant comments.
2. Execute the script to generate the new SDK and validate your changes.
   • These should be visible in the Marqeta.Core.Sdk/CoreAPI.yaml diff, and also in the generated code SDK.
3. Add tests if needed, and validate these pass.
4. Update documentation (this README for example).
5. Commit and push.

Important

Make sure to include the changes to kiota-lock.json, SourceCoreAPI.yaml, and CoreAPI.yaml in your commit.

Caution

Emergency changes (bypass script)

If for some reason the script isn't working, or we need to make a quick and dirty fix due to a production issue, we can make a very quick change with the following instructions.

Please note that this should only be done as a last resort, and any changes MUST be added to both the script and documentation in a subsequent PR as soon as possible.

1. Make your change directly to the Marqeta.Core.Sdk/CoreAPI.yaml file.
2. Ensure the required dotnet tools are installed locally by executing the dotnet tool restore command.
3. Execute the kiota update command dotnet tool run kiota update -o Marqeta.Core.Sdk/Generated --clean-output --clear-cache.
4. Build the solution and run all tests.
5. Commit and push.

Note

Gotchas and known issues

1. Kiota performs type trimming to remove unused types, so it won't generate models that aren't directly referenced, if models are missing, please manually add a representative model to Marqeta.Core.Sdk/Extensions
2. Kiota doesn't handle different response content types for the same status code, Marqeta can return HTML error responses sometimes, this is unstructured data so we can't bind it to a model, out of the box this information is lost, however, we have added a text/html parser to manually add this data to our ApiError model. This can be found in Marqeta.Core.Sdk/Serialization/Text.
   • This change makes it so that on calling GetObjectValue<T> for the IParseNode, will create a new ApiError type (if applicable) and set the MessageEscaped to the HTML text returned.
3. Kiota doesn't generate enum path parameters (for C#, it was added to other languages), we don't have a workaround yet, so during usage we're converting the enum to it's string representation, one problem is that this causes the enum types not to be generated, the webhook EventType for example is not generated, so we've manually added this enum, an issue is raised on the Kiota GitHub repository here.
4. The default Kiota JSON deserialization implementation will populate null if it can't parse a value, this obviously isn't great for us, we want to have loud shouting errors if we're unable to correctly parse a response rather than null values, so we've implemented our own IParseNode for JSON in Marqeta.Core.Sdk/Serialization/Json (modified version of the default Kiota JSON deserialization implementation), and there is an issue raised on the Kiota GitHub repository here.
5. The generated client doesn't have an interface, which makes unit testing difficult, please refer to Kiota unit testing docs for more information.

Current changes made to OpenAPI specfication

Changes to the schema models (#/components/schemas)

• Global (applied to all models)
  • Mark properties as readonly false, some requests have properties set as readonly true which breaks SDK generation, meaning we can't set these values.
  • Done in the applySchemaPropertiesModifications function.
• #/components/schemas/mcc_group_model docs
  • Change the property mcc from an array of objects to an array of strings.
  • Done in the applyMccGroupModelModifications function.
• #/components/schemas/card_holder_model docs
  • Add a new missing property status, this is an enum that adds the following values: UNVERIFIED, LIMITED, ACTIVE, SUSPENDED, CLOSED
  • Done in the applyCardHolderModelModifications function.
• #/components/schemas/transaction_model docs
  • Add missing enum values to the type property, the missing values added are: address.verification, authorization.clearing.representment, billpayment, billpayment.clearing, billpayment.reversal, fee.charge.pending.refund, transaction.unknown
  • Done in the applyTransactionModelTypeModifications function.
• #/components/schemas/transaction_model/transaction_metadata JIT Funding decision: Transaction Metadata docs, Transaction docs
  • Add the EU_MOTO_NON_SECURE enum to payment_channel property, this is because Marqeta keep sending it via webhooks, although this is meant to be an internal enum, and causes transactions to fail deserialization.
  • Done in the applyTransactionMetadataPaymentChannelModifications function.
• Remove the #/components/schemas/BadRequestError, #/components/schemas/Error, #/components/schemas/ForbiddenError, #/components/schemas/InternalServerError, #/components/schemas/UnauthorizedError models from the schema. This is because most paths/operations in the OpenAPI specification don't have any error models defined, there's also the fact we don't want an error model per response code, Kiota adds the response status code to the base ApiException for us, so we created our own shared ApiError (mentioned below).
  • Done in the removeUnusedErrorSchemas function.
• Add a new #/components/schemas/ApiError model, this has the properties error_code and error_message on it, which bind to the API error response typically returned by Marqeta (note their docs don't explicitly mention this format).
  • Done in the addErrorSchema function.
• #/components/schemas/pos docs
  • Add the UNSCHEDULED_CARD_ON_FILE missing enum value to the transaction_initiated_category property. This value is sent via webhooks, and causes deserialization errors if absent (note: this value is not documented as of writing).
  • Done in the applyPosModelModifications function.
• #/components/schemas/Terminal_model
  • Add the UNSCHEDULED_CARD_ON_FILE missing enum value to the transaction_initiated_category property. This value is sent via webhooks, and causes deserialization errors if absent (note: this value is not documented as of writing).
  • Done in the applyTerminalModelModifications function.

Changes to the Paths (e.g. /api/customer) and Operations (GET, POST, PUT, etc...)

• Adds/replace default response on all operations for all paths to be ApiError.
  • This specifies that all unspecified responses are to try to bind to ApiError, in practice this means all 4XX and 5XX responses, but could include other unhandled response codes.
  • Done in the addOrReplaceDefaultErrorResponse function.
• Remove all existing 4XX and 5XX responses on all operations for all paths.
  • This is because we add a default response of ApiError ourselves, most of the 4XX and 5XX response specifications are actually empty objects anyway, so won't generate anything to bind to.
    The only operations that currently have a valid response specification schema are the POST /feedback/fraud endpoint, but we remove these and use our own model (they're removed from #/components/schemas too as part of schema model modifications mentioned above).
  • Done in the applyOperationsModifications function.
• Remove all examples for every response and request for all paths and operations.
  • These don't actually add any value to the SDK generation, but they do create a lot of noise in validation output due to the examples not matching the specification in a lot of cases.
  • Done in the applyRequestModifications → removeOpenApiMediaTypeExamples, applyResponseModifications → removeOpenApiMediaTypeExamples functions.
• Add the authorization reversal path to path list manually
  • This endpoint is currently undocumented. A method has been added to append it the paths present if it is not added by Marqeta
  • Done in the addAuthorizationReversalPath function

Custom serialization

As alluded to in the Gotchas and known issues section, we've had to add some custom deserializers to support our needs.

Kiota doesn't use standard deserialization methods, but have instead opted to use a common interface across all languages supported by its generator, there are some docs on this.

However, the tl;dr is that we need an IParseNodeFactory as well as an IParseNode for each MIME type we want to deserialize (application/json and text/html) in our case.

For these to get used by the generated client they need to be specified in the kiota-lock.json as below:

"deserializers": [
    "Marqeta.Core.Sdk.Serialization.Text.TextHtmlParseNodeFactory", // Our text/html parse node factory
    "Marqeta.Core.Sdk.Serialization.Json.CustomJsonParseNodeFactory", // Our application/json parse node factory
    "Microsoft.Kiota.Serialization.Text.TextParseNodeFactory",
    "Microsoft.Kiota.Serialization.Form.FormParseNodeFactory"
]

These are added as part of the original kiota generate command by adding the following arguments to the command --deserializer Marqeta.Core.Sdk.Serialization.Text.TextHtmlParseNodeFactory and --deserializer Marqeta.Core.Sdk.Serialization.Json.CustomJsonParseNodeFactory Deserializer argument docs, Serializer argument docs.

Important

Specifying deserializers (or serializers for that matter) as part of the kiota generate command will remove all defaults, so you need to add the other required options manually like so --deserializer Microsoft.Kiota.Serialization.Text.TextParseNodeFactory.

If updating an existing SDK, anything already in the kiota-lock.json will be used, so if you need to add a new serializer/deserializer for an update of a client, manually add it there.

text/html

The implementation for text/html is borrowed from the default Kiota TextParseNodeFactory and TextParseNode supplied in Microsoft.Kiota.Serialization.Text GitHub, and can be found in Marqeta.Core.Sdk/Serialization/Text.

We change the GetObjectValue<T> to check if the type we're trying to deserialize into is of ApiError, if so we just put the contents of the _text property on the IParseNode into ApiError.MessageEscaped.

application/json

The implementation for application/json is borrowed from default Kiota JsonParseNodeFactory and JsonParseNode supplied in Microsoft.Kiota.Serialization.Json GitHub, and can be found in Marqeta.Core.Sdk/Serialization/Json.

The main changes we've made here is to remove the safety around parsing so it will fail loudly, this is because by default Kiota will return null for values it can't parse, this doesn't quite work for us.

So instead we remove all the safety checks and wrap the field assignments in AssignFieldValues<T> in a try-catch to throw a JsonException when we fail to parse.

We also customised the JsonSerializerOptions with JsonSerializerDefaults.Web in our CustomJsonParseNodeFactory which gets set on the KiotaJsonSerializationContext.

Lastly we've added a check for an empty/null enum before parsing in GetEnumValue<T>() so that we don't throw an exception if no enum value is provided.