Skip to content

Fail fast on creating mutations that cannot be stored #1919

@alugowski

Description

@alugowski

This is in the context of Beam CloudBigtableIO. The client has a bulk mutation limit of 100k. As of #1497 this means as long as each individual Put is under that limit everything will be fine.

A Dataflow job can have large variability in the amount of data per row. It is also natural to write processElement using a single Put, especially since that is what the examples guide new users to do.

If a single Put is larger than 100k mutations then it is impossible to store. The exception it causes is thrown later, and from a different thread.

The client should throw the TooManyMutations exception much earlier, preferably so the user code that caused the 100001th mutation it is still in the stack trace.

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: bigtableIssues related to the googleapis/java-bigtable-hbase API.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions