SPARKNLP-1260 Introducing Reader2Table Annotator #14640
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces the new
Reader2Tableannotator to spark-nlp, providing a streamlined and user-friendly interface for interacting with Spark NLP readers and integrating with spark-nlp pipelines.Key Improvements:
Simplifies integration with Spark NLP readers through a unified interface
Adds flexibility by enabling more reader-specific configurations
Enhances the maintainability and scalability of data loading workflows
Supported formats include:
Motivation and Context
The current approach to interfacing with Spark NLP readers is fragmented and lacks flexibility, often requiring custom code for handling various input sources and options. This makes onboarding harder for new users and hinders reuse across pipelines.
The
Reader2Tablecomponent abstracts these complexities by:This feature is part of the requirements described in issue Reader Annotators #14624
How Has This Been Tested?
Screenshots (if appropriate):
Types of changes
Checklist: