Skip to content

SparkNLP 1256 - Introducing AutoGGUFReranker #14649

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Aug 19, 2025
Merged

SparkNLP 1256 - Introducing AutoGGUFReranker #14649

merged 5 commits into from
Aug 19, 2025

Conversation

@prabod
Copy link
Contributor

@prabod prabod commented Aug 15, 2025

Description

This PR introduces AutoGGUFReranker to SparkNLP

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Code improvements with no or little impact
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING page.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@prabod prabod requested a review from DevinTDHa August 15, 2025 06:56
@coveralls
Copy link

coveralls commented Aug 15, 2025

Pull Request Test Coverage Report for Build 16984826744

Details

  • 0 of 2 (0.0%) changed or added relevant lines in 1 file are covered.
  • 12 unchanged lines in 11 files lost coverage.
  • Overall coverage increased (+0.02%) to 53.911%
Changes Missing Coverage Covered Lines Changed/Added Lines %
src/main/scala/com/johnsnowlabs/nlp/pretrained/ResourceDownloader.scala 0 2 0.0%
Files with Coverage Reduction New Missed Lines %
src/main/scala/com/johnsnowlabs/nlp/annotators/sentence_detector_dl/SentenceDetectorDLModel.scala 1 82.72%
src/main/scala/com/johnsnowlabs/nlp/serialization/Feature.scala 1 50.38%
src/main/scala/com/johnsnowlabs/nlp/util/GraphBuilder.scala 1 95.45%
src/main/scala/com/johnsnowlabs/nlp/util/io/OutputHelper.scala 1 55.13%
src/main/scala/com/johnsnowlabs/nlp/util/io/ResourceHelper.scala 1 47.99%
src/main/scala/com/johnsnowlabs/partition/TitleChunker.scala 1 90.77%
src/main/scala/com/johnsnowlabs/reader/ExcelReader.scala 1 90.11%
src/main/scala/com/johnsnowlabs/reader/Reader2Table.scala 1 80.95%
src/main/scala/com/johnsnowlabs/storage/StorageHelper.scala 1 55.88%
src/main/scala/com/johnsnowlabs/util/ZipArchiveUtil.scala 1 86.67%
Totals Coverage Status
Change from base Build 16749746984: 0.02%
Covered Lines: 11109
Relevant Lines: 20606
💛 - Coveralls

@DevinTDHa DevinTDHa changed the base branch from master to release/612-release-candidate August 18, 2025 09:05
DevinTDHa
DevinTDHa requested changes Aug 18, 2025
View reviewed changes
docs/en/annotator_entries/AutoGGUFReranker.md
.setQuery("A man is eating pasta.")
```

The default model is `"bge-reranker-v2-m3-Q4_K_M"`, if no name is provided.
Copy link
Member

@DevinTDHa DevinTDHa Aug 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to make sure this is uploaded

@prabod
Changes requested
dd33ec9 
Signed-off-by: Prabod Rathnayaka <[email protected]>
@DevinTDHa DevinTDHa merged commit abd4b32 into release/612-release-candidate Aug 19, 2025
4 checks passed
DevinTDHa pushed a commit that referenced this pull request Aug 20, 2025
@prabod @DevinTDHa
[SPARKNLP-1256] - Introducing AutoGGUFReranker (#14649)
335bf1b 
* [SPARKNLP-1256] Add AutoGGUFReranker annotator and its tests

* add resource downloader

Signed-off-by: Prabod Rathnayaka <[email protected]>

* Add example notebook for AutoGGUFReranker model integration in Spark NLP

* Add documentation for AutoGGUFReranker annotator and update annotators list

* Changes requested

Signed-off-by: Prabod Rathnayaka <[email protected]>

---------

Signed-off-by: Prabod Rathnayaka <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@DevinTDHa DevinTDHa DevinTDHa requested changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@prabod @coveralls @DevinTDHa