Skip to content

Conversation

@prabod
Copy link
Contributor

@prabod prabod commented Aug 20, 2025

Description

This PR adds the ranking finisher

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Code improvements with no or little impact
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING page.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@prabod prabod requested a review from DevinTDHa August 20, 2025 07:48
@prabod prabod self-assigned this Aug 20, 2025
@coveralls
Copy link

Pull Request Test Coverage Report for Build 17203281051

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 73 of 89 (82.02%) changed or added relevant lines in 1 file are covered.
  • 20 unchanged lines in 14 files lost coverage.
  • Overall coverage increased (+0.1%) to 54.022%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/main/scala/com/johnsnowlabs/nlp/finisher/GGUFRankingFinisher.scala 73 89 82.02%
Files with Coverage Reduction New Missed Lines %
src/main/scala/com/johnsnowlabs/nlp/training/CoNLLHelper.scala 1 98.18%
src/main/scala/com/johnsnowlabs/partition/Partition.scala 1 77.78%
src/main/scala/com/johnsnowlabs/reader/ExcelReader.scala 1 90.11%
src/main/scala/com/johnsnowlabs/reader/Reader2Doc.scala 1 95.81%
src/main/scala/com/johnsnowlabs/reader/TextReader.scala 1 89.29%
src/main/scala/com/johnsnowlabs/storage/HasStorageModel.scala 1 75.0%
src/main/scala/com/johnsnowlabs/storage/HasStorage.scala 1 51.35%
src/main/scala/com/johnsnowlabs/util/Benchmark.scala 1 70.59%
src/main/scala/com/johnsnowlabs/util/FileHelper.scala 1 36.0%
src/main/scala/com/johnsnowlabs/nlp/pretrained/S3ResourceDownloader.scala 2 53.23%
Totals Coverage Status
Change from base Build 17064382814: 0.1%
Covered Lines: 11181
Relevant Lines: 20697

💛 - Coveralls

@DevinTDHa DevinTDHa changed the base branch from release/612-release-candidate to release/613-release-candidate September 1, 2025 10:33
@DevinTDHa DevinTDHa force-pushed the SPARKNLP-1286-GGUFRankingFinisher branch from 22ed751 to d20e359 Compare September 1, 2025 10:34
@DevinTDHa DevinTDHa merged commit 1d534f7 into release/613-release-candidate Sep 1, 2025
1 of 4 checks passed
@DevinTDHa DevinTDHa mentioned this pull request Sep 1, 2025
10 tasks
DevinTDHa added a commit that referenced this pull request Sep 1, 2025
* Add GGUFRankingFinisher and corresponding tests for ranking capabilities

* Add GGUFRankingFinisher implementation and tests for ranking functionality

* Update test case to tag "finisher the reranked documents" as SlowTest

* Add documentation for GGUFRankingFinisher with features, usage examples, and output schema

* Resolve partition warning for windowing

* Add GGUFRankingFinisher notebook

* Change pretrained AutoGGUFReranking model

---------

Co-authored-by: Devin Ha <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants