CNDB-15485: Fix ResultRetriever key comparison to prevent dupes in result set

[michaeljmarshall]

(cherry picked from commit ada025c)

Copy of #2023, but targeting main

What is the issue

https://github.com/riptano/cndb/issues/15485

What does this PR fix and why was it fixed

This PR fixes a bug introduced to this branch via #1884. The bug only impacts SAI file format aa when the index file was produced via compaction, which is why the modified test simply adds coverage to compact the table and hit the bug.

The bug happens when an iterator produces the same partition across two different batch fetches from storage. These keys were not collapsed in the key.equals(lastKey) logic because compacted indexes use a row id per row instead of per partition, and the logic in PrimaryKeyWithSource considers rows with different row ids to be distinct. However, when we went to materialize a batch from storage, we hit this code:

        ClusteringIndexFilter clusteringIndexFilter = command.clusteringIndexFilter(firstKey.partitionKey());
        if (cfs.metadata().comparator.size() == 0 || firstKey.hasEmptyClustering())
        {
            return clusteringIndexFilter;
        }
        else
        {
            nextClusterings.clear();
            for (PrimaryKey key : keys)
                nextClusterings.add(key.clustering());
            return new ClusteringIndexNamesFilter(nextClusterings, clusteringIndexFilter.isReversed());
        }

which returned clusteringIndexFilter for aa because those indexes do not have the clustering information. Therefore, each batch fetched the whole partition (which was subsequently filtered to the proper results), and produced a multiplier effect where we saw batch many duplicates.

This fix works by comparing partition keys and clustering keys directly, which is a return to the old comparison logic from before #1884. There was actually a discussion about this in the PR to main, but unfortunately, we missed this case #1883 (comment).

A more proper long term fix might be to remove the logic of creating a PrimaryKeyWithSource for AA indexes. However, I preferred this approach because it is essentially a revert instead of fixing forward solution.

[github-actions]

Checklist before you submit for review

• This PR adheres to the Definition of Done
• Make sure there is a PR in the CNDB project updating the Converged Cassandra version
• Use NoSpamLogger for log lines that may appear frequently in the logs
• Verify test results on Butler
• Test coverage for new/modified code is > 80%
• Proper code formatting
• Proper title for each commit staring with the project-issue number, like CNDB-1234
• Each commit has a meaningful description
• Each commit is not very long and contains related changes
• Renames, moves and reformatting are in distinct commits
• All new files should contain the DataStax copyright header instead of the Apache License one

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
89.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[cassci-bot]

✔️ Build ds-cassandra-pr-gate/PR-2024 approved by Butler

---

Approved by Butler
See build details here