feat: ArrayEquals kernel
#4057
Draft
feat: ArrayEquals kernel
#4057
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: blaginin <[email protected]>
# Conflicts: # vortex-array/src/compute/mod.rs
Deploying vortex-bench with
|
| Latest commit: |
a907efe
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://15d5d328.vortex-93b.pages.dev |
| Branch Preview URL: | https://db-kernel-eq.vortex-93b.pages.dev |
Signed-off-by: blaginin <[email protected]>
- Change array_equals to take two arrays as arguments - Fix imports and add missing 'use' keyword - Update ArrayEqualsArgs to expect 2 inputs instead of 3 - Use ARRAY_EQUALS_FN instead of IS_SORTED_FN 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add chunked comparison with early exit for performance - Use constant array checks to quickly detect differences - Add fallback to canonical arrays for non-canonical inputs - Use 64K batch size for chunked processing 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Treat NULL == NULL as true for array equality - Check individual comparison results when not constant - Add proper null handling in chunked comparison 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Major changes: - Extract chunked comparison logic into separate functions - Move statistics checking into dedicated helper - Add configurable batch_size parameter (with 64K default) - Split comparison result checking into modular functions - Add comprehensive tests including float precision and chunked arrays Functions added: - compare_chunked(): Main chunked comparison with configurable batch size - compare_batch(): Single batch comparison - check_constant_result(): Handle constant comparison results - check_non_constant_result(): Handle element-wise checking with stats optimization - check_comparison_stats(): Quick rejection via min/max stats - check_null_equality(): Proper null comparison handling - check_stats_equality(): Early exit via statistics comparison Tests added: - test_float_precision(): Verify float comparison behavior - test_batch_size_functionality(): Test large array handling - test_primitive_vs_dict_array(): Test different encodings 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Add missing patterns found in compare.rs, filter.rs, and cast.rs: - Add debug logging for fallback operations - Add constant null array handling (NULL constants equal only to NULL constants) - Add TODO comment for future floating-point optimizations - Add comprehensive test for constant null arrays These changes align array_equals with established patterns in the codebase and improve debuggability and correctness for edge cases. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…ations - Add early return for empty arrays regardless of type - Implement comprehensive constant array optimization with statistics checks - Use min/max statistics to rule out equality for non-null constants - Add null count statistics check for null constant comparisons - Remove redundant empty array and constant null checks - Add mixed constant/non-constant array test coverage - Fix clippy warnings by removing unnecessary clones 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
This test demonstrates that arrays containing -0.0 and +0.0 are not considered equal, even though they should be according to IEEE 754 (-0.0 == +0.0 returns true). The failing test shows a correctness issue in the array equality implementation that needs to be addressed. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
robert3005
reviewed
Jul 30, 2025
|
I always found this weird to be a kernel. I think once we land #4188 this will be easier to implement in a reasonable way. Otherwise I would define this as PartialEq for Canonical since you really shouldn't have to dispatch this to the encoding |
|
Agreed! Joe and Nick mentioned the kernel trait may change quite a bit, so holding this off until it's merged |
|
I would say define partial eq for canonical. I think this will be useful |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #1424