-
Notifications
You must be signed in to change notification settings - Fork 1.7k
feat: Implement log for Decimal256 #17918
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements support for the log
function with Decimal256 values that are larger than i128. Previously, the function would return a "Not yet implemented" error for large Decimal256 values.
- Adds
decimal256_to_i256
utility function for handling Decimal256 scaling - Implements proper Decimal256 logarithm calculation with fallback to f64 approximation for very large values
- Updates test cases to verify the new functionality works correctly
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
File | Description |
---|---|
datafusion/functions/src/utils.rs | Adds decimal256_to_i256 utility function for converting Decimal256 values to unscaled i256 |
datafusion/functions/src/math/log.rs | Implements full Decimal256 support in logarithm functions with i256_to_f64 conversion helper |
datafusion/sqllogictest/test_files/decimal.slt | Updates test cases to verify Decimal256 logarithm functionality instead of expecting errors |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
// For larger values, use string conversion (less efficient but more accurate) | ||
// Parse the string representation | ||
let value_str = value.to_string(); | ||
value_str.parse::<f64>().unwrap_or(f64::INFINITY) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
String conversion and parsing is inefficient for i256 to f64 conversion. Consider implementing a more direct conversion method using bit manipulation or splitting the i256 into manageable chunks.
// For larger values, use string conversion (less efficient but more accurate) | |
// Parse the string representation | |
let value_str = value.to_string(); | |
value_str.parse::<f64>().unwrap_or(f64::INFINITY) | |
// For larger values, convert using bit manipulation | |
// i256 is represented as 4 u64 limbs (little-endian) | |
let limbs = value.0; | |
let mut abs = [0u64; 4]; | |
let negative = value.is_negative(); | |
if negative { | |
// Two's complement: invert and add 1 | |
let mut carry = true; | |
for (i, limb) in limbs.iter().enumerate() { | |
let inv = !limb; | |
let (res, overflow) = if carry { | |
inv.overflowing_add(1) | |
} else { | |
(inv, false) | |
}; | |
abs[i] = res; | |
carry = overflow; | |
} | |
} else { | |
abs.copy_from_slice(&limbs); | |
} | |
// Combine limbs into f64 | |
let mut result = 0f64; | |
for i in (0..4).rev() { | |
result = result * (1u64 << 64) as f64 + abs[i] as f64; | |
} | |
if negative { | |
-result | |
} else { | |
result | |
} |
Copilot uses AI. Check for mistakes.
// For larger values, use string conversion (less efficient but more accurate) | ||
// Parse the string representation | ||
let value_str = value.to_string(); | ||
value_str.parse::<f64>().unwrap_or(f64::INFINITY) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using f64::INFINITY
as a fallback for parse errors could mask legitimate parsing failures and produce incorrect logarithm results. Consider returning an error or using a more appropriate fallback value.
Copilot uses AI. Check for mistakes.
Which issue does this PR close?
Rationale for this change
What changes are included in this PR?
Decimal256
values forlog
Are these changes tested?
Yes
Are there any user-facing changes?