-
Notifications
You must be signed in to change notification settings - Fork 403
QuantityValue implemented as a fractional number 🐲 #1544
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
- IQuantity interfaces optimized (some methods refactored as extensions) - QuantityInfo/UnitInfo hierachy re-implemented (same properties, different constructors) - QuantityInfoLookup is now public - UntAbbreviationsCache, UnitParser, QuantityParser optimized - UnitConverter: re-implemented (multiple versions) - removed the IConvertible interface - updated the JsonNet converters - introducing the SystemTextJson project - added a new UnitsNetConfiguration to the Samples project showcasing the new configuration options - many more tests and benchmarks (perhaps too many)
@angularsen Clearly, I don't expect this to get merged in the Gitty up! fashion, but at least we have the whole picture, with sources that I can reference. If you want, send me an e-mail, we could do a quick walk-through / discussion. |
…lection constructors with IEnumerable - `UnitAbbreviationsCacheInitializationBenchmarks`: replaced some obsolete usages
I tried to create this PR twice before (many months ago), while the changes to the unit definitions were still not merged- and the web interface was giving me an error when trying to browse the files changed.. Something like "Too many files to display" 😄 |
Ok, I'm not going to get through a review of this many files anytime soon. On the surface though, it seems like this could be split up into chunks. I know it's tedious and extra work, but it will be way faster to review. Do you see any chunks of changes to easily split off into separate PRs? |
Sofia (GMT+3), but time zones are not relevant to my sleep schedule - so basically any time you want.
Yes, I do have some ideas:
Hopefully by the time we get to 5) you'd be up to speed (and fed up with PRs) and we can turn back to reviewing / working on the rest of it as a whole 😄 |
Ok, sounds good. Just send PRs my way and I'll try to get to them. I have a little bit of extra time this weekend. |
…ions into their own folder
- UnitAbbreviationsCache: removing the obsolete overload - adding a few more tests
This PR is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days. |
This PR was automatically closed due to inactivity. |
…ramework (net8.0) with 9.0 on the QuantityInfoBenchmarks
…OfType / IQuantityInstanceInfo)
@angularsen I've synced the changes from upstream, however instead of bringing down the diffs, this has added another ~200 file changes (the new I've updated the task list (completing the
... but that can only start after I've had a confirmation, that we're moving towards replacing the |
@lipchev I really like the concept of the decimal fractions, it solves equality issues and precision, and if I recall correctly it remains fairly backwards compatible. I want to say let's go, but if you could please help me out and give a short summary for a couple of things that would help a lot as I don't recall and I'm pressed for time.
|
I spent a full Sunday back in February creating a comparison chart, but before I could finish up setting up all the sheets, I accidentally executed the build script which erased all of my benchmark results. 😠 I'll attach the file as it is, but the gist is that compared to The conversions are about the same, or faster (with units other than the For example the value The initialization is slower but no more than 10x, judging from these benchmarks but that can be reduced by loading a select subset of quantities (I imagine loading just 10 PS The PS2 Aside for the static benchmarks, I've also observed a visible improvement (in the performance logs) after migrating my code-base, but that's probably due to the improvements to the string handling stuff around the
Compared to the
While inconvenient, this is the correct way to do it (implicit conversions from, and explicit conversion to any primitive number). By the way, given that in |
Thank you. I'll post some thoughts before reviewing the actual PR code changes. Excel sheetNice overview, here is my read of it:
NormalizingRegarding not normalizing and running into allocations when crossing InitializationCan't we just lazy initialize the quantities actually used? Not sure how much work that would be. Breaking changesGood overview. What are the disadvantages of supporting implicit cast to double and decimal though? It seems really convenient to me. |
Initial review by Claude. PR Review: QuantityValue as Fractional Number 🐲PR: #1544 OverviewThis is a massive, fundamental rewrite of UnitsNet's core architecture:
Key Changes1. QuantityValue → Fractional Representation
// Internal representation
private readonly BigInteger _numerator;
private readonly BigInteger? _denominator;
// Example: Exact fractions instead of floating-point
new QuantityValue(5000, 127) // Exact: 5000/127 for inches Benefits:
2. API Breaking Changes
3. New Features
4. Architecture Improvements
Migration ImpactBreaking Changes ExamplesEvery consumer will need code changes: // ❌ Breaking - no longer compiles
double meters = length.Value;
double ratio = length1 / length2;
var newLength = new Length(5.0, LengthUnit.Meter);
void ProcessDistance(double meters) { }
ProcessDistance(length.Meters);
// ✅ Migration needed
QuantityValue meters = length.Value;
double metersAsDouble = length.Value.ToDouble();
QuantityValue ratio = length1 / length2;
var newLength = new Length(new QuantityValue(5), LengthUnit.Meter);
void ProcessDistance(double meters) { }
ProcessDistance(length.Meters.ToDouble()); Interface Changes// ❌ Removed from IQuantity
double As(Enum unit);
double As(UnitKey unitKey);
IQuantity ToUnit(Enum unit);
IQuantity<T>.ToUnit(UnitSystem unitSystem);
// ✅ Now available as extension methods
QuantityExtensions.As(this IQuantity quantity, UnitKey unitKey);
QuantityExtensions.ToUnit(this IQuantity quantity, UnitKey unitKey);
QuantityExtensions.ToUnit<T>(this IQuantity<T> quantity, UnitSystem unitSystem); Serialization CompatibilityGood News 🎉JSON serialization is mostly backwards compatible with the right configuration: // Old format (v5): {"Value":10.0,"Unit":"m"}
// New format (v6): {"Value":10,"Unit":"m"}
// Deserialize old data with RoundedDouble option:
var converter = new AbbreviatedUnitsConverter(
new QuantityValueFormatOptions(
SerializationFormat: QuantityValueSerializationFormat.DoublePrecision,
DeserializationFormat: QuantityValueDeserializationFormat.RoundedDouble
));
Length length = JsonConvert.DeserializeObject<Length>(oldJson, converter);
// ✅ Can read old JSON! Default Behavior Change
|
On Rider, this was needed to actually run the samples with 'Official' configuration. Only Debug/Release were shown.
This reverts commit 896f765.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Posting review so far, I got down through SystemTextJson.
"UnitsNet.NumberExtensions.CS14.Tests\UnitsNet.NumberExtensions.CS14.Tests.csproj", | ||
"UnitsNet.Serialization.JsonNet.Tests\UnitsNet.Serialization.JsonNet.Tests.csproj" | ||
"UnitsNet.Serialization.JsonNet.Tests\UnitsNet.Serialization.JsonNet.Tests.csproj", | ||
"UnitsNet.Serialization.SystemTextJson.Tests\UnitsNet.Serialization.SystemTextJson.csproj" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SystemTextJson is candidate for separate PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Never mind, let's just review and merge everything
/// </summary> | ||
[DataMember(Name = ""Value"", Order = 1)] | ||
private readonly double _value; | ||
[DataMember(Name = ""Value"", Order = 1, EmitDefaultValue = false)] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean Lenght.Zero would exclude the value in the JSON/XML?
I think I prefer both the value and unit to always be included. Optional values can make sense to omit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My philosophy regarding data DataContractSerializer
and the DataContractJsonSerializer
is that these are only used/useful in machine-to-machine communications (such as WCF or the GRPC gateway thing, which AFAIK builds up the protos based off the data contracts)- as such I've dropped all constraints regarding the human-readability of the output and prioritized the reduction of the payload size.
The difference isn't huge, but given how bloated the xml output is, and how rarely one (i.e. a person) actually reads it, I figured it wouldn't hurt to shave off a few bytes when possible.
} | ||
|
||
Writer.WL($"<{relation.LeftQuantity.Name}, {relation.RightQuantity.Name}, {relation.ResultQuantity.Name}>,"); | ||
Writer.WL($"<{relation.LeftQuantity.Name}, {relation.RightQuantity.Name}, {relation.ResultQuantity.Name.Replace("double", "QuantityValue")}>,"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Out of scope, but double
here seems like it could be renamed to avoid this string replace.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we should probably replace it in the UnitRelations.json
(marking it as a potential breaking change).
} | ||
else | ||
{ | ||
// note: omitting the extra parameter (where possible) saves us 36 KB |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is kinda hard to read, some example code in comments would be helpful
public static {_quantity.Name} operator +({_quantity.Name} left, {_quantity.Name} right) | ||
{{ | ||
// Logarithmic addition | ||
// Formula: {x} * log10(10^(x/{x}) + 10^(y/{x})) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can remove these comments, and put them in the extension method if not already documented there. Inline the leftUnit
also so this becomes more or less a one-liner.
Same for below method.
/// <summary> | ||
/// Construct a converter using the default list of quantities (case insensitive) and unit abbreviation provider | ||
/// Initializes a new instance of the <see cref="AbbreviatedUnitsConverter" /> class using the default | ||
/// case-insensitive comparer and the specified <see cref="QuantityValueFormatOptions" />. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The default ctor xmldoc could be more clear about what the defaults are.
Same with QuantityValueFormatOptions
, it could also more plainly state the defaults in its xmldoc.
|
||
namespace UnitsNet.Serialization.JsonNet; | ||
|
||
internal static class QuantityValueExtensions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit, but should this not be renamed to something like XxxJsonWriterExtensions?
It also has Reader stuff, so maybe suffix XxxJsonExtensions or split into reader/writer files.
<!-- NuGet properties --> | ||
<PropertyGroup> | ||
<PackageId>UnitsNet.Serialization.JsonNet</PackageId> | ||
<Version>6.0.0-pre007</Version> | ||
<Version>6.0.0-pre019</Version> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd use a real high number to avoid having to bump this, but this will soon be a thing of the past.
We must remember to reset this back when merging.
[RequiresUnreferencedCode( | ||
"If some of the generic arguments are annotated (either with DynamicallyAccessedMembersAttribute, or generic constraints), trimming can't validate that the requirements of those annotations are met.")] | ||
#endif | ||
public class InterfaceQuantityWithUnitTypeConverter : JsonConverter<IQuantity> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm only skimming all the serialization stuff. At first glance I don't see why we have both InterfaceQuantityWithUnitTypeConverter
and InterfaceQuantityConverterBase
.
It seems InterfaceQuantityWithUnitTypeConverter
is unused/untested?
<PackageLicenseExpression>MIT-0</PackageLicenseExpression> | ||
<PackageRequireLicenseAcceptance>false</PackageRequireLicenseAcceptance> | ||
<PackageTags>unit units measurement json System.Text.Json serialize deserialize serialization deserialization</PackageTags> | ||
<PackageReleaseNotes>Upgrade JSON.NET to 12.0.3. Support arrays.</PackageReleaseNotes> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Package release notes can maybe be set to something like initial release.
…rseInvariant Specify invariant culture for Fraction and `int` parsing.
This is typically not a problem with integers, but let's be explicit. Integer parsing is done for quantity string formatting to control number formats, such as `a2` for the 2nd unit abbreviation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another batch of comments.
I got down to: ConversionExpressionTests.cs
namespace CodeGen.Helpers.ExpressionAnalyzer.Functions.Math; | ||
|
||
/// <summary> | ||
/// We should try to push these extensions to the original library (working on the PRs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update xmldoc
// { | ||
// return Value.ToString("G", CultureInfo.InvariantCulture).Length < 16; | ||
// } | ||
// } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove commmented code or add clear TODO
} | ||
|
||
[Fact] | ||
public void Deserialize_WithoutPositiveNumeratorAndZeroDenominator_ReturnsPositiveInfinity() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very thorough set of tests, I like it
.ToList(); | ||
var customMeterDefinition = new UnitDefinition<LengthUnit>(LengthUnit.Meter, "UpdatedMeter", "UpdatedMeters", BaseUnits.Undefined); | ||
|
||
var result = unitDefinitions.Configure(LengthUnit.Meter, _ => customMeterDefinition).ToList(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does look a bit weird to have extension method Configure
on a collection of UnitDefinition
items.
I think it would be better to either create a wrapper type to hold the collection and offer these methods, or convert the extension methods to a named static helper class.
{ | ||
UnitDefinition<LengthUnit> oldDefinition = Length.LengthInfo.GetDefaultMappings().First(); | ||
|
||
UnitDefinition<LengthUnit> newDefinition = oldDefinition.WithConversionFromBase(20); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I pushed a rename to WithConversionFactorFromBase
[InlineData(-0.1, "P0", 11, 2)] // n- % | ||
[InlineData(-0.1, "P0", 11, 3)] // n- % | ||
[InlineData(-0.1, "P0", 11, 4)] // n- % | ||
public void TryFormat_NegativeNumber_WithInvalidSpanLength_ReturnsFalse(decimal decimalValue, string format, int pattern, int testLength) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean, I love tests, but holy shit you've produced a lot of tests 🤯
We'll keep them all of course, but yes I do think it's maybe a bit excessive at times 😅
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I had most of these implemented in the Fractions
library. As I decided to move them in here, I also implemented the ReadOnlySpan
overloads- which made the whole thing explode.
// | ||
// Assert.Equal(0.0508, inSI.Value); | ||
// Assert.Equal(LengthUnit.Meter, inSI.Unit); | ||
// } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Either remove commented tests or add clear TODO on how to add them back
public static HowMuch From(QuantityValue value, HowMuchUnit unit) | ||
{ | ||
return new HowMuch(value, unit); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
HowMuch is gradually getting cleaner 👍
{ | ||
var expectedUnit = MassUnit.Milligram; | ||
var unitInt = (int)expectedUnit; | ||
var json = $"{{\"Value\":1.2,\"Unit\":{unitInt}}}"; | ||
var json = $$$"""{"Value":{"N":{"_bits":null,"_sign":12},"D":{"_bits":null,"_sign":10}},"Unit":{{{unitInt}}}}"""; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we ensuring backwards compatibility for deserializing JSON like {"Value": 1.2, "Unit": 5}
elsewhere?
Also, the _bits
stuff looks very internal. Is it feasible to have this match the output of SystemTextJson?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The DataContractJsonSerializer
is a lost cause- not even microsoft bothers with it any more.
I did try to customize it (trying to preserve the original output) - but there was a bug (I've left a link to the github issues in a comment somewhere) which prevented me.
var expectedXml = $"<Mass {Namespace} {XmlSchema}><Value>1.2</Value><Unit>Milligram</Unit></Mass>"; | ||
var numeratorFormat = $"<N xmlns:a={NumericsNamespace}><a:_bits i:nil=\"true\" xmlns:b={ArraysNamespace}/><a:_sign>12</a:_sign></N>"; | ||
var denominatorFormat = $"<D xmlns:a={NumericsNamespace}><a:_bits i:nil=\"true\" xmlns:b={ArraysNamespace}/><a:_sign>10</a:_sign></D>"; | ||
var expectedXml = $"<Mass {Namespace} {XmlSchema}><Value>{numeratorFormat}{denominatorFormat}</Value><Unit>Milligram</Unit></Mass>"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similar for xml, what is the plan for backwards compat and migration steps here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I believe there is an example which uses a data-contract-surrogate.
PS I originally tried implementing the IXmlSerializable
interface, but then I encountered another bug in the proxy-generation stack (not sure if I left the link to that issue or not).
I saw quite a bit of commented code here and there, so I asked Claude to identify all the commented out parts. Something to consider going over. PR #1544 - Commented Code ReviewPR Title: QuantityValue implemented as a fractional number 🐲 SummaryThis review identifies commented-out code in PR #1544 that should either be removed or have TODO comments explaining why it's commented out and what the plan is for uncommenting later. 🔴 High Priority - Remove Immediately1. QuantitiesSelector.cs - Duplicate Commented Class (112 lines!)File: // /// <summary>
// /// Provides functionality to select and configure quantities for use within the UnitsNet library.
// /// </summary>
// ... [full duplicate class implementation - 112 lines]
// } Recommendation: DELETE - This appears to be an old version of the same class that should have been removed. Git history preserves this for reference if needed. 2. ExpressionEvaluator.cs - "No Longer Necessary" CodeFile: // these are no longer necessary
// var expressionEvaluator = new ExpressionEvaluator(parameter, constantExpressions,
// new SqrtFunctionEvaluator(),
// new PowFunctionEvaluator(),
// new SinFunctionEvaluator(),
// new AsinFunctionEvaluator());
var expressionEvaluator = new ExpressionEvaluator(parameter, constantExpressions); Recommendation: DELETE - Explicitly marked as "no longer necessary". Git history preserves the old constructor call pattern. 🟡 Medium Priority - Remove or Add TODO3. QuantitiesSelector.cs - Unused PropertyFile: // internal Lazy<IEnumerable<QuantityInfo>> QuantitiesSelected { get; } Recommendation: DELETE - No explanation provided; appears to be leftover from refactoring. 4. QuantitiesSelector.cs - ToList() AlternativeFile: return enumeration;
// return enumeration.ToList(); Recommendation: Either:
5. DynamicQuantityConverter.cs - Old Implementation (2 instances)File: // TryGetUnitInfo(conversionKey.FromUnitKey with { UnitValue = conversionKey.ToUnitValue }, out UnitInfo? toUnitInfo)) Line 347: // return targetQuantityInfo.Create(conversionFunction.Convert(value), conversionFunction.TargetUnit);
return targetQuantityInfo.From(conversionFunction.Convert(value), conversionFunction.TargetUnit.ToUnit<TTargetUnit>()); Recommendation: Either:
6. FrozenQuantityConverter.cs - Old ImplementationFile: // TryGetUnitInfo(conversionKey.FromUnitKey with { UnitValue = conversionKey.ToUnitValue }, out UnitInfo? toUnitInfo)) Recommendation: Same as DynamicQuantityConverter - either DELETE or add TODO with explanation. 7. FractionExtensions.cs - Old Double-Based ApproachFile: // return FromDoubleRounded(System.Math.Pow(x.ToDouble(), power.ToDouble()));
return PowRational(x, power); Recommendation: Either:
✅ Keep (Already Have Adequate Explanation)ConversionExpression.cs - Inline DocumentationFile: These comments show simplified lambda forms that enhance readability. Keep as-is. // scaleFunction = value => value;
// scaleFunction = value => value * coefficient; UnitsNetSetup.cs - Future Work TODOsFile: // TODO see about allowing eager loading
// private AbbreviationsCachingMode AbbreviationsCaching { get; set; } = AbbreviationsCachingMode.Lazy;
// TODO see about caching the regex associated with the UnitParser
// private UnitsCachingMode UnitParserCaching { get; set; } = UnitsCachingMode.Lazy; Status: ✅ Properly documented future work. Keep as-is. UnitTestBaseClassGenerator.cs - Documented LimitationFile: // note: it's currently not possible to test this due to the rounding error from (quantity - otherQuantity)
// Assert.True(quantity.Equals(otherQuantity, maxTolerance)); Status: ✅ Well-documented limitation. Keep as-is. MainWindowVM.cs - Sample Code Alternative ApproachFile: // note: starting from v6 it is possible to store (and invoke here) a conversion expression
// ConvertValueDelegate _convertValueToUnit = UnitsNet.UnitConverter.Default.GetConversionFunction(...);
// ToValue = _convertValueToUnit(FromValue); Status: ✅ Sample/demo code showing users an alternative v6 feature. Keep as-is. ExpressionEvaluationHelpers.cs - Implicit Constructor NoteFile: // return $"new ConversionExpression({coefficientTermFormat})";
return coefficientTermFormat; // using the implicit constructor from QuantityValue Status: ✅ Has explanation, though could be slightly improved: Optional improvement: // Old explicit approach: return $"new ConversionExpression({coefficientTermFormat})";
return coefficientTermFormat; // using the implicit constructor from QuantityValue Summary Statistics
Recommended Actions
This will remove ~125+ lines of dead code and improve code maintainability. |
QuantityValue
implemented as a fractional numberIQuantity
interfaces optimized (some methods refactored as extensions)UnitInfo
: introduced two new properties:ConversionFromBase
andConversionToBase
which are used instead of theswitch(Unit)
conversionUnitsNetSetup
: introduced helper methods for adding external quantities, or re-configuring one or more of the existing onesUntAbbreviationsCache
: introduced additional factory methods (using a configuration delegate)UnitParser
: introduced additional factory methods (using a configuration delegate)UnitConverter
: re-implemented (multiple versions)Inverse
relationship mapping implemented as a type of implicit conversion