Proposal: Support Hex Display and Input in NumberBox

[robloo]

Proposal: Support Hex Display and Input in NumberBox

Summary

Support inputting "FF", "FF2813" or "0xFF" in the NumberBox and preserve the hex values when evaluated and re-displayed.

Rationale

There are lots of use cases for this -- it's needed for the Hex input box of the ColorPicker for one.

Scope

Capability	Priority
Support input of hex numerical values	Must
Support display of hex numerical values	Must
Support configuration between hex and decimal display/input	Must

[StephenLPeters]

@SavoySchuler FYI, this seems cool to me.

[Felix-Dev]

@StephenLPeters @teaP @SavoySchuler Is this something the community can contribute to the NumberBox as the team is busy with WinUI 3? I have started to think about how we could approach this and created an API proposal including an API demo below. Let me know if this is something I/the community can further pursue here with the goal of adding this to the NumberBox as a community contribution before WinUI 3 is done and will free up some team resources to work on this proposal.

@robloo If you can, please play around with the proposed API, read through the API proposal and let me know if you think anything else is missing here which we should cover for hexadecimal support for the NumberBox.

API Overview

My current thinking is to create a new HexadecimalFormatter API to enable hexadecimal support, similar to the existing DecimalFormatter API which is used by the NumberBox right now to display decimal numbers. This new API will enable us to provide great customization options to customers - for example giving them the option to configure the supported input prefixes, the desired output prefix and how many digits should be displayed as a minimum. The goal with this proposal is to position the NumberBox control to be well-suited as a versatile Hex input/output control.

Now, before I start a detailed post about the API and the current open design questions I am seeing, I want to let you know that you can check out the proposed API here "live" as I've uploaded my current work on the HexadecimalFormatter in a branch in my WinUI fork. So, instead of just having to read through pontentially lots of text, questions and descriptions which can turn out be become quite tedious, you can check out the proposed APIs and play around with them while reading through this proposal! You will find more information on the provided API demo in the "API Demo" section at the bottom of this post. (And if you have trouble getting a full WinUI build to complete successfully, I've added a note about using the "inner loop" solution instead, which should considerably improve the build experience for you.)

With that out of the way, let's start.

API Examples

Assign a HexadecimalFormatter instance to the NumberBox

To enable hexadecimal input/display support, we can simply create an instance of the HexadecimalFormatter class and assign it to the NumberFormatter API on the NumberBox:

XAML

<!-- Defined in a resource dictionary -->
<muxc:HexadecimalFormatter x:Key="HexFormatter"/>

<muxc:NumberBox 
     x:Name="TestNumberBox" 
     Value="256"
     NumberFormatter="{StaticResource HexFormatter}"/>

C#

this.TestNumberBox.NumberFormatter = new Microsoft.UI.Xaml.Controls.HexadecimalFormatter();

Note: For the following examples, I will only show XAML markup code. You can of course always use code-behind as well.

Customize the display output

You can customize the display output of the HexadecimalFormatter in several ways:

Customize the output prefix

You can use the OutputPrefix API of the HexadecimalFormatter to set your desired display output prefix:

<muxc:HexadecimalFormatter 
     x:Key="HexFormatter"
     OutputPrefix="0x"/>

Customize the minimum number of digits to display

To configure the minimum number of digits to be displayed, the MinDigits API can be used:

<muxc:HexadecimalFormatter 
     x:Key="HexFormatter"
     MinDigits="6"/>

Visually group the digits of the output.

Apps like Windows Calculator, for example, take an input like "89ABCDEF" and display it as a grouped output like "89AB CDEF". The IsGrouped API will enable to developers to opt into this behavior as well.

<muxc:HexadecimalFormatter 
     x:Key="HexFormatter"
     IsGrouped="True"/>

Remark

Support for this API required changes to the existing NumberBoxParser (which is used to parse entered expressions and sends the actual numbers to parse to the specified NumberBox.NumberFormatter). See the "Remarks" section below for more details.

Customize the display input

Customize the supported input prefix

By default, no input prefixes like "#" or "0x" are supported. However, you can use the InputPrefixes API of the HexadecimalFormatter to set a list of supported input prefixes:

<muxc:HexadecimalFormatter 
     x:Key="HexFormatter">
    <muxc:HexadecimalFormatter.InputPrefixes>
        <x:String>#</x:String>
        <x:String>0x</x:String>
    </muxc:HexadecimalFormatter.InputPrefixes>
</muxc:HexadecimalFormatter>

Remark

Due to the nature of how the NumberBox string parsing currently works, if you specify a custom output prefix and want to use arithmetic expressions (NumberBox.AcceptsExpression = true), you will also have to add the specified output prefix to the InputPrefixes collection. Otherwise, the expression won't be successfully processed. The input parser will receive the entered expression string, including any existing specified custom out prefix. However, if the InputPrefixes collection does not contain the output prefix, the output prefix will be treated as an invalid input, thus ensuring the expression parsing fails.

Switch between hexadecimal display/input and decimal display/input.

If you want to switch between the hexadecimal and decimal numeral systems, all you have to do is assign a new HexadecimalFormatter or DecimalFormatter to the NumberBox.NumberFormatter API:

where we handle the "Numeral System Mode" ComboBox selection in code-behind like this:

private void NumeralSystemComboBox_SelectionChanged(object sender, SelectionChangedEventArgs e)
{
    if (this.NumeralSystemComboBox.SelectedIndex == 0)
    {
        // Decimal numeral system

        var formatter = new DecimalFormatter() {
            IntegerDigits = 1,
            FractionDigits = 0
        };

        this.TestNumberBox.NumberFormatter = formatter;
    }
    else
    {
        // Hexadecimal numeral system

        var formatter = new HexadecimalFormatter() {
            OutputPrefix = "#",
        };

        this.TestNumberBox.NumberFormatter = formatter;
    }
}

Conversion of floating-point numbers to hexadecimal display output

The HexadecimalFormatter follows the behavior of WinForm's NumericUpDown control which, if given a non-integer number of the form X.YZ, throws away the fractional part and keeps the integer part. In other words, a value like 10.75 will be displayed as A in the hexadecimal output mode. The HexadecimalFormatter currently works the same way:

API Notes

Notable Properties

Name	Description
InputPrefixes	Gets the set of string prefixes an input can pick a valid string prefix to use from.
OutputPrefix	Gets or sets the string prefix used for the display output. The default is the empty string (""). Note: If you have specified a custom output prefix, please make sure to also include it in the InputPrefixes collection. Otherwise, if you want to use the NumberBox with arithmetic expressions, the entered expressions will not be processed successfully if they contain the output prefix.
MinDigits	Gets or sets the minimum number of digits to be displayed. The default is 1.
IsGrouped	Gets or sets whether the hexadecimal number should be grouped in digits of four. The default is false.

API Details

namespace MU_XC_NAMESPACE
{

[WUXC_VERSION_PREVIEW]
[webhosthidden]
[default_interface]
runtimeclass HexadecimalFormatter : Windows.UI.Xaml.DependencyObject, Windows.Globalization.NumberFormatting.INumberFormatter2, Windows.Globalization.NumberFormatting.INumberParser
{
    HexadecimalFormatter();

    String OutputPrefix{ get; set; };
    Windows.Foundation.Collections.IVector<String> InputPrefixes{ get; };

    [MUX_DEFAULT_VALUE("1")]
    Int32 MinDigits{ get; set; };

    [MUX_DEFAULT_VALUE("false")]
    Boolean IsGrouped{ get; set; };

    static Windows.UI.Xaml.DependencyProperty OutputPrefixProperty{ get; };
    static Windows.UI.Xaml.DependencyProperty InputPrefixesProperty{ get; };

    static Windows.UI.Xaml.DependencyProperty MinDigitsProperty{ get; };

    static Windows.UI.Xaml.DependencyProperty IsGroupedProperty{ get; };
};

}

Remarks

The API proposal outlined above specified a HexadecimalFormatter.IsGrouped API. Support for this API requires changes to the NumberBoxParser and how it handles expressions. Currently, the NumberBox expression parser does not treat digits separated by spaces as a single number. An example: In a given expression input like "1234 5678 + 9ABC DEF0" the grouped value "1234 5678" is currently treated as two distinct values - and since there is no operator defined in the input string to take these two values as operands - the parsing fails.

To enable support for the IsGrouped API (which could create an expression input with numbers containing spaces) the idea here is to change the used regular expression to also include whitespace between digits of the value passed to numberParser.ParseDouble(). This will delegate the task of handling an input number with spaces inside to the specified NumberBox.NumberFormatter. This would also match the code flow when the NumberBox does not allow expressions to be entered:

microsoft-ui-xaml/dev/NumberBox/NumberBox.cpp

Line 422 in 5f101c7

const winrt::IReference<double> value = AcceptsExpression()

Even if the NumberBox does not support expressions, a user might paste a grouped hex value into the NumberBox (like 0123 4567) and it would then be up to the specified hex parser to determine if an input with spaces put between digits is valid. (That determination would happen in the different INumberParser.Parse*() methods.)

Such a change to the NumberBoxParser will still keep the current default behavior for decimal numbers intact (no spaces between digits of a number allow) as the DecimalFormatter used does not treat such input as valid.

This change has been implemented with commit Felix-Dev@f2db82e in my local fork.

Additional Notes

The HexadecimalFormatter API shown above does not yet implement the INumberFormatter API, as done by the DecimalFormatter, for example. The reason is that my goal for now was to create an API which works primarily in the NumberBox use case in order to gather feedback on the general direction of this API design. If this API is something the team and the community like @robloo think is the right approach here, we can always implement the INumberFormatter interface as well if deemed useful.

The HexadecimalFormatter API also does not implement the INumberFormatterOptions compared to the DecimalFormatter. In this case, I think most of the APIs defined by that interface are not useful for the HexadecimalFormatter, such as FractionDigits, GeographicRegion, IsDecimalPointAlwaysDisplayed, NumeralSystem, etc.... As such, I believe implementing this interface would not add anything useful to the HexadecimalFormatter API.

Open Questions

• The most important question first: Is this an API approach which makes sense to the community and the team? Should further time be invested into it?
• Which additional interfaces - like INumberFormatter - should HexadecimalFormatter implement which are currently implemented by the DecimalFormatter + related formatters and why?
• Which additional APIs should the HexadecimalFormatter provide to satisfy common hex input/output configurations?
• Currently, passing null to the NumberBox.NumberFormatter API will result in an exception: https://github.com/microsoft/microsoft-ui-xaml/blob/master/dev/NumberBox/NumberBox.cpp#L294 Would it make sense to change this behavior so that specifying null here would cause the NumberBox to fall-back to its default decimal number formatter? This would allow developers to specify a different NumberFormatter without having to re-create the default number formatter if they want to switch back to it (for example by providing a UI toggle to change from hex to dec input/output mode of the NumberBox back-and-forth). What do you think about this? This sounds like this would be a welcomed improvement to me by removing work developers might have to additionally commit to as a side-effect of them assigning a non-default NumberFormatter.
• In the future, we might build formatters for different numeral system like the binary system or the octal system to be used by NumberBox. Their public APIs will presumably be quite similar to the proposed HexadecimalFormatter API here. For example, an IsGrouped API will also make sense for the OctalFormatter. The APIs OutputPrefix and InputPrefixes will probably make sense for both a future OctalFormatter and a BinaryFormatter.

As such, we could create a new interface called INumberFormatterOptions2 which exposes these APIs and will be implemented by future binary, hex and octal number formatters:

public interface INumberFormatterOptions2
{
    string OutputPrefix { get; set; }

    IList<string> InputPrefixes { get; }

    int MinDigits { get; set; }

    bool IsGrouped { get; set; }
}

The NumberBoxParser can then simply query these above APis by checking if the specified number formatter implements the INumberFormatterOptions2 interface.

Your thoughts on that?

API Demo

As mentioned in the beginning of this post, I've uploaded my current work on the HexadecimalFormatter in a branch in my WinUI fork here. If you compile the MUXControlsTestApp, you can open the NumberBox test page and start playing around:

Notice the ComboBox on the left titled "Numeral System Mode". Use it to switch between hex and decimal mode for the NumberBox with the value "256". In code-behind, you can checkout the NumeralSystemComboBox_SelectionChanged method to set more configuration options on the HexadecimalFormatter object (or you can use XAML markup).

If you are interested in the current implementation of the HexadecimalFormatter, you will find it here.

If you have trouple getting a successful full build of WinUI, you can also try out the building the InnerLoop solution. Simply open "MUXControlsInnerLoop.sln" and add <FeatureNumberBoxEnabled>true</FeatureNumberBoxEnabled> to its InnerLoopArea.props file, so that it will look like this:

<?xml version="1.0" encoding="utf-8"?>
<!-- Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT License. See LICENSE in the project root for license information. -->
<Project ToolsVersion="14.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <!-- Feature to include for inner loop build -->
  <PropertyGroup Condition="$(SolutionName) == 'MUXControlsInnerLoop'">
    <FeatureNumberBoxEnabled>true</FeatureNumberBoxEnabled>
  </PropertyGroup>
</Project>

Now build MUXControlsTestApp project and you should notice a considerably improved build process experience. Once the app laoded, you should be greeted with a single "NumberBox" entry in the app. (If you are seeing a blank window, you will have to close and reopen the solution. This should solve that issue.). Once you see the single "NumberBox" entry you can start testing the new NumberBox features :)

Specific Implementation Remarks

No longer relevant thanks to @sylveon's merged PR #2955 :).

[robloo]

@Felix-Dev Lots of good stuff here to dig through! Give me a bit to provide feedback -- if any is really required. You have lots of good ideas already.

I do remember in the original NumberBox spec discussion I was excited about using custom implementations (in C#) of the INumberFormatter2 interface to customize formatting. However, this was quickly shot down by @SavoySchuler for some reason it wasn't supported -- I don't know if there are issues in interop layers or the problem was with parsing, etc. Maybe it was just a miscommunication. If there are no technical limitations though its a great idea to add a new HexadecimalFormatter -- but it's going to have to go into WinRT I'm afraid (which means the impossibly slow and unresponsive Windows team).

Also, this whole discussion is related to the generalized topic of supporting custom formatting and parsing: #2684.

[Felix-Dev]

@robloo Thanks already for taking an early look.

I do remember in the original NumberBox spec discussion I was excited about using custom implementations (in C#) of the INumberFormatter2 interface to customize formatting. However, this was quickly shot down by @SavoySchuler for some reason it wasn't supported -- I don't know if there are issues in interop layers or the problem was with parsing, etc. Maybe it was just a miscommunication.

Hmm, prior to working on this API proposal I did read the NumberBox initial discussion thread and spec again. I also checked the ppec PR discussion just now. I'm afraid to say I didn't find the message you are remembering here. I did find one comment by @SavoySchuler here (under "Localization") where he said that the "the intention of the current implementation" of the NumberBox is the support of " any INumberFormatter/INumberFormatter2". Then again, these threads are massive so perhaps I just missed the comment you are remembering here.

That said, I'm not aware of any "technical limitations" regarding the approach I chose above. The proposed HexadecimalFormatter is implemented in C++/WinRT and works great as you will hopefully notice when playing around with it (or just checking out the posted images/gifs). You could also, in your C# project, just write your own HexadecimalFormatter all in C# and pass it to the NumberBox as well - there shouldn't be any issues.

What I like about the current API proposal is that we are providing customers with a fully implemented hexadecimal formatter, yet still aim to give them the relevant customization options they expect here to easily customize the supported inputs (like input prefixes) and the final display output (output prefix, min digits to show (useful for displaying colors, for example) and whether the output should be grouped for better visibility). Ideally, we would implement this HexadecimalFormatter and no customer would ever have to write their own hex formatter/parser for the NumberBox - except for if they have some really uncommon requirements.

If there are no technical limitations though its a great idea to add a new HexadecimalFormatter -- but it's going to have to go into WinRT I'm afraid (which means the impossibly slow and unresponsive Windows team).

Yes, this requires some thought here. Currently, formatters like the DecimalFormatter are in the Windows.Globalization.NumberFormatting namespace. It seems as if the proposed HexadecimalFormatter should also go into that namespace then. This would present a significant barrier for a community contribution though as that part of the SDK is not open-source. It will also mean all the other known disadvantages of shipping it in the OS (no default downlevel support, etc...). Perhaps Project Reunion could be used here to distribute a community provided HexadecimalFormatter implementation.

Perhaps the fastest way of shipping an API like the proposed HexadecimalFormatter to customers would be to implement and distribute it via the Windows Community Toolkit. Unfortunately, this would not be a solution for non C#/VB projects, so I would love to see a solution here where we can reach all WinUI developers. Whether that means having to wait until Project Reunion is ready for a community contribution like this or if there's a place in WinUI for this API... I'm sure the WinUI/Project Reunion team will let us know here how to best proceed with this API proposal - if it's accepted.

[robloo]

@Felix-Dev I checked the original thread again. My memory wasn't entirely clear and there isn't a technical limitation in supporting INumberFormatter2; however, the reason I'm remembering it isn't supported is because the number formatter can't handle parsing. Therefore, as soon as you use it to add something like a degree symbol, extra spaces or '0x' it breaks #483 (comment). So INumberFormatter2 becomes almost useless.

Parsing/Formatting need to be coupled together as discussed in #2684

[Felix-Dev]

@robloo It's true that currently, spaces between digits of a number is not supported when parsing expressions. However, this is a limitation of the current NumberBoxParser (which consumes the specified NumberBox.NumberFormatter to parse strings to doubles) which I will fix in a commit in my experimental branch soon. This will result in the NumberBoxParser sending an input like "4567 89AB" to the specified NumberFormatter which implements INumberParser. In other words, with this change, it will now be up to the developer provided INumberParser if spaces between digits of a value are allowed or not. This will enable me to create a working HexFormatter.IsGrouped API.

An inportant point to make clear here is that the proposed HexadecimalFormatter implements both the INumberFormatter2 interface and the INumberParser interface (and thus gives you the desired parsing/formatting coupling) to implement custom prefixes for both input and output as shown in the images/gifs above.

The case of custom prefixes like "0x" or "#": The NumberBox will currently send an input like "0x12345678" directly to the specified NumberFormatter to parse - thus again leaving it up to the developer if those are valid inputs. For an example, see the current implementation of the HexadecimalFormatter here: https://github.com/Felix-Dev/microsoft-ui-xaml/blob/4e8079500a38b7c4912e4291bd5cdaea4379f259/dev/NumberBox/HexadecimalFormatter.cpp#L136

In the code above, the HexadecimalFormatter first checks if the entered text contains one of the supported HexadecimalFormatter.InputPrefixes. If "0x" or #" have been added as valid input prefixes then all is good: They will be stripped from the input and allow further parsing to proceed. If those have not been added to the list of valid input prefixes, yet are still used as a number prefix, then parsing will fail.

And just as how the INumberParser implementation part of the HexadecimalFormatter can deal with custom prefixes/suffixes like "0x" or "#" as input, so can the INumberFormatter2 part of it take a numerical value and add custom prefixes/suffixes to its output string. This is exactly how I am currently implementing the HexadecimalFormatter.OutputPrefix API.

And just as how I am adding support for custom prefixes here, so can a developer create another NumberFormatter (implementing both INumberFormatter2 and INumberParser) to handle the degree symbol accordingly - or any other prefix/suffix specified. That will be entirely up to the implementation provided by the developer.

Reading through the post you've linked, I have a feeling we are looking at a missunderstanding here because developers can provide input with custom prefix/suffix support just fine right now - when implementing their own INumberFormatter2 and INumberParser. There is one limitation currently in-built into the NumberBoxParser which concerns spaces between digits but that is a limitation I will soon address in my linked branch. Other than that, I'm not aware right now of any other limitations preventing developers from providing support for custom value prefixes/suffixes in general.