Skip to content

Add parser tests from nim-regex #62093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Dec 3, 2021
Merged

Add parser tests from nim-regex #62093

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
8e0cdeb
new parser tests
danmoseley Nov 27, 2021
87ca318
baseline
danmoseley Nov 27, 2021
9baa06c
Nim tests
danmoseley Nov 27, 2021
424ae3b
typos
danmoseley Nov 27, 2021
e9844c8
positive cases
danmoseley Nov 27, 2021
5a9dc84
new parser tests
danmoseley Nov 27, 2021
8e143ac
change to \u
danmoseley Nov 27, 2021
64e3d85
Address feedback
danmoseley Nov 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 24 additions & 2 deletions src/libraries/System.Text.RegularExpressions/tests/RegexParserTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -714,18 +714,40 @@ private static void Parse(string pattern, RegexOptions options, RegexParseError?
if (error != null)
{
Assert.InRange(offset, 0, int.MaxValue);
Throws(error.Value, offset, () => new Regex(pattern, options));
Throws(pattern, options, error.Value, offset, () => new Regex(pattern, options));
return;
}

Assert.Equal(-1, offset);
LogActual(pattern, options, RegexParseError.Unknown, -1);

// Nothing to assert here without having access to internals.
new Regex(pattern, options); // Does not throw

ParsePatternFragments(pattern, options);
}

private static void LogActual(string pattern, RegexOptions options, RegexParseError error, int offset)
{
// To conveniently add new interesting patterns to these tests, add them to the code in the format:
//
// [InlineData("SOMEREGEX1", RegexOptions.None, null)]
// [InlineData("SOMEREGEX2", RegexOptions.None, null)]
// ...
//
// then uncomment the lines below, and the correct baseline will be written to the file, eg
//
// [InlineData(@"SOMEREGEX1", RegexOptions.None, RegexParseError.UnrecognizedEscape, 3)]
// [InlineData(@"SOMEREGEX2", RegexOptions.None, InsufficientClosingParentheses, 2)]
// ...
//
//string s = (error == RegexParseError.Unknown) ?
// @$" [InlineData(@""{pattern}"", RegexOptions.{options.ToString()}, null)]" :
// @$" [InlineData(@""{pattern}"", RegexOptions.{options.ToString()}, RegexParseError.{error.ToString()}, {offset})]";

// File.AppendAllText(@"/tmp/out.cs", s + "\n");
}
Copy link
Member

@stephentoub stephentoub Nov 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand... what's the purpose of this method?

Copy link
Member Author

@danmoseley danmoseley Nov 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It updates the baseline: I can take a bunch of presumably-invalid regex inputs from someplace, and turn them into this format:

[InlineData("SOMEREGEX1", RegexOptions.None, null)]
[InlineData("SOMEREGEX2", RegexOptions.None, null)]
...

I paste those into the test, uncomment this line, and run the test, and it emits into that file exactly what I need to paste in eg

[InlineData(@"SOMEREGEX1", RegexOptions.None, RegexParseError.UnrecognizedEscape, 3)]
[InlineData(@"SOMEREGEX2", RegexOptions.None, InsufficientClosingParentheses, 2)]
...

I then replace what I pasted in with those, and I have my updated test.

Copy link
Member Author

@danmoseley danmoseley Nov 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is that OK? I'm finding this a useful approach: have the test construct the baseline.

Of course, I'll eyeball to satisfy myself that the results are good. In some cases, I'm emitting in a different format locally to diff against the original test output as well.

Copy link
Member Author

@danmoseley danmoseley Nov 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment.

Copy link
Member

@stephentoub stephentoub Nov 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to do easily as a standalone, commented out test? I think what I'm objecting to is it's modifying other tests with dead code that is almost certain to rot.

Copy link
Member Author

@danmoseley danmoseley Nov 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I can change it to that.


private static void ParsePatternFragments(string pattern, RegexOptions options)
{
// Trim the input in various places and parse.
Expand Down Expand Up @@ -755,7 +777,7 @@ private static void ParsePatternFragments(string pattern, RegexOptions options)
/// </summary>
/// <param name="error">The expected parse error</param>
/// <param name="action">The action to invoke.</param>
static partial void Throws(RegexParseError error, int offset, Action action);
static partial void Throws(string pattern, RegexOptions options, RegexParseError error, int offset, Action action);

/// <summary>
/// Checks that action succeeds or throws either a RegexParseException or an ArgumentException depending on the
Expand Down
118 changes: 115 additions & 3 deletions src/libraries/System.Text.RegularExpressions/tests/RegexParserTests.netcoreapp.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ public partial class RegexParserTests
[InlineData(@"(?P<a>.)(?P<a>.)", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 3)]
[InlineData(@"[a-\A]", RegexOptions.None, RegexParseError.UnrecognizedEscape, 5)]
[InlineData(@"[a-\z]", RegexOptions.None, RegexParseError.UnrecognizedEscape, 5)]
[InlineData(@"[a-\b]", RegexOptions.None, RegexParseError.ReversedCharacterRange, 5)]
[InlineData(@"[a-\b]", RegexOptions.None, RegexParseError.ReversedCharacterRange, 5)] // Nim: not an error
[InlineData(@"[a-\-]", RegexOptions.None, RegexParseError.ReversedCharacterRange, 5)]
[InlineData(@"[a-\-b]", RegexOptions.None, RegexParseError.ReversedCharacterRange, 5)]
[InlineData(@"[a-\-\-b]", RegexOptions.None, RegexParseError.ReversedCharacterRange, 5)]
Expand All @@ -127,6 +127,115 @@ public partial class RegexParserTests
[InlineData(@"[a-[:lower:]]", RegexOptions.None, null)] // errors in rust: range_end_no_class
// End of Rust parser tests ==============

// Following are borrowed from Nim tests
// https://github.com/nitely/nim-regex/blob/eeefb4f51264ff3bc3b36caf55672a74f52f5ef5/tests/tests.nim
[InlineData(@"?", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 1)]
[InlineData(@"?|?", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 1)]
[InlineData(@"?abc", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 1)]
[InlineData(@"(?P<abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_>abc", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 3)] // Nim: not an error
[InlineData(@"(?Pabc", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 3)]
[InlineData(@"(?u-q)", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 3)]
[InlineData(@"(?uq)", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 3)]
[InlineData(@"(\b)", RegexOptions.None, null)]
[InlineData(@"(+)", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 2)]
[InlineData(@"(a)b)", RegexOptions.None, RegexParseError.InsufficientOpeningParentheses, 5)]
[InlineData(@"(b(a)", RegexOptions.None, RegexParseError.InsufficientClosingParentheses, 5)]
[InlineData(@"[-", RegexOptions.None, RegexParseError.UnterminatedBracket, 2)]
[InlineData(@"[-a", RegexOptions.None, RegexParseError.UnterminatedBracket, 3)]
[InlineData(@"[[:abc:]]", RegexOptions.None, null)] // Nim: "Invalid ascii set. `abc` is not a valid name"
[InlineData(@"[[:alnum:", RegexOptions.None, RegexParseError.UnterminatedBracket, 9)]
[InlineData(@"[[:alnum]]", RegexOptions.None, null)] // Nim: "Invalid ascii set. Expected [:name:]"
[InlineData(@"[]", RegexOptions.None, RegexParseError.UnterminatedBracket, 2)]
[InlineData(@"[]a", RegexOptions.None, RegexParseError.UnterminatedBracket, 3)]
[InlineData(@"[]abc", RegexOptions.None, RegexParseError.UnterminatedBracket, 5)]
[InlineData(@"[\\", RegexOptions.None, RegexParseError.UnterminatedBracket, 3)]
[InlineData(@"[^]", RegexOptions.None, RegexParseError.UnterminatedBracket, 3)]
[InlineData(@"[a-", RegexOptions.None, RegexParseError.UnterminatedBracket, 3)]
[InlineData(@"[a-\w]", RegexOptions.None, RegexParseError.ShorthandClassInCharacterRange, 5)]
[InlineData(@"[a", RegexOptions.None, RegexParseError.UnterminatedBracket, 2)]
[InlineData(@"[abc", RegexOptions.None, RegexParseError.UnterminatedBracket, 4)]
[InlineData(@"[d-c]", RegexOptions.None, RegexParseError.ReversedCharacterRange, 4)]
[InlineData(@"[z-[:alnum:]]", RegexOptions.None, null)] // Nim: "Invalid set range. Start must be lesser than end"
[InlineData(@"{10}", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 1)]
[InlineData(@"*abc", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 1)]
[InlineData(@"\12", RegexOptions.None, null)] // Nim: "Invalid octal literal. Expected 3 octal digits, but found 2"
[InlineData(@"\12@", RegexOptions.None, null)] // Nim: "Invalid octal literal. Expected octal digit, but found @"
[InlineData(@"\b?", RegexOptions.None, null)]
[InlineData(@"\b*", RegexOptions.None, null)]
[InlineData(@"\b+", RegexOptions.None, null)]
[InlineData(@"\p{11", RegexOptions.None, RegexParseError.InvalidUnicodePropertyEscape, 5)]
[InlineData(@"\p{11}", RegexOptions.None, RegexParseError.UnrecognizedUnicodeProperty, 6)]
[InlineData(@"\p{Bb}", RegexOptions.None, RegexParseError.UnrecognizedUnicodeProperty, 6)]
[InlineData(@"\p11", RegexOptions.None, RegexParseError.InvalidUnicodePropertyEscape, 2)]
[InlineData(@"\pB", RegexOptions.None, RegexParseError.InvalidUnicodePropertyEscape, 2)]
[InlineData(@"\u123", RegexOptions.None, RegexParseError.InsufficientOrInvalidHexDigits, 2)]
[InlineData(@"\U123", RegexOptions.None, RegexParseError.UnrecognizedEscape, 2)]
[InlineData(@"\U123@a", RegexOptions.None, RegexParseError.UnrecognizedEscape, 2)]
[InlineData(@"\u123@abc", RegexOptions.None, RegexParseError.InsufficientOrInvalidHexDigits, 6)]
[InlineData(@"\UFFFFFFFF", RegexOptions.None, RegexParseError.UnrecognizedEscape, 2)]
[InlineData(@"\x{00000000A}", RegexOptions.None, RegexParseError.InsufficientOrInvalidHexDigits, 3)]
[InlineData(@"\x{2f894", RegexOptions.None, RegexParseError.InsufficientOrInvalidHexDigits, 3)]
[InlineData(@"\x{61@}", RegexOptions.None, RegexParseError.InsufficientOrInvalidHexDigits, 3)]
[InlineData(@"\x{7fffffff}", RegexOptions.None, RegexParseError.InsufficientOrInvalidHexDigits, 3)] // Nim: not an error (supports Unicode beyond basic multilingual plane)
[InlineData(@"\x{FFFFFFFF}", RegexOptions.None, RegexParseError.InsufficientOrInvalidHexDigits, 3)]
[InlineData(@"+", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 1)]
[InlineData(@"+abc", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 1)]
[InlineData(@"a???", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 4)]
[InlineData(@"a??*", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 4)]
[InlineData(@"a??+", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 4)]
[InlineData(@"a?*", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 3)]
[InlineData(@"a?+", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 3)]
[InlineData(@"a(?P<>abc)", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 4)]
[InlineData(@"a(?P<asd)", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 4)]
[InlineData(@"a{,}", RegexOptions.None, null)] // Nim error
[InlineData(@"a{,1}", RegexOptions.None, null)] // Nim error
[InlineData(@"a{0,101}", RegexOptions.None, null)] // Nim error: "Invalid repetition range. Expected 100 repetitions or less, but found: 101"
[InlineData(@"a{0,a}", RegexOptions.None, null)] // Nim error
[InlineData(@"a{0,bad}", RegexOptions.None, null)] // Nim error: "Invalid repetition range. Range can only contain digits"
[InlineData(@"a{1,,,2}", RegexOptions.None, null)] // Nim error
[InlineData(@"a{1,,}", RegexOptions.None, null)] // Nim error
[InlineData(@"a{1,,2}", RegexOptions.None, null)] // Nim error
[InlineData(@"a{1,", RegexOptions.None, null)] // Nim error
[InlineData(@"a{1,}??", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 7)]
[InlineData(@"a{1,}?", RegexOptions.None, null)]
[InlineData(@"a{1,}*", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 6)]
[InlineData(@"a{1,}+", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 6)]
[InlineData(@"a{1,101}", RegexOptions.None, null)]
[InlineData(@"a{1,x}", RegexOptions.None, null)] // Nim error
[InlineData(@"a{1", RegexOptions.None, null)] // Nim error
[InlineData(@"a{1}??", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 6)]
[InlineData(@"a{1}?", RegexOptions.None, null)]
[InlineData(@"a{1}*", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 5)]
[InlineData(@"a{1}+", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 5)]
[InlineData(@"a{1111111111}", RegexOptions.None, null)] // Nim error: "Invalid repetition range. Max value is 32767."
[InlineData(@"a{1x}", RegexOptions.None, null)] // Nim error
[InlineData(@"a*??", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 4)]
[InlineData(@"a*{,}", RegexOptions.None, null)] // Nim error
[InlineData(@"a*{0}", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 3)]
[InlineData(@"a*{1}", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 3)]
[InlineData(@"a**", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 3)]
[InlineData(@"a*****", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 3)]
[InlineData(@"a*+", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 3)]
[InlineData(@"a+??", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 4)]
[InlineData(@"a+*", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 3)]
[InlineData(@"a++", RegexOptions.None, RegexParseError.NestedQuantifiersNotParenthesized, 3)]
[InlineData(@"a|?", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 3)]
[InlineData(@"a|?b", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 3)]
[InlineData(@"a|*", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 3)]
[InlineData(@"a|*b", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 3)]
[InlineData(@"a|+", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 3)]
[InlineData(@"a|+b", RegexOptions.None, RegexParseError.QuantifierAfterNothing, 3)]
[InlineData(@"aaa(?Pabc", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 6)]
[InlineData(@"abc(?P<abc", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 6)]
[InlineData(@"abc(?Pabc)", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 6)]
[InlineData(@"abc(?q)", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 6)]
[InlineData(@"abc[]", RegexOptions.None, RegexParseError.UnterminatedBracket, 5)]
[InlineData(@"abc\A{10}", RegexOptions.None, null)] // Nim error: "Invalid repetition range, either char, shorthand (i.e: \\w), group, or set expected before repetition range"
[InlineData(@"\uD87E\uDC94(?Pabc", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 15)]
[InlineData(@"\uD87E\uDC94aaa(?Pabc", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 18)]
[InlineData(@"\uD87E\uDC94\uD87E\uDC94\uD87E\uDC94(?Pabc", RegexOptions.None, RegexParseError.InvalidGroupingConstruct, 39)]
// End of Nim parser tests ==============

[SkipOnTargetFramework(TargetFrameworkMonikers.NetFramework)]
public void Parse_Netcoreapp(string pattern, RegexOptions options, RegexParseError? error, int offset = -1)
{
Expand Down Expand Up @@ -157,7 +266,7 @@ public void RegexParseException_Serializes()
/// </summary>
/// <param name="error">The expected parse error</param>
/// <param name="action">The action to invoke.</param>
static partial void Throws(RegexParseError error, int offset, Action action)
static partial void Throws(string pattern, RegexOptions options, RegexParseError error, int offset, Action action)
{
try
{
Expand All @@ -171,16 +280,19 @@ static partial void Throws(RegexParseError error, int offset, Action action)
if (error == regexParseError)
{
Assert.Equal(offset, e.Offset);
LogActual(pattern, options, regexParseError, e.Offset);
return;
}

LogActual(pattern, options, regexParseError, e.Offset);
throw new XunitException($"Expected RegexParseException with error {error} offset {offset} -> Actual error: {regexParseError} offset {e.Offset})");
}
catch (Exception e)
{
throw new XunitException($"Expected RegexParseException -> Actual: ({e})");
throw new XunitException($"Expected RegexParseException for pattern '{pattern}' -> Actual: ({e})");
}

LogActual(pattern, options, RegexParseError.Unknown, -1);
throw new XunitException($"Expected RegexParseException with error: ({error}) -> Actual: No exception thrown");
}

Expand Down
2 changes: 1 addition & 1 deletion src/libraries/System.Text.RegularExpressions/tests/RegexParserTests.netfx.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ public partial class RegexParserTests
/// </summary>
/// <param name="error">The expected parse error</param>
/// <param name="action">The action to invoke.</param>
static partial void Throws(RegexParseError error, int offset, Action action)
static partial void Throws(string pattern, RegexOptions options, RegexParseError error, int offset, Action action)
{
try
{
Expand Down
27 changes: 26 additions & 1 deletion src/libraries/System.Text.RegularExpressions/tests/THIRD-PARTY-NOTICES.TXT
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,29 @@ SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
DEALINGS IN THE SOFTWARE.

License notice for https://github.com/nitely/nim-regex
-------------------------------

MIT License

Copyright (c) 2017 Esteban Castro Borsani

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.