Skip to content

Commit 442287a

Browse files
viiryacloud-fan
authored andcommitted
[SPARK-20399][SQL][FOLLOW-UP] Add a config to fallback string literal parsing consistent with old sql parser behavior
## What changes were proposed in this pull request? As srowen pointed in 609ba5f#commitcomment-22221259, the previous tests are not proper. This follow-up is going to fix the tests. ## How was this patch tested? Jenkins tests. Please review http://spark.apache.org/contributing.html before opening a pull request. Author: Liang-Chi Hsieh <[email protected]> Closes #18048 from viirya/SPARK-20399-follow-up.
1 parent d06610f commit 442287a

File tree

1 file changed

+19
-11
lines changed

1 file changed

+19
-11
lines changed

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala

Lines changed: 19 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -463,22 +463,30 @@ class ExpressionParserSuite extends PlanTest {
463463
assertEqual("'pattern\\\\\\%'", "pattern\\\\\\%", parser)
464464

465465
// Escaped characters.
466-
assertEqual("'\0'", "\u0000", parser) // ASCII NUL (X'00')
466+
// Unescape string literal "'\\0'" for ASCII NUL (X'00') doesn't work
467+
// when ESCAPED_STRING_LITERALS is enabled.
468+
// It is parsed literally.
469+
assertEqual("'\\0'", "\\0", parser)
467470

468471
// Note: Single quote follows 1.6 parsing behavior when ESCAPED_STRING_LITERALS is enabled.
469472
val e = intercept[ParseException](parser.parseExpression("'\''"))
470473
assert(e.message.contains("extraneous input '''"))
471474

472-
assertEqual("'\"'", "\"", parser) // Double quote
473-
assertEqual("'\b'", "\b", parser) // Backspace
474-
assertEqual("'\n'", "\n", parser) // Newline
475-
assertEqual("'\r'", "\r", parser) // Carriage return
476-
assertEqual("'\t'", "\t", parser) // Tab character
477-
478-
// Octals
479-
assertEqual("'\110\145\154\154\157\041'", "Hello!", parser)
480-
// Unicode
481-
assertEqual("'\u0057\u006F\u0072\u006C\u0064\u0020\u003A\u0029'", "World :)", parser)
475+
// The unescape special characters (e.g., "\\t") for 2.0+ don't work
476+
// when ESCAPED_STRING_LITERALS is enabled. They are parsed literally.
477+
assertEqual("'\\\"'", "\\\"", parser) // Double quote
478+
assertEqual("'\\b'", "\\b", parser) // Backspace
479+
assertEqual("'\\n'", "\\n", parser) // Newline
480+
assertEqual("'\\r'", "\\r", parser) // Carriage return
481+
assertEqual("'\\t'", "\\t", parser) // Tab character
482+
483+
// The unescape Octals for 2.0+ don't work when ESCAPED_STRING_LITERALS is enabled.
484+
// They are parsed literally.
485+
assertEqual("'\\110\\145\\154\\154\\157\\041'", "\\110\\145\\154\\154\\157\\041", parser)
486+
// The unescape Unicode for 2.0+ doesn't work when ESCAPED_STRING_LITERALS is enabled.
487+
// They are parsed literally.
488+
assertEqual("'\\u0057\\u006F\\u0072\\u006C\\u0064\\u0020\\u003A\\u0029'",
489+
"\\u0057\\u006F\\u0072\\u006C\\u0064\\u0020\\u003A\\u0029", parser)
482490
} else {
483491
// Default behavior
484492

0 commit comments

Comments
 (0)