Skip to content

Commit 13eb37c

Browse files
HyukjinKwonrxin
authored andcommitted
[MINOR][SQL] Fix the test title from =!= to <=>, remove a duplicated test and add a test for =!=
## What changes were proposed in this pull request? This PR proposes three things as below: - This test looks not testing `<=>` and identical with the test above, `===`. So, it removes the test. ```diff - test("<=>") { - checkAnswer( - testData2.filter($"a" === 1), - testData2.collect().toSeq.filter(r => r.getInt(0) == 1)) - - checkAnswer( - testData2.filter($"a" === $"b"), - testData2.collect().toSeq.filter(r => r.getInt(0) == r.getInt(1))) - } ``` - Replace the test title from `=!=` to `<=>`. It looks the test actually testing `<=>`. ```diff + private lazy val nullData = Seq( + (Some(1), Some(1)), (Some(1), Some(2)), (Some(1), None), (None, None)).toDF("a", "b") + ... - test("=!=") { + test("<=>") { - val nullData = spark.createDataFrame(sparkContext.parallelize( - Row(1, 1) :: - Row(1, 2) :: - Row(1, null) :: - Row(null, null) :: Nil), - StructType(Seq(StructField("a", IntegerType), StructField("b", IntegerType)))) - checkAnswer( nullData.filter($"b" <=> 1), ... ``` - Add the tests for `=!=` which looks not existing. ```diff + test("=!=") { + checkAnswer( + nullData.filter($"b" =!= 1), + Row(1, 2) :: Nil) + + checkAnswer(nullData.filter($"b" =!= null), Nil) + + checkAnswer( + nullData.filter($"a" =!= $"b"), + Row(1, 2) :: Nil) + } ``` ## How was this patch tested? Manually running the tests. Author: hyukjinkwon <[email protected]> Closes #17842 from HyukjinKwon/minor-test-fix.
1 parent 6b9e49d commit 13eb37c

File tree

1 file changed

+14
-17
lines changed

1 file changed

+14
-17
lines changed

sql/core/src/test/scala/org/apache/spark/sql/ColumnExpressionSuite.scala

Lines changed: 14 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,9 @@ class ColumnExpressionSuite extends QueryTest with SharedSQLContext {
3939
StructType(Seq(StructField("a", BooleanType), StructField("b", BooleanType))))
4040
}
4141

42+
private lazy val nullData = Seq(
43+
(Some(1), Some(1)), (Some(1), Some(2)), (Some(1), None), (None, None)).toDF("a", "b")
44+
4245
test("column names with space") {
4346
val df = Seq((1, "a")).toDF("name with space", "name.with.dot")
4447

@@ -283,23 +286,6 @@ class ColumnExpressionSuite extends QueryTest with SharedSQLContext {
283286
}
284287

285288
test("<=>") {
286-
checkAnswer(
287-
testData2.filter($"a" === 1),
288-
testData2.collect().toSeq.filter(r => r.getInt(0) == 1))
289-
290-
checkAnswer(
291-
testData2.filter($"a" === $"b"),
292-
testData2.collect().toSeq.filter(r => r.getInt(0) == r.getInt(1)))
293-
}
294-
295-
test("=!=") {
296-
val nullData = spark.createDataFrame(sparkContext.parallelize(
297-
Row(1, 1) ::
298-
Row(1, 2) ::
299-
Row(1, null) ::
300-
Row(null, null) :: Nil),
301-
StructType(Seq(StructField("a", IntegerType), StructField("b", IntegerType))))
302-
303289
checkAnswer(
304290
nullData.filter($"b" <=> 1),
305291
Row(1, 1) :: Nil)
@@ -321,7 +307,18 @@ class ColumnExpressionSuite extends QueryTest with SharedSQLContext {
321307
checkAnswer(
322308
nullData2.filter($"a" <=> null),
323309
Row(null) :: Nil)
310+
}
324311

312+
test("=!=") {
313+
checkAnswer(
314+
nullData.filter($"b" =!= 1),
315+
Row(1, 2) :: Nil)
316+
317+
checkAnswer(nullData.filter($"b" =!= null), Nil)
318+
319+
checkAnswer(
320+
nullData.filter($"a" =!= $"b"),
321+
Row(1, 2) :: Nil)
325322
}
326323

327324
test(">") {

0 commit comments

Comments
 (0)