Skip to content

Commit 28a6cca

Browse files
viiryacloud-fan
authored andcommitted
[SPARK-21721][SQL][FOLLOWUP] Clear FileSystem deleteOnExit cache when paths are successfully removed
## What changes were proposed in this pull request? Fix a typo in test. ## How was this patch tested? Jenkins tests. Author: Liang-Chi Hsieh <[email protected]> Closes #19005 from viirya/SPARK-21721-followup.
1 parent 41e0eb7 commit 28a6cca

File tree

1 file changed

+4
-3
lines changed

1 file changed

+4
-3
lines changed

sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2023,18 +2023,19 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
20232023
}
20242024

20252025
test("SPARK-21721: Clear FileSystem deleterOnExit cache if path is successfully removed") {
2026-
withTable("test21721") {
2026+
val table = "test21721"
2027+
withTable(table) {
20272028
val deleteOnExitField = classOf[FileSystem].getDeclaredField("deleteOnExit")
20282029
deleteOnExitField.setAccessible(true)
20292030

20302031
val fs = FileSystem.get(spark.sparkContext.hadoopConfiguration)
20312032
val setOfPath = deleteOnExitField.get(fs).asInstanceOf[Set[Path]]
20322033

20332034
val testData = sparkContext.parallelize(1 to 10).map(i => TestData(i, i.toString)).toDF()
2034-
sql("CREATE TABLE test21721 (key INT, value STRING)")
2035+
sql(s"CREATE TABLE $table (key INT, value STRING)")
20352036
val pathSizeToDeleteOnExit = setOfPath.size()
20362037

2037-
(0 to 10).foreach(_ => testData.write.mode(SaveMode.Append).insertInto("test1"))
2038+
(0 to 10).foreach(_ => testData.write.mode(SaveMode.Append).insertInto(table))
20382039

20392040
assert(setOfPath.size() == pathSizeToDeleteOnExit)
20402041
}

0 commit comments

Comments
 (0)