Commit 3772d93
[SPARK-26307][SQL] Fix CTAS when INSERT a partitioned table using Hive serde
This is a Spark 2.3 regression introduced in #20521. We should add the partition info for InsertIntoHiveTable in CreateHiveTableAsSelectCommand. Otherwise, we will hit the following error by running the newly added test case:
```
[info] - CTAS: INSERT a partitioned table using Hive serde *** FAILED *** (829 milliseconds)
[info] org.apache.spark.SparkException: Requested partitioning does not match the tab1 table:
[info] Requested partitions:
[info] Table partitions: part
[info] at org.apache.spark.sql.hive.execution.InsertIntoHiveTable.processInsert(InsertIntoHiveTable.scala:179)
[info] at org.apache.spark.sql.hive.execution.InsertIntoHiveTable.run(InsertIntoHiveTable.scala:107)
```
Added a test case.
Closes #23255 from gatorsmile/fixCTAS.
Authored-by: gatorsmile <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit 3bc83de)
Signed-off-by: Wenchen Fan <[email protected]>1 parent 1899dd2 commit 3772d93
File tree
2 files changed
+13
-1
lines changed- sql/hive/src
- main/scala/org/apache/spark/sql/hive/execution
- test/scala/org/apache/spark/sql/hive
2 files changed
+13
-1
lines changedLines changed: 3 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
| 60 | + | |
| 61 | + | |
60 | 62 | | |
61 | 63 | | |
62 | | - | |
| 64 | + | |
63 | 65 | | |
64 | 66 | | |
65 | 67 | | |
| |||
Lines changed: 10 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
750 | 750 | | |
751 | 751 | | |
752 | 752 | | |
| 753 | + | |
| 754 | + | |
| 755 | + | |
| 756 | + | |
| 757 | + | |
| 758 | + | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
753 | 763 | | |
0 commit comments