Skip to content

Commit 48d760d

Browse files
maropugatorsmile
authored andcommitted
[SPARK-20281][SQL] Print the identical Range parameters of SparkContext APIs and SQL in explain
## What changes were proposed in this pull request? This pr modified code to print the identical `Range` parameters of SparkContext APIs and SQL in `explain` output. In the current master, they internally use `defaultParallelism` for `splits` by default though, they print different strings in explain output; ``` scala> spark.range(4).explain == Physical Plan == *Range (0, 4, step=1, splits=Some(8)) scala> sql("select * from range(4)").explain == Physical Plan == *Range (0, 4, step=1, splits=None) ``` ## How was this patch tested? Added tests in `SQLQuerySuite` and modified some results in the existing tests. Author: Takeshi Yamamuro <[email protected]> Closes #17670 from maropu/SPARK-20281.
1 parent 760c8d0 commit 48d760d

File tree

3 files changed

+4
-3
lines changed

3 files changed

+4
-3
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -332,6 +332,7 @@ case class RangeExec(range: org.apache.spark.sql.catalyst.plans.logical.Range)
332332
extends LeafExecNode with CodegenSupport {
333333

334334
def start: Long = range.start
335+
def end: Long = range.end
335336
def step: Long = range.step
336337
def numSlices: Int = range.numSlices.getOrElse(sparkContext.defaultParallelism)
337338
def numElements: BigInt = range.numElements
@@ -538,7 +539,7 @@ case class RangeExec(range: org.apache.spark.sql.catalyst.plans.logical.Range)
538539
}
539540
}
540541

541-
override def simpleString: String = range.simpleString
542+
override def simpleString: String = s"Range ($start, $end, step=$step, splits=$numSlices)"
542543
}
543544

544545
/**

sql/core/src/test/resources/sql-tests/results/sql-compatibility-functions.sql.out

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ Project [coalesce(cast(id#xL as string), x) AS ifnull(`id`, 'x')#x, id#xL AS nul
8888

8989
== Physical Plan ==
9090
*Project [coalesce(cast(id#xL as string), x) AS ifnull(`id`, 'x')#x, id#xL AS nullif(`id`, 'x')#xL, coalesce(cast(id#xL as string), x) AS nvl(`id`, 'x')#x, x AS nvl2(`id`, 'x', 'y')#x]
91-
+- *Range (0, 2, step=1, splits=None)
91+
+- *Range (0, 2, step=1, splits=2)
9292

9393

9494
-- !query 9

sql/core/src/test/resources/sql-tests/results/table-valued-functions.sql.out

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,4 +102,4 @@ EXPLAIN select * from RaNgE(2)
102102
struct<plan:string>
103103
-- !query 8 output
104104
== Physical Plan ==
105-
*Range (0, 2, step=1, splits=None)
105+
*Range (0, 2, step=1, splits=2)

0 commit comments

Comments
 (0)