File tree Expand file tree Collapse file tree 1 file changed +3
-4
lines changed
core/src/main/scala/org/apache/spark/scheduler Expand file tree Collapse file tree 1 file changed +3
-4
lines changed Original file line number Diff line number Diff line change @@ -1189,10 +1189,9 @@ class DAGScheduler(
11891189 // This task was for the currently running attempt of the stage. Since the task
11901190 // completed successfully from the perspective of the TaskSetManager, mark it as
11911191 // no longer pending (the TaskSetManager may consider the task complete even
1192- // when the output needs to be ignored because the task's epoch is too small below,
1193- // if so, this can result in inconsistency between pending partitions and output
1194- // locations of stage. When pending partitions is empty, the scheduler will check
1195- // output locations, if there is missing, the stage will be resubmitted.
1192+ // when the output needs to be ignored because the task's epoch is too small below.
1193+ // In this case, when pending partitions is empty, there will still be missing
1194+ // output locations, which will cause the DAGScheduler to resubmit the stage below.)
11961195 shuffleStage.pendingPartitions -= task.partitionId
11971196 }
11981197 if (failedEpoch.contains(execId) && smt.epoch <= failedEpoch(execId)) {
You can’t perform that action at this time.
0 commit comments