Skip to content

Commit 6809d1f

Browse files
author
jinxing
committed
refine comments.
1 parent d225565 commit 6809d1f

File tree

1 file changed

+3
-4
lines changed

1 file changed

+3
-4
lines changed

core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1189,10 +1189,9 @@ class DAGScheduler(
11891189
// This task was for the currently running attempt of the stage. Since the task
11901190
// completed successfully from the perspective of the TaskSetManager, mark it as
11911191
// no longer pending (the TaskSetManager may consider the task complete even
1192-
// when the output needs to be ignored because the task's epoch is too small below,
1193-
// if so, this can result in inconsistency between pending partitions and output
1194-
// locations of stage. When pending partitions is empty, the scheduler will check
1195-
// output locations, if there is missing, the stage will be resubmitted.
1192+
// when the output needs to be ignored because the task's epoch is too small below.
1193+
// In this case, when pending partitions is empty, there will still be missing
1194+
// output locations, which will cause the DAGScheduler to resubmit the stage below.)
11961195
shuffleStage.pendingPartitions -= task.partitionId
11971196
}
11981197
if (failedEpoch.contains(execId) && smt.epoch <= failedEpoch(execId)) {

0 commit comments

Comments
 (0)