[SPARK-3292][streaming]Fix:Shuffle Tasks run incessantly even though there's no inputs

[guowei2]

with this PR: no job is commited when there's no imputs.

[AmplabJenkins]

Can one of the admins verify this patch?

[guowei2]

i recommit this.for the last PR is compicated by mistake

[SparkQA]

Can one of the admins verify this patch?

[rxin]

@tdas can you take a look at this?

[tdas]

This is not a good idea. Not returning an RDD can mess up a lot of the logic and semantics. For example if there is a transform() followed by updateStateByKey(), the result will be unpredictable. updateStateByKey expects the previous batch to have a state RDD. If it does not find any state RDD it will assume that this the start of the streamign computation and effectively initialize again, forgetting the previous states from 2 batches ago. So this change is incorrect.

[tdas]

@guowei2 As i had explained, this is not a good idea because it breaks semantics for a state dstream. Mind closing this PR?