[SPARK-18278] [Scheduler] Documentation to point to Kubernetes cluster scheduler

[foxish]

What changes were proposed in this pull request?

Adding documentation to point to Kubernetes cluster scheduler being developed out-of-repo in https://github.com/apache-spark-on-k8s/spark
cc @rxin @srowen @tnachen @ash211 @mccheah @erikerlandson

How was this patch tested?

Docs only change

[AmplabJenkins]

Can one of the admins verify this patch?

[markhamstra]

I'd change "will" to "may" or "is intended to" or something similar since there is not yet a formal, hard commitment to definitely merge this work into Apache Spark.

[mridulm]

I dont think we should be pointing to third party projects in spark documentation - for example, it might be possible that some other effort gets merged in instead of the above.

If/when it does eventually get merged, we can add the appropriate cluster manager entry for it - until then, there are other means of evangelizing user participation.

[rxin]

Seems fine to me, since the number of external resource managers are small. We should definitely make it clear there is no firm commitment currently to merge this into Spark though.

[foxish]

@mridulm, I understand your concern here. This is however an effort from the Kubernetes community (kubernetes/kubernetes#34377), so, the eventuality of a different parallel effort, is unlikely.
@rxin thanks for reviewing. I've updated the wording as @markhamstra just suggested. Do we want more clarification about the level of commitment or does this look ok?

[srowen]

This seems fine, except that I would not represent that it will be merged

[foxish]

Removed the controversial line about the merging upstream. PTAL

[rxin]

Thanks - merging in master.