[SPARK-5477] refactor stat.py

[mengxr]

There is only a single stat.py file for the mllib.stat package. We recently added MultivariateGaussian under mllib.stat.distribution in Scala/Java. It would be nice to refactor stat.py and make it easy to expand. Note that ChiSqTestResult is moved from mllib.stat to mllib.stat.test. The latter is used in Scala/Java. It is only used in the return value of Statistics.chiSqTest, so this should be an okay change.

@davies

[SparkQA]

Test build #26305 has started for PR 4266 at commit 1a5e1db.

• This patch merges cleanly.

[SparkQA]

Test build #26305 has finished for PR 4266 at commit 1a5e1db.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26305/
Test PASSed.

[davies]

Is it OK to break compatibility? Otherwise looks good to me.

[mengxr]

It is unfortunate that we didn't put ChiSqTestResult under stat.test in the first version. We are not expecting users to construct ChiSqTestResult directly, but get it from Statistic.chiSqTest() instead. So I'd like to make this change earlier to keep the API consistent with Scala/Java's. I'm merging this in and will leave a note on the migration guide.

[mengxr]

Merged into master.