[SPARK-1824] Remove <master> from Python examples #802
Conversation
|
Merged build triggered. |
|
Merged build started. |
Note that this reflects changes incorporated in apache#799.
|
Merged build triggered. |
|
Merged build started. |
docs/index.md
Outdated
There was a problem hiding this comment.
We should also use bin/spark-submit instead of bin/pyspark here.
|
I think we should deprecate using |
|
Merged build finished. All automated tests passed. |
1 similar comment
|
Merged build finished. All automated tests passed. |
|
All automated tests passed. |
|
All automated tests passed. |
|
I see. I didn't realize we want to deprecate using |
|
Yeah, it wasn't super obvious but that's a good direction to go. I replaced pyspark with spark-submit in many of the docs on running applications, but I missed some. |
|
Merged build triggered. |
|
Merged build started. |
|
Merged build finished. All automated tests passed. |
|
All automated tests passed. |
|
This is ready to merge (after #799) |
|
Merged build triggered. |
|
Merged build started. |
|
Merged build finished. All automated tests passed. |
|
All automated tests passed. |
A recent PR (#552) fixed this for all Scala / Java examples. We need to do it for python too. Note that this blocks on #799, which makes `bin/pyspark` go through Spark submit. With only the changes in this PR, the only way to run these examples is through Spark submit. Once #799 goes in, you can use `bin/pyspark` to run them too. For example, ``` bin/pyspark examples/src/main/python/pi.py 100 --master local-cluster[4,1,512] ``` Author: Andrew Or <[email protected]> Closes #802 from andrewor14/python-examples and squashes the following commits: cf50b9f [Andrew Or] De-indent python comments (minor) 50f80b1 [Andrew Or] Remove pyFiles from SparkContext construction c362f69 [Andrew Or] Update docs to use spark-submit for python applications 7072c6a [Andrew Or] Merge branch 'master' of github.com:apache/spark into python-examples 427a5f0 [Andrew Or] Update docs d32072c [Andrew Or] Remove <master> from examples + update usages (cherry picked from commit cf6cbe9) Signed-off-by: Patrick Wendell <[email protected]>
A recent PR (apache#552) fixed this for all Scala / Java examples. We need to do it for python too. Note that this blocks on apache#799, which makes `bin/pyspark` go through Spark submit. With only the changes in this PR, the only way to run these examples is through Spark submit. Once apache#799 goes in, you can use `bin/pyspark` to run them too. For example, ``` bin/pyspark examples/src/main/python/pi.py 100 --master local-cluster[4,1,512] ``` Author: Andrew Or <[email protected]> Closes apache#802 from andrewor14/python-examples and squashes the following commits: cf50b9f [Andrew Or] De-indent python comments (minor) 50f80b1 [Andrew Or] Remove pyFiles from SparkContext construction c362f69 [Andrew Or] Update docs to use spark-submit for python applications 7072c6a [Andrew Or] Merge branch 'master' of github.com:apache/spark into python-examples 427a5f0 [Andrew Or] Update docs d32072c [Andrew Or] Remove <master> from examples + update usages
3 participants
A recent PR (#552) fixed this for all Scala / Java examples. We need to do it for python too.
Note that this blocks on #799, which makes
bin/pysparkgo through Spark submit. With only the changes in this PR, the only way to run these examples is through Spark submit. Once #799 goes in, you can usebin/pysparkto run them too. For example,