Skip to content

[SPARK-8063][SPARKR] Spark master URL conflict between MASTER env variable and --master command line option. #6605

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[SPARK-8063][SPARKR] Spark master URL conflict between MASTER env variable and --master command line option. #6605

wants to merge 1 commit into from

Conversation

@sun-rui
Copy link
Contributor

@sun-rui sun-rui commented Jun 3, 2015

No description provided.

@shivaram
Copy link
Contributor

shivaram commented Jun 3, 2015

hmm -- so just to confirm, does the MASTER env variable get read by spark-submit automatically ?

@SparkQA
Copy link

SparkQA commented Jun 3, 2015

Test build #34063 has finished for PR 6605 at commit 51ca48b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@andrewor14
Copy link
Contributor

@shivaram yes it does:

spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala

Line 138 in 2c4d550

.orElse(env.get("MASTER"))

@andrewor14
Copy link
Contributor

I am not familiar with the R code but I believe this change means we delegate the master check to spark-submit, where the precedence order is correct (--master > MASTER). If this is what this patch does then this LGTM, though @shivaram should verify.

@shivaram
Copy link
Contributor

shivaram commented Jun 3, 2015

Thanks @andrewor14 - LGTM. Merging this.

asfgit pushed a commit that referenced this pull request Jun 3, 2015
@shivaram
[SPARK-8063] [SPARKR] Spark master URL conflict between MASTER env va…
f67a27d 
…riable and --master command line option.

Author: Sun Rui <[email protected]>

Closes #6605 from sun-rui/SPARK-8063 and squashes the following commits:

51ca48b [Sun Rui] [SPARK-8063][SPARKR] Spark master URL conflict between MASTER env variable and --master command line option.

(cherry picked from commit 708c63b)
Signed-off-by: Shivaram Venkataraman <[email protected]>
@asfgit asfgit closed this in 708c63b Jun 3, 2015
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request Jun 12, 2015
@jeanlyn
[SPARK-8063] [SPARKR] Spark master URL conflict between MASTER env va…
c969913 
…riable and --master command line option.

Author: Sun Rui <[email protected]>

Closes apache#6605 from sun-rui/SPARK-8063 and squashes the following commits:

51ca48b [Sun Rui] [SPARK-8063][SPARKR] Spark master URL conflict between MASTER env variable and --master command line option.
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
@nemccarthy
[SPARK-8063] [SPARKR] Spark master URL conflict between MASTER env va…
e6b4688 
…riable and --master command line option.

Author: Sun Rui <[email protected]>

Closes apache#6605 from sun-rui/SPARK-8063 and squashes the following commits:

51ca48b [Sun Rui] [SPARK-8063][SPARKR] Spark master URL conflict between MASTER env variable and --master command line option.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sun-rui @shivaram @SparkQA @andrewor14