Skip to content

Commit b395ed1

Browse files
committed
Fix weirdness re: how scala SparkConf.get(x) gets called from python SparkConf.get(x)
1 parent a4f2c72 commit b395ed1

File tree

1 file changed

+4
-3
lines changed

1 file changed

+4
-3
lines changed

python/pyspark/context.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -221,9 +221,10 @@ def _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize,
221221
'MODULE$')
222222

223223
packagesSeq = self._conf.get(internal_config.CONDA_BOOTSTRAP_PACKAGES())
224-
packagesList = self._jvm.scala.collection.JavaConversions.seqAsJavaList(packagesSeq)
225-
# Add the bootstrap packages to the list to be installed on executors
226-
self._conda_packages += (pkg for pkg in packagesList)
224+
if packagesSeq is not None:
225+
packagesList = self._jvm.scala.collection.JavaConversions.seqAsJavaList(packagesSeq)
226+
# Add the bootstrap packages to the list to be installed on executors
227+
self._conda_packages += (pkg for pkg in packagesList)
227228

228229
# Deploy code dependencies set by spark-submit; these will already have been added
229230
# with SparkContext.addFile, so we just need to add them to the PYTHONPATH

0 commit comments

Comments
 (0)