Macos 无法为pyspark OSX启动Jupyter笔记本:Spark 2.0中删除了IPYTHON和IPYTHON_选项+
我已经下载了:spark-2.2.0-bin-hadoop2.7 在我的~/.bash_档案中,我有以下内容:Macos 无法为pyspark OSX启动Jupyter笔记本:Spark 2.0中删除了IPYTHON和IPYTHON_选项+,macos,pyspark,jupyter-notebook,Macos,Pyspark,Jupyter Notebook,我已经下载了:spark-2.2.0-bin-hadoop2.7 在我的~/.bash_档案中,我有以下内容: export PATH="/Users/spandan.chakraborty/anaconda/bin:$PATH" export SPARK_PATH=~/spark-2.2.0-bin-hadoop2.7 export PYSPARK_DRIVER_PYTHON='jupyter' export PYSPARK_DRIVER_PYTHON_OPTS='notebook'
export PATH="/Users/spandan.chakraborty/anaconda/bin:$PATH"
export SPARK_PATH=~/spark-2.2.0-bin-hadoop2.7
export PYSPARK_DRIVER_PYTHON='jupyter'
export PYSPARK_DRIVER_PYTHON_OPTS='notebook'
alias snote='$SPARK_PATH/bin/pyspark --master local[2]'
但当我尝试使用命令别名:snote启动Jupyter笔记本时,我遇到以下错误:
pyspark启动时出错:
IPYTHON和IPYTHON_选项在Spark 2.0+中被删除。把这些从
环境和设置PYSPARK_驱动程序_PYTHON和
PYSPARK_驱动程序_PYTHON_选择
我已经提到类似的问题,张贴在
但是没有用。以下内容解决了问题:
export PYSPARK_DRIVER_PYTHON=jupyter
export PYSPARK_DRIVER_PYTHON_OPTS='notebook'
这对我很有效,我在MAC上运行 PYSPARK_DRIVER_PYTHON=ipython PYSPAK_DRIVER_PYTHON_OPTS=notebook/Users/abhishekdutta/Downloads/Spark/Spark-2.2.0-bin-hadoop2.7/bin/PYSPARK