Python Pypspark未打开jupyter
我尝试在pyspark(Ubuntu)中运行Graphframe,并遵循以下步骤: 我编辑了mu.profile文件,如下所示:Python Pypspark未打开jupyter,python,pyspark,jupyter-notebook,graphframes,Python,Pyspark,Jupyter Notebook,Graphframes,我尝试在pyspark(Ubuntu)中运行Graphframe,并遵循以下步骤: 我编辑了mu.profile文件,如下所示: SPARK_PATH=/home/spark/spark-2.4.4-bin-hadoop2.7 # set PATH so it includes user's private bin directories PATH="$HOME/bin:$HOME/.local/bin:$PATH" $SPARK_PATH/bin/pyspark --master local[
SPARK_PATH=/home/spark/spark-2.4.4-bin-hadoop2.7
# set PATH so it includes user's private bin directories
PATH="$HOME/bin:$HOME/.local/bin:$PATH"
$SPARK_PATH/bin/pyspark --master local[2]
export PYSPARK_DRIVER_PYTHON="jupyter"
然后我用下面的命令打开pyspark
pyspark --packages graphframes:graphframes:0.7.0-spark2.4-s_2.11
通过终端。但不是打开jupyter笔记本,而是打开终端中的spark会话。不确定哪一步没有通过jupyter打开
非常感谢您的帮助。为了让笔记本从pyspark启动,您必须使用两个环境变量
export PYSPARK_DRIVER_PYTHON=jupyter
export PYSPARK_DRIVER_PYTHON_OPTS=notebook