Python ModuleNotFoundError:没有名为';图形框架';
我想用Python ModuleNotFoundError:没有名为';图形框架';,python,apache-spark,pyspark,graphframes,Python,Apache Spark,Pyspark,Graphframes,我想用pyspark运行graphframes 我发现了这个,并按照它的指示,但它不工作 这是我的代码你好\u spark.py: File "<frozen zipimport>", line 259, in load_module File "/opt/spark/python/lib/pyspark.zip/pyspark/ml/param/__init__.py", line 26, in <module> Module
pyspark
运行graphframes
我发现了这个,并按照它的指示,但它不工作
这是我的代码你好\u spark.py
:
File "<frozen zipimport>", line 259, in load_module
File "/opt/spark/python/lib/pyspark.zip/pyspark/ml/param/__init__.py", line 26, in <module>
ModuleNotFoundError: No module named 'numpy'
导入pyspark
conf=pyspark.SparkConf().set(“spark.driver.host”,“127.0.0.1”)
sc=pyspark.SparkContext(master=“local”,appName=“myAppName”,conf=conf)
sc.addPyFile(“/opt/spark/jars/spark-graphx_2.12-3.0.2.jar”)
从graphframes导入*
使用此命令运行时:
spark-submit hello_spark.py
pyspark --packages graphframes:graphframes:0.8.1-spark3.0-s_2.12
它返回以下错误:
from graphframes import *
ModuleNotFoundError: No module named 'graphframes'
这是我的.bashrc
配置:
# For Spark setup
export SPARK_HOME=/opt/spark
export PATH=$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin
export PYSPARK_PYTHON=/usr/bin/python3
export SPARK_LOCAL_IP=localhost
export SPARK_OPTS="--packages graphframes:graphframes:0.8.1-spark3.0-s_2.12"
我的spark:3.0.2
,scala:2.12.10
我使用以下命令安装了图形框架
:
spark-submit hello_spark.py
pyspark --packages graphframes:graphframes:0.8.1-spark3.0-s_2.12
有人知道如何解决这个问题吗?谢谢。我发现如果我使用这个命令,它会工作:
spark-submit hello_spark.py --packages graphframes:graphframes:0.8.1-spark3.0-s_2.12
您应该注意到,您必须为pyspark
安装一些依赖项,如numpy
:
File "<frozen zipimport>", line 259, in load_module
File "/opt/spark/python/lib/pyspark.zip/pyspark/ml/param/__init__.py", line 26, in <module>
ModuleNotFoundError: No module named 'numpy'
您可以通过激活环境路径找到它,然后运行哪个命令:
(基本)username@user:~$conda激活pyenv
(pyenv)username@user:~$whichpython
/home/username/miniconda3/envs/pyenv/bin/python