Apache spark 无法在Spark上导入Tensorflow
我无法在Pysark2中导入Tensorflow。我得到以下错误Apache spark 无法在Spark上导入Tensorflow,apache-spark,hadoop,tensorflow,bigdata,Apache Spark,Hadoop,Tensorflow,Bigdata,我无法在Pysark2中导入Tensorflow。我得到以下错误 Python 3.4.8 (default, Apr 9 2018, 11:43:18) [GCC 4.4.7 20120313 (Red Hat 4.4.7-18)] on linux Type "help", "copyright", "credits" or "license" for more information. Setting default log level to "WARN". To adjust log
Python 3.4.8 (default, Apr 9 2018, 11:43:18)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-18)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/__ / .__/\_,_/_/ /_/\_\ version 2.3.0.cloudera4
/_/
Using Python version 3.4.8 (default, Apr 9 2018 11:43:18)
SparkSession available as 'spark'.
>>> impo
>>> import tensorflow as tf
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/usr/lib64/python3.4/imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.17' not found (required by /usr/lib/python3.4/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.4/site-packages/tensorflow/__init__.py", line 24, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "/usr/lib/python3.4/site-packages/tensorflow/python/__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/usr/lib64/python3.4/imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.17' not found (required by /usr/lib/python3.4/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors
for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
Python 3.4.8(默认,2018年4月9日11:43:18)
linux上的[GCC 4.4.7 20120313(Red Hat 4.4.7-18)]
有关详细信息,请键入“帮助”、“版权”、“信用证”或“许可证”。
将默认日志级别设置为“警告”。
要调整日志记录级别,请使用sc.setLogLevel(newLevel)。对于SparkR,使用setLogLevel(newLevel)。
欢迎来到
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/__/.\uuu/\\ uuu/\\ u/\\版本2.3.0.cloudera4
/_/
使用Python版本3.4.8(默认值,2018年4月9日11:43:18)
SparkSession可用作“spark”。
>>>英普
>>>导入tensorflow作为tf
回溯(最近一次呼叫最后一次):
文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap_tensorflow.py”,第58行,在
从tensorflow.python.pywrap\u tensorflow\u内部导入*
文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap\u tensorflow\u internal.py”,第28行,在
_pywrap\u tensorflow\u internal=swig\u import\u helper()
swig\u import\u helper中的文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap\u tensorflow\u internal.py”,第24行
_mod=imp.load_模块(“pywrap_tensorflow_internal”,fp,路径名,描述)
加载模块中的文件“/usr/lib64/python3.4/imp.py”,第243行
返回加载动态(名称、文件名、文件)
导入错误:/usr/lib64/libstdc++.so.6:找不到版本'GLIBCXX_3.4.17'(由/usr/lib/python3.4/site packages/tensorflow/python/_pywrap_tensorflow_internal.so所需)
在处理上述异常期间,发生了另一个异常:
回溯(最近一次呼叫最后一次):
文件“”,第1行,在
文件“/usr/lib/python3.4/site packages/tensorflow/_init__.py”,第24行,在
从tensorflow.python导入pywrapu tensorflow 35; pylint:disable=未使用的导入
文件“/usr/lib/python3.4/site packages/tensorflow/python/_init__.py”,第49行,在
从tensorflow.python导入pywrap\u tensorflow
文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap_tensorflow.py”,第74行,在
提高效率(msg)
ImportError:回溯(最近一次呼叫上次):
文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap_tensorflow.py”,第58行,在
从tensorflow.python.pywrap\u tensorflow\u内部导入*
文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap\u tensorflow\u internal.py”,第28行,在
_pywrap\u tensorflow\u internal=swig\u import\u helper()
swig\u import\u helper中的文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap\u tensorflow\u internal.py”,第24行
_mod=imp.load_模块(“pywrap_tensorflow_internal”,fp,路径名,描述)
加载模块中的文件“/usr/lib64/python3.4/imp.py”,第243行
返回加载动态(名称、文件名、文件)
导入错误:/usr/lib64/libstdc++.so.6:找不到版本'GLIBCXX_3.4.17'(由/usr/lib/python3.4/site packages/tensorflow/python/_pywrap_tensorflow_internal.so所需)
未能加载本机TensorFlow运行时。
看见https://www.tensorflow.org/install/errors
因为一些常见的原因和解决方案。包括整个堆栈跟踪
以上是请求帮助时的错误消息。
环境详情:
pyspark2
命令运行pyspark2李>
可能重复:@DuyNguyenHoang我正在docker实例上运行TF。如果我的docker映像(centos)中的TF将引用底层(ubuntu 18.04)的c库,那么您建议的问题可能有效。可能重复:@DuyNguyenHoang我正在docker实例上运行TF。如果我的docker镜像(centos)中的TF是指底层(Ubuntu18.04)的c库,那么您建议的问题可能会起作用。