Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/hadoop/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark 无法在Spark上导入Tensorflow_Apache Spark_Hadoop_Tensorflow_Bigdata - Fatal编程技术网

Apache spark 无法在Spark上导入Tensorflow

Apache spark 无法在Spark上导入Tensorflow,apache-spark,hadoop,tensorflow,bigdata,Apache Spark,Hadoop,Tensorflow,Bigdata,我无法在Pysark2中导入Tensorflow。我得到以下错误 Python 3.4.8 (default, Apr 9 2018, 11:43:18) [GCC 4.4.7 20120313 (Red Hat 4.4.7-18)] on linux Type "help", "copyright", "credits" or "license" for more information. Setting default log level to "WARN". To adjust log

我无法在Pysark2中导入Tensorflow。我得到以下错误

Python 3.4.8 (default, Apr  9 2018, 11:43:18) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-18)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /__ / .__/\_,_/_/ /_/\_\   version 2.3.0.cloudera4
      /_/

Using Python version 3.4.8 (default, Apr  9 2018 11:43:18)
SparkSession available as 'spark'.
>>> impo
>>> import tensorflow as tf
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/lib64/python3.4/imp.py", line 243, in load_module
    return load_dynamic(name, filename, file)
ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.17' not found (required by /usr/lib/python3.4/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/site-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/usr/lib/python3.4/site-packages/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/usr/lib/python3.4/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/lib64/python3.4/imp.py", line 243, in load_module
    return load_dynamic(name, filename, file)
ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.17' not found (required by /usr/lib/python3.4/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)


Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/errors

for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.
Python 3.4.8(默认,2018年4月9日11:43:18)
linux上的[GCC 4.4.7 20120313(Red Hat 4.4.7-18)]
有关详细信息,请键入“帮助”、“版权”、“信用证”或“许可证”。
将默认日志级别设置为“警告”。
要调整日志记录级别,请使用sc.setLogLevel(newLevel)。对于SparkR,使用setLogLevel(newLevel)。
欢迎来到
____              __
/ __/__  ___ _____/ /__
_\ \/ _ \/ _ `/ __/  '_/
/__/.\uuu/\\ uuu/\\ u/\\版本2.3.0.cloudera4
/_/
使用Python版本3.4.8(默认值,2018年4月9日11:43:18)
SparkSession可用作“spark”。
>>>英普
>>>导入tensorflow作为tf
回溯(最近一次呼叫最后一次):
文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap_tensorflow.py”,第58行,在
从tensorflow.python.pywrap\u tensorflow\u内部导入*
文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap\u tensorflow\u internal.py”,第28行,在
_pywrap\u tensorflow\u internal=swig\u import\u helper()
swig\u import\u helper中的文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap\u tensorflow\u internal.py”,第24行
_mod=imp.load_模块(“pywrap_tensorflow_internal”,fp,路径名,描述)
加载模块中的文件“/usr/lib64/python3.4/imp.py”,第243行
返回加载动态(名称、文件名、文件)
导入错误:/usr/lib64/libstdc++.so.6:找不到版本'GLIBCXX_3.4.17'(由/usr/lib/python3.4/site packages/tensorflow/python/_pywrap_tensorflow_internal.so所需)
在处理上述异常期间,发生了另一个异常:
回溯(最近一次呼叫最后一次):
文件“”,第1行,在
文件“/usr/lib/python3.4/site packages/tensorflow/_init__.py”,第24行,在
从tensorflow.python导入pywrapu tensorflow 35; pylint:disable=未使用的导入
文件“/usr/lib/python3.4/site packages/tensorflow/python/_init__.py”,第49行,在
从tensorflow.python导入pywrap\u tensorflow
文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap_tensorflow.py”,第74行,在
提高效率(msg)
ImportError:回溯(最近一次呼叫上次):
文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap_tensorflow.py”,第58行,在
从tensorflow.python.pywrap\u tensorflow\u内部导入*
文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap\u tensorflow\u internal.py”,第28行,在
_pywrap\u tensorflow\u internal=swig\u import\u helper()
swig\u import\u helper中的文件“/usr/lib/python3.4/site packages/tensorflow/python/pywrap\u tensorflow\u internal.py”,第24行
_mod=imp.load_模块(“pywrap_tensorflow_internal”,fp,路径名,描述)
加载模块中的文件“/usr/lib64/python3.4/imp.py”,第243行
返回加载动态(名称、文件名、文件)
导入错误:/usr/lib64/libstdc++.so.6:找不到版本'GLIBCXX_3.4.17'(由/usr/lib/python3.4/site packages/tensorflow/python/_pywrap_tensorflow_internal.so所需)
未能加载本机TensorFlow运行时。
看见https://www.tensorflow.org/install/errors
因为一些常见的原因和解决方案。包括整个堆栈跟踪
以上是请求帮助时的错误消息。
环境详情:

  • 我正在Linux 18.04上运行Cloudera的quickstart docker映像
  • 我已在docker图像中将pyspark升级为pyspark2。Currnet Spark版本为2.3.0
  • 已在docker映像上安装python3
  • 使用sudo pip3安装Tensorflow安装Tensorflow
  • 使用
    pyspark2
    命令运行pyspark2
    可能重复:@DuyNguyenHoang我正在docker实例上运行TF。如果我的docker映像(centos)中的TF将引用底层(ubuntu 18.04)的c库,那么您建议的问题可能有效。可能重复:@DuyNguyenHoang我正在docker实例上运行TF。如果我的docker镜像(centos)中的TF是指底层(Ubuntu18.04)的c库,那么您建议的问题可能会起作用。