Python 模块';numpy';没有属性';gcd&x27;当试图在火花中加载numba时

Python 模块';numpy';没有属性';gcd&x27;当试图在火花中加载numba时,python,pyspark,numba,Python,Pyspark,Numba,我正在使用Spark,我想安装一些库,然后将它们用于进一步的步骤。我想在data proc cluster上安装这些库,但当我运行此代码时,由于模块'numpy'没有属性'gcd' 这是我的全部代码- import subprocess import sys print("###") print(sys.version) print("###") from time import time import os import os.path from os

我正在使用Spark,我想安装一些库,然后将它们用于进一步的步骤。我想在data proc cluster上安装这些库,但当我运行此代码时,由于模块'numpy'没有属性'gcd'

这是我的全部代码-

import subprocess
import sys
print("###")
print(sys.version)
print("###")
from time import time
import os
import os.path
from os import path
import pickle



# Installing the packages that don't come with the cluster
def installpip():
    subprocess.call([sys.executable, "-m", "easy_install", "pip"])
    
def install(package):
    subprocess.call([sys.executable, "-m", "pip", "install", package])
    
installpip()


os.system('pip install llvmlite --ignore-installed')
os.system('pip install nemo_toolkit[all]')


installation_list = ['umap-learn==0.5.0','pynndescent==0.5.0','sentence-transformers','nltk','numba==0.51.2']

os.system('pip install numpy')

print("installation started..")
for lib in installation_list:
  install(lib)
  print("library " + str(lib) + " installed.")
    
  
print("installation ended..")


print("downloading stop words")
os.system('python -m nltk.downloader stopwords')
print("done downloading stop words..")

print("verifying installations..")
import numba
print("numba library loaded..")
import pandas as pd
print("pandas library loaded..")
import numpy as np
print("numpy library loaded..")
from sklearn import metrics
from sklearn.cluster import KMeans
print("sklearn library loaded..")
from nltk.corpus import stopwords
print("nltk library loaded..")
from sentence_transformers import SentenceTransformer, util
print("sentence-transformers library loaded..")
import pickle
print("pickle library loaded..")
import umap
print("umap library loaded..")

print("verified all installations..")
这是堆栈跟踪-

###
3.6.13 |Anaconda, Inc.| (default, Feb 23 2021, 21:15:04) 
[GCC 7.3.0]
###

Building wheels for collected packages: sentence-transformers
  Building wheel for sentence-transformers (setup.py): started
  Building wheel for sentence-transformers (setup.py): finished with status 'done'
  Created wheel for sentence-transformers: filename=sentence_transformers-1.1.0-py3-none-any.whl size=119616 sha256=be99776275ec7098b7b77c506249281e69dc8951fd2e868063f29a7af3ea924f
  Stored in directory: /root/.cache/pip/wheels/6e/fc/93/fa5873037e04b951ffd3fa8b768677e3d943db1cb6609c37bb
Successfully built sentence-transformers
Installing collected packages: sentence-transformers
Successfully installed sentence-transformers-1.1.0
library sentence-transformers installed.
Requirement already satisfied: nltk in /opt/conda/anaconda/lib/python3.6/site-packages (3.3)
Requirement already satisfied: six in /opt/conda/anaconda/lib/python3.6/site-packages (from nltk) (1.15.0)
library nltk installed.
Collecting numba==0.51.2
  Downloading numba-0.51.2-cp36-cp36m-manylinux2014_x86_64.whl (3.1 MB)
Collecting llvmlite<0.35,>=0.34.0.dev0
  Downloading llvmlite-0.34.0-cp36-cp36m-manylinux2010_x86_64.whl (24.6 MB)
Requirement already satisfied: setuptools in /opt/conda/anaconda/lib/python3.6/site-packages (from numba==0.51.2) (52.0.0.post20210125)
Requirement already satisfied: numpy>=1.15 in /opt/conda/anaconda/lib/python3.6/site-packages (from numba==0.51.2) (1.19.5)
Installing collected packages: llvmlite, numba
  Attempting uninstall: llvmlite
    Found existing installation: llvmlite 0.36.0
    Uninstalling llvmlite-0.36.0:
      Successfully uninstalled llvmlite-0.36.0
  Attempting uninstall: numba
    Found existing installation: numba 0.53.1
    Uninstalling numba-0.53.1:
      Successfully uninstalled numba-0.53.1
Successfully installed llvmlite-0.34.0 numba-0.51.2
library numba==0.51.2 installed.
installation ended..
downloading stop words
/opt/conda/default/lib/python3.6/runpy.py:125: RuntimeWarning: 'nltk.downloader' found in sys.modules after import of package 'nltk', but prior to execution of 'nltk.downloader'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
[nltk_data] Downloading package stopwords to /root/nltk_data...
[nltk_data]   Unzipping corpora/stopwords.zip.
done downloading stop words..
verifying installations..
Traceback (most recent call last):
  File "/tmp/install_te-spark_proc-30937144-5790346e/jobs/30937144-401f-4fd0-92f4-3e41c11953c4/PySparkProcessCode.py", line 367, in <module>
    import numba
  File "/opt/conda/default/lib/python3.6/site-packages/numba/__init__.py", line 34, in <module>
    from numba.core.decorators import (cfunc, generated_jit, jit, njit, stencil,
  File "/opt/conda/default/lib/python3.6/site-packages/numba/core/decorators.py", line 12, in <module>
    from numba.stencils.stencil import stencil
  File "/opt/conda/default/lib/python3.6/site-packages/numba/stencils/stencil.py", line 11, in <module>
    from numba.core import types, typing, utils, ir, config, ir_utils, registry
  File "/opt/conda/default/lib/python3.6/site-packages/numba/core/registry.py", line 4, in <module>
    from numba.core import utils, typing, dispatcher, cpu
  File "/opt/conda/default/lib/python3.6/site-packages/numba/core/dispatcher.py", line 15, in <module>
    from numba.core import utils, types, errors, typing, serialize, config, compiler, sigutils
  File "/opt/conda/default/lib/python3.6/site-packages/numba/core/compiler.py", line 6, in <module>
    from numba.core import (utils, errors, typing, interpreter, bytecode, postproc,
  File "/opt/conda/default/lib/python3.6/site-packages/numba/core/callconv.py", line 11, in <module>
    from numba.core.base import PYOBJECT, GENERIC_POINTER
  File "/opt/conda/default/lib/python3.6/site-packages/numba/core/base.py", line 23, in <module>
    from numba.cpython import builtins
  File "/opt/conda/default/lib/python3.6/site-packages/numba/cpython/builtins.py", line 490, in <module>
    from numba.core.typing.builtins import IndexValue, IndexValueType
  File "/opt/conda/default/lib/python3.6/site-packages/numba/core/typing/builtins.py", line 8, in <module>
    from numba.parfors.parfor import internal_prange
  File "/opt/conda/default/lib/python3.6/site-packages/numba/parfors/__init__.py", line 1, in <module>
    from numba.parfors import parfor_lowering
  File "/opt/conda/default/lib/python3.6/site-packages/numba/parfors/parfor_lowering.py", line 18, in <module>
    from numba.parfors import parfor
  File "/opt/conda/default/lib/python3.6/site-packages/numba/parfors/parfor.py", line 83, in <module>
    from numba.core.typing import npydecl, signature
  File "/opt/conda/default/lib/python3.6/site-packages/numba/core/typing/npydecl.py", line 313, in <module>
    _numpy_ufunc(func)
  File "/opt/conda/default/lib/python3.6/site-packages/numba/core/typing/npydecl.py", line 297, in _numpy_ufunc
    func = getattr(np, name)
AttributeError: module 'numpy' has no attribute 'gcd'
###
3.6.13 | Anaconda,Inc.|(默认,2021年2月23日,21:15:04)
[GCC 7.3.0]
###
为收集的包裹构建轮子:句子变压器
为句子转换器构建控制盘(setup.py):已启动
为句子转换器构建控制盘(setup.py):完成状态为“完成”
为句子转换器创建的控制盘:文件名=句子转换器-1.1.0-py3-none-any.whl大小=119616 sha256=be99776275ec7098b7b77c506249281e69dc8951fd2e868063f29a7af3ea924f
存储在目录:/root/.cache/pip/wheels/6e/fc/93/fa5873037e04b951ffd3fa8b768677e3d943db1cb6609c37bb中
成功构建句子转换器
安装收集的软件包:句子转换器
已成功安装SENTURE-transformers-1.1.0
图书馆句子变压器已安装。
已满足要求:nltk in/opt/conda/anaconda/lib/python3.6/site-packages(3.3)
已满足要求:六个in/opt/conda/anaconda/lib/python3.6/site-packages(来自nltk)(1.15.0)
已安装库nltk。
收集numba==0.51.2
下载numba-0.51.2-cp36-cp36m-manylinux2014_x86_64.whl(3.1MB)
收集llvmlite=0.34.0.dev0
下载llvmlite-0.34.0-cp36-cp36m-manylinux2010_x86_64.whl(24.6MB)
已满足要求:setuptools in/opt/conda/anaconda/lib/python3.6/site-packages(from numba==0.51.2)(52.0.0.post20210125)
已满足要求:numpy>=1.15 in/opt/conda/anaconda/lib/python3.6/site-packages(来自numba==0.51.2)(1.19.5)
安装收集的软件包:llvmlite、numba
正在尝试卸载:llvmlite
找到现有安装:llvmlite 0.36.0
正在卸载llvmlite-0.36.0:
已成功卸载llvmlite-0.36.0
正在尝试卸载:numba
找到现有安装:numba 0.53.1
卸载numba-0.53.1:
已成功卸载numba-0.53.1
已成功安装llvmlite-0.34.0 numba-0.51.2
库numba==0.51.2已安装。
安装结束。。
下载停止词
/opt/conda/default/lib/python3.6/runpy.py:125:RuntimeWarning:在导入包“nltk”之后,但在执行“nltk.downloader”之前,在sys.modules中发现“nltk.downloader”;这可能导致不可预测的行为
警告(运行时警告(msg))
[nltk_数据]正在将包停止字下载到/root/nltk_数据。。。
[nltk_数据]解压语料库/stopwords.zip。
完成下载停止词。。
正在验证安装。。
回溯(最近一次呼叫最后一次):
文件“/tmp/install_te-spark_proc-30937144-5790346e/jobs/30937144-401f-4fd0-92f4-3e41c11953c4/PySparkProcessCode.py”,第367行
进口麻木
文件“/opt/conda/default/lib/python3.6/site-packages/numba/__-init___.py”,第34行,在
从numba.core.decorators导入(cfunc、生成的_jit、jit、njit、模具、,
文件“/opt/conda/default/lib/python3.6/site packages/numba/core/decorators.py”,第12行,在
从numba.stencils.stencil导入模具
文件“/opt/conda/default/lib/python3.6/site packages/numba/stencils/stencil.py”,第11行,在
从numba.core导入类型、类型、utils、ir、config、ir_utils、注册表
文件“/opt/conda/default/lib/python3.6/site packages/numba/core/registry.py”,第4行,在
从numba.core导入utils、键入、调度程序、cpu
文件“/opt/conda/default/lib/python3.6/site packages/numba/core/dispatcher.py”,第15行,在
从numba.core导入utils、类型、错误、类型、序列化、配置、编译器、sigutils
文件“/opt/conda/default/lib/python3.6/site packages/numba/core/compiler.py”,第6行,在
从numba.core导入(UTIL、错误、键入、解释器、字节码、后处理程序、,
文件“/opt/conda/default/lib/python3.6/site packages/numba/core/callconv.py”,第11行,在
从numba.core.base导入PYOBJECT,通用指针
文件“/opt/conda/default/lib/python3.6/site packages/numba/core/base.py”,第23行,在
从numba.cpython导入内置
文件“/opt/conda/default/lib/python3.6/site packages/numba/cpython/builtins.py”,第490行,在
从numba.core.typing.builtins导入IndexValue,IndexValueType
文件“/opt/conda/default/lib/python3.6/site packages/numba/core/typing/builtins.py”,第8行,在
从numba.parfors.parfor导入内部\u prange
文件“/opt/conda/default/lib/python3.6/site packages/numba/parfors/__init__.py”,第1行,在
从numba.parfors进口Parforu
文件“/opt/conda/default/lib/python3.6/site packages/numba/parfors/parfor_.py”,中第18行
从numba.parfors进口parfor
文件“/opt/conda/default/lib/python3.6/site packages/numba/parfors/parfor.py”,第83行,在
从numba.core.type导入npydecl,签名
文件“/opt/conda/default/lib/python3.6/site packages/numba/core/typing/npydecl.py”,第313行,在
_numpy_ufunc(func)
文件“/opt/conda/default/lib/python3.6/site packages/numba/core/typing/npydecl.py”,第297行,在
func=getattr(np,名称)
AttributeError:模块“numpy”没有属性“gcd”
不确定什么是
错误:pip的依赖项解析程序当前未考虑已安装的所有软件包。此行为是以下依赖项冲突的根源。distributed 1.21.8需要msgpack,但未安装。
这意味着