Python RuntimeError:预期所有张量都在同一个设备上,但至少找到两个设备,cuda:0和cpu
尝试在asyncio包中运行pytorch余弦相似性以获得并行结果。使用flair模型嵌入文本。需要将一个文本与一个巨大的数据帧进行比较,得到最相似的文本,结果和响应应该非常快。你能建议一个替代的方法吗?我还需要在CPU内存上运行此代码,而不是在GPU Cuda系统上 错误:Python RuntimeError:预期所有张量都在同一个设备上,但至少找到两个设备,cuda:0和cpu,python,pytorch,cosine-similarity,flair,Python,Pytorch,Cosine Similarity,Flair,尝试在asyncio包中运行pytorch余弦相似性以获得并行结果。使用flair模型嵌入文本。需要将一个文本与一个巨大的数据帧进行比较,得到最相似的文本,结果和响应应该非常快。你能建议一个替代的方法吗?我还需要在CPU内存上运行此代码,而不是在GPU Cuda系统上 错误: import asyncio import torch import os import pandas as pd from flair.data import Sentence from flair.embedding
import asyncio
import torch
import os
import pandas as pd
from flair.data import Sentence
from flair.embeddings import FlairEmbeddings, DocumentPoolEmbeddings, WordEmbeddings
device = torch.device("cpu")
print(device)
# first, declare how you want to embed
embeddings = DocumentPoolEmbeddings(
[WordEmbeddings('glove'), FlairEmbeddings('news-forward'), FlairEmbeddings('news-backward')])
path = os.getcwd()
df=pd.read_pickle(path+'/embedding_all_courses_2.pkl')
query_emd=[]
cos = torch.nn.CosineSimilarity(dim=0, eps=1e-6)
query= Sentence("some text")
embeddings.embed([query])
query_emd.append(query.embedding)
async def count(index,row):
for i in query_emd:
print(words,row['course_name'],cos(i, row['embedding']))
print(index)
async def main():
await asyncio.gather(*(count(index,row) for index,row in df.iterrows()))
if __name__ == "__main__":
import time
s = time.perf_counter()
asyncio.run(main())
elapsed = time.perf_counter() - s
print(f"{__file__} executed in {elapsed:0.2f} seconds.")
raceback(最近一次通话最后一次):
文件“asyn_emd.py”,第74行,在
asyncio.run(main())
文件“/home/linuxbrew/.linuxbrew/ceral/python/3.7.5/lib/python3.7/asyncio/runners.py”,第43行,运行中
返回循环。运行直到完成(主)
文件“/home/linuxbrew/.linuxbrew/cillar/python/3.7.5/lib/python3.7/asyncio/base_events.py”,第579行,运行直到完成
返回future.result()
文件“/asyn_emd.py”,第68行,主
等待asyncio.gather(*(df.iterrows()中索引、行的计数(索引、行))
文件“/asyn_emd.py”,第61行,计数
打印(文字,第['course_name']行,cos(i,第['Embedded']行)
文件“/home/karthickaravindan/.virtualenvs/test/lib/python3.7/site-packages/torch/nn/modules/module.py”,第722行,在
结果=自我转发(*输入,**kwargs)
文件“/home/karthickaravindan/.virtualenvs/test/lib/python3.7/site packages/torch/nn/modules/distance.py”,第75行,向前
返回F.cosine_相似性(x1,x2,self.dim,self.eps)
RuntimeError:预期所有张量都在同一个设备上,但至少找到两个设备,cuda:0和cpu!
确保在cos(i,行[“嵌入”])
i和行[“嵌入”]都在CPU上。我不知道这些变量是什么,但我认为其中一个变量在gpu上。因此,将.to(device=“cpu”)
添加到i和行[“嵌入”]@TheodorPeifer谢谢。成功了。打印(单词,行['course\u name'],cos(i.to(device=“cpu”),行['embedding'].to(device=“cpu”))
raceback (most recent call last):
File "asyn_emd.py", line 74, in <module>
asyncio.run(main())
File "/home/linuxbrew/.linuxbrew/Cellar/python/3.7.5/lib/python3.7/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/home/linuxbrew/.linuxbrew/Cellar/python/3.7.5/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete
return future.result()
File "/asyn_emd.py", line 68, in main
await asyncio.gather(*(count(index,row) for index,row in df.iterrows()))
File "/asyn_emd.py", line 61, in count
print(words,row['course_name'],cos(i, row['embedding']))
File "/home/karthickaravindan/.virtualenvs/test/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/karthickaravindan/.virtualenvs/test/lib/python3.7/site-packages/torch/nn/modules/distance.py", line 75, in forward
return F.cosine_similarity(x1, x2, self.dim, self.eps)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!