如何使用python将新数据附加到pickle文件
我正在提取图像的人脸嵌入并将其附加到现有的pickle文件中。但看起来它不起作用,因为当我解开文件时,它不包含添加的新数据。代码如下:如何使用python将新数据附加到pickle文件,python,file,pickle,Python,File,Pickle,我正在提取图像的人脸嵌入并将其附加到现有的pickle文件中。但看起来它不起作用,因为当我解开文件时,它不包含添加的新数据。代码如下: file = client_dir + '\embeddings.pickle' data = {"embeddings": known_embeddings, "names": known_names} with open(file, 'ab+') as fp: pickle.dump(data, fp) fp.close() log("[IN
file = client_dir + '\embeddings.pickle'
data = {"embeddings": known_embeddings, "names": known_names}
with open(file, 'ab+') as fp:
pickle.dump(data, fp)
fp.close()
log("[INFO] Data appended to embeddings.pickle ")
当前pickle文件包含以下数据:
{'embeddings': [array([-0.03656099, 0.11354745, -0.00438912, 0.0367547 , 0.06391761,
0.18440282, 0.06150107, -0.17380905, 0.03094344, -0.00182147,
0.00969766, 0.06890091, 0.04974053, -0.0502388 , -0.03414046,
-0.13550822, -0.02251128, 0.14556041, -0.04045469, 0.06500552,
0.0726142 , -0.04139924, -0.04662199, 0.08869533, -0.00061307,
-0.11912274, 0.13141112, -0.00648551, 0.00296356, 0.03682912,
-0.15076959, 0.03989822, 0.02799555, 0.03429572, 0.09865954,
0.14113557, -0.08355764, 0.09193961, -0.00819231, -0.01184336,
-0.12519744, 0.00668721, 0.0816237 , 0.00464355, -0.00339399,
0.07501812, 0.11679655, -0.09211859, 0.06211261, -0.00543289,
0.10347278, 0.06651585, -0.01512023, 0.09477805, 0.09886038,
-0.03837246, 0.02265131, -0.14867221, 0.00781244, 0.04845129,
-0.0363168 , -0.00186919, -0.16163988, 0.09539618, 0.14983718,
0.09159472, -0.05315595, -0.05073383, 0.01501674, -0.03789762,
0.07116041, 0.07650694, -0.02975985], dtype=float32)], 'names': ['rock']}
我尝试附加的新数据如下:
{'embeddings': [array([-0.03656099, 0.11354745, -0.00438912, 0.0367547 , 0.06391761,
0.18440282, 0.06150107, -0.17380905, 0.03094344, -0.00182147,
0.00969766, 0.06890091, 0.04974053, -0.0502388 , -0.03414046,
0.07501812, 0.11679655, -0.09211859, 0.06211261, -0.00543289,
-0.13550822, -0.02251128, 0.14556041, -0.04045469, 0.06500552,
0.0726142 , -0.04139924, -0.04662199, 0.08869533, -0.00061307,
-0.11912274, 0.13141112, -0.00648551, 0.00296356, 0.03682912,
-0.15076959, 0.03989822, 0.02799555, 0.03429572, 0.09865954,
0.14113557, -0.08355764, 0.09193961, -0.00819231, -0.01184336,
-0.12519744, 0.00668721, 0.0816237 , 0.00464355, -0.00339399,
0.10347278, 0.06651585, -0.01512023, 0.09477805, 0.09886038,
-0.03837246, 0.02265131, -0.14867221, 0.00781244, 0.04845129,
-0.0363168 , -0.00186919, -0.16163988, 0.09539618, 0.14983718,
0.09159472, -0.05315595, -0.05073383, 0.01501674, -0.03789762,
0.07116041, 0.07650694, -0.02975985], dtype=float32)], 'names': ['john']}
但当我解开文件时,它只有岩石的数据,而没有约翰的数据。谁能帮我一下我做错了什么。下面是我用来解压和观察添加了什么数据的代码。可能我取消勾选文件的方式是错误的,因为当我追加数据时,我可以看到文件大小增加
import pickle
file = open('G:\\output\\embeddings.pickle', 'rb')
data = pickle.load(file)
file.close()
print(data)
请帮忙。谢谢
更新代码:
file_path = client_dir + '\embeddings.pickle'
file = open(file_path, 'rb')
old_data = pickle.load(file)
new_embeddings = old_data['embeddings']
new_names = old_data['names']
new_embeddings.append(known_embeddings[0])
new_names.append(known_names[0])
data1 = {"embeddings": new_embeddings, "names": new_names}
with open(file_path, 'ab+') as fp:
pickle.dump(data1, fp)
fp.close()
log.error("[INFO] Data appended to embeddings.pickle ")
在上面的代码中,我首先将数据从pickle文件加载到列表中,然后将新数据追加到列表中,然后将所有新旧数据添加到pickle文件中。有人能告诉我这是不是正确的方法吗
在这之后,当我解开文件时,我并没有得到所有的数据。谢谢
file_path = client_dir + '\embeddings.pickle'
file = open(file_path, 'rb')
old_data = pickle.load(file)
new_embeddings = old_data['embeddings']
new_names = old_data['names']
new_embeddings.append(known_embeddings[0])
new_names.append(known_names[0])
data1 = {"embeddings": new_embeddings, "names": new_names}
with open(file_path, 'ab+') as fp:
pickle.dump(data1, fp)
fp.close()
log.error("[INFO] Data appended to embeddings.pickle ")
我觉得这很接近正确。您成功地加载了pickle数据并向其中添加了新元素。问题似乎在于openfile_路径“ab+”作为fp:call。如果以模式打开文件,则写入的pickle数据将添加到末尾,位于旧pickle数据之后。然后,在后续执行程序时,pickle.load将只加载旧的pickle数据
尝试用新的pickle数据完全覆盖旧的pickle数据。您可以改为在w模式下打开
with open(file_path, 'wb') as fp:
pickle.dump(data1, fp)
顺便说一句,你不需要fp.close通话。with语句会自动关闭块末尾打开的文件。无需先加载数据即可完成此操作,以提高速度:
如果文件不存在,请使用mode='ab'创建新文件,如果文件存在,请附加数据:
pickle.dump((data), open('data folder/' + filename2save + '.pkl', 'ab'))
如果在同一个文件上多次调用dump,我认为pickle协议不能保证工作。不要先酸洗一个dict,然后再酸洗另一个dict,而是尝试酸洗一个包含这两个dict的列表。@Kevin您的意思是首先在list/dict中加载所有pickle数据,然后将新数据附加到list/dict中,然后将其保存在pickle文件中。?我认为这是可行的。我试试看it@Kevin我已经更新了代码,你能检查一下吗。