如何保存numpy阵列图像并将其放入单个文件夹?
我有一个numpy数组,包含5000个28乘28个图像(5000,28,28)。我想将所有这些图像保存为jpg文件,并将它们全部保存在一个文件夹中。实现这一目标的最快和最有效的方法是什么 我尝试使用以下方法将50000张28x28的图像以JPEG格式写入磁盘:如何保存numpy阵列图像并将其放入单个文件夹?,numpy,matplotlib,scipy,python-imaging-library,Numpy,Matplotlib,Scipy,Python Imaging Library,我有一个numpy数组,包含5000个28乘28个图像(5000,28,28)。我想将所有这些图像保存为jpg文件,并将它们全部保存在一个文件夹中。实现这一目标的最快和最有效的方法是什么 我尝试使用以下方法将50000张28x28的图像以JPEG格式写入磁盘: 顺序代码(25秒) 多线程代码(19秒) 多处理代码(5秒) 在12核MacBook Pro上安装SSD。时间在上面列表中每个项目后面的括号中给出 #!/usr/bin/env python3 import numpy as np
- 顺序代码(25秒)
- 多线程代码(19秒)
- 多处理代码(5秒)
#!/usr/bin/env python3
import numpy as np
import imageio
from multiprocessing import Pool
from multiprocessing.pool import ThreadPool
def WriteOne(i,data):
"""Write a single image to disk"""
#print(f"WriteOne called with i={i}")
imageio.imwrite(f"image-{i:04d}.jpg", data)
def WriteSequential(images):
"""Write all images sequentially to disk"""
print("Sequential")
for i, data in enumerate(images):
WriteOne(i, data)
return
def WriteIndices(p):
"""Write given index to disk as JPEG"""
#print(f"WriteIndices {p[0]}")
WriteOne(p[0], p[1])
return
def WriteMultiThread(images):
"""Write all images to disk with multi-threading"""
print("MultiThread")
nThreads = 8
with ThreadPool(nThreads) as pool:
pool.map(WriteIndices, list(enumerate(images)))
return
def WriteMultiProcess(images):
"""Write all images to disk with multi-processing"""
print("MultiProcess")
nProcesses = 8
with Pool(nProcesses) as pool:
pool.map(WriteIndices, list(enumerate(images)))
return
if __name__ == '__main__':
# Synthesize 50000 images, each 28x28
N = 50000
images = np.random.randint(0,256,(N,28,28), dtype=np.uint8)
WriteSequential(images)
WriteMultiThread(images)
WriteMultiProcess(images)
我尝试使用以下方法将50000张28x28的图像以JPEG格式写入磁盘:
- 顺序代码(25秒)
- 多线程代码(19秒)
- 多处理代码(5秒)
#!/usr/bin/env python3
import numpy as np
import imageio
from multiprocessing import Pool
from multiprocessing.pool import ThreadPool
def WriteOne(i,data):
"""Write a single image to disk"""
#print(f"WriteOne called with i={i}")
imageio.imwrite(f"image-{i:04d}.jpg", data)
def WriteSequential(images):
"""Write all images sequentially to disk"""
print("Sequential")
for i, data in enumerate(images):
WriteOne(i, data)
return
def WriteIndices(p):
"""Write given index to disk as JPEG"""
#print(f"WriteIndices {p[0]}")
WriteOne(p[0], p[1])
return
def WriteMultiThread(images):
"""Write all images to disk with multi-threading"""
print("MultiThread")
nThreads = 8
with ThreadPool(nThreads) as pool:
pool.map(WriteIndices, list(enumerate(images)))
return
def WriteMultiProcess(images):
"""Write all images to disk with multi-processing"""
print("MultiProcess")
nProcesses = 8
with Pool(nProcesses) as pool:
pool.map(WriteIndices, list(enumerate(images)))
return
if __name__ == '__main__':
# Synthesize 50000 images, each 28x28
N = 50000
images = np.random.randint(0,256,(N,28,28), dtype=np.uint8)
WriteSequential(images)
WriteMultiThread(images)
WriteMultiProcess(images)
这取决于你在哪种机器上运行。对于单核Raspberry Pi Zero和带有SSD磁盘的8核台式机来说,答案大不相同。这取决于您运行的机器类型。对于单核Raspberry Pi Zero和带有SSD磁盘的8核桌面,答案截然不同。