Python 如何使用线程使一个操作在继续之前等待另一个操作完成?
我正在一个100GB视频数据集上训练一个深度学习模型。我正在尝试将其全部转换为帧,然后在每个帧上应用Haar Cascade来裁剪面。但是我正在努力找到最快的方法 注意: 有5个褶皱,每个褶皱中有2个部分每个部分都有6个文件夹,其中包含3类不同视频格式的数据(0、5、10)(mp4、MOV、MOV)。总共有144个视频文件,每个文件大小约为700MB 编辑: 我遇到了这样一个问题,即视频到帧的处理不是等待其工作完成后再继续下一个文件 错误消息: 它会立即为每个文件抛出一个错误,而不是等待转换当前文件的帧,然后移动到下一个文件Python 如何使用线程使一个操作在继续之前等待另一个操作完成?,python,video,Python,Video,我正在一个100GB视频数据集上训练一个深度学习模型。我正在尝试将其全部转换为帧,然后在每个帧上应用Haar Cascade来裁剪面。但是我正在努力找到最快的方法 注意: 有5个褶皱,每个褶皱中有2个部分每个部分都有6个文件夹,其中包含3类不同视频格式的数据(0、5、10)(mp4、MOV、MOV)。总共有144个视频文件,每个文件大小约为700MB 编辑: 我遇到了这样一个问题,即视频到帧的处理不是等待其工作完成后再继续下一个文件 错误消息: 它会立即为每个文件抛出一个错误,而不是等待转换当
Error!!!
Reading from /Volumes/HDD/Data/Fold4_part2/44/5.mov
Category:5
Writing to /Volumes/HDD/Data/Fold4_part2/44
Number of frames: 7353
Converting video..
Error!!!
Reading from /Volumes/HDD/Data/Fold4_part2/45/0.mp4
Category:0
Writing to /Volumes/HDD/Data/Fold4_part2/45
Number of frames: 7716
Converting video..
代码:
import cv2
import time
import os
path_HDD = "/Volumes/HDD/Data"
def files(path):
"""
Function to get the files and add them to a list
Args:
path: path of the file
Not sure what is DS_Store but I do not need it
"""
for root, directories, files in os.walk(path, topdown=False):
for name in files:
file_path = os.path.join(root, name)
if (name == ".DS_Store"):
continue
else:
category = name.split(".")[0]
# Category returns the video category
try:
print("Reading from " + file_path)
print("Category:" + category)
print("Writing to " + root)
video_to_frames(file_path, category, root)
except:
print("Error!!!")
def video_to_frames(input_loc, label,output_loc):
"""Function to extract frames from input video file
and save them as separate frames in an output directory.
Args:
input_loc: Input video file.
output_loc: Output directory to save the frames.
Returns:
None
"""
# Log the time
time_start = time.time()
# Start capturing the feed
cap = cv2.VideoCapture(input_loc)
# Find the number of frames
video_length = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) - 1
print ("Number of frames: ", video_length)
count = 0
print ("Converting video..\n")
# Start converting the video
while cap.isOpened():
# Extract the frame
ret, frame = cap.read()
# Write the results back to output location.
cv2.imwrite(output_loc + "/" + label + "/%#05d.jpg" % (count+1), frame)
count = count + 1
# If there are no more frames left
if (count > (video_length-1)):
# Log the time again
time_end = time.time()
# Release the feed
cap.release()
# Print stats
print ("Done extracting frames.\n%d frames extracted" % count)
print ("It took %d seconds forconversion." % (time_end-time_start))
break
if __name__=="__main__":
files(path_HDD)
这个问题实际上是二合一的
第二个问题应该是可以实现的,只需掌握一点Python的一般知识,并知道如何使用获取文件路径。通过上面的答案,我们设法解决了这个问题,但可能需要6个小时来处理所有100GB。这不是一个真正的问题,但我可能会找到一种方法,使它更快地使用线程
import cv2
import time
import os
import argparse
import sys
path_HDD = "/Volumes/HDD/Data"
def files(path):
"""
Function to get the files and add them to a list
Args:
path: path of the file
Not sure what is DS_Store but I do not need it
"""
for root, directories, files in os.walk(path, topdown=False):
for name in files:
file_path = os.path.join(root, name)
if (name == ".DS_Store"):
continue
else:
category = name.split(".")[0]
# Category returns the video category
try:
print("Reading from " + file_path)
print("Category:" + category)
print("Writing to " + root)
video_to_frames(file_path, category, root)
except:
sys.exit("Unable to extract the frames from the video!")
"""
Root : /Volumes/HDD/Data/Fold1_part1/01/
Category : Label of the video
File path :
"""
def video_to_frames(input_loc, label,output_loc):
"""Function to extract frames from input video file
and save them as separate frames in an output directory.
Args:
input_loc: Input video file.
output_loc: Output directory to save the frames.
Returns:
None
"""
#Checking if the file exists before creating it
pathFrame = os.path.join(output_loc, label)
if (os.path.exists(pathFrame)):
pass
else:
try:
os.mkdir(pathFrame)
print("Directory '%s' created" %pathFrame)
except IOError:
print("Unable to create the new directory")
sys.exit()
# Log the time
time_start = time.time()
# Start capturing the feed
cap = cv2.VideoCapture(input_loc)
# Find the number of frames
video_length = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) - 1
print ("Number of frames: ", video_length)
count = 0
print ("Converting video..\n")
# Start converting the video
while cap.isOpened():
# Extract the frame
ret, frame = cap.read()
# Write the results back to output location.
cv2.imwrite(pathFrame + "/%#05d.jpg" % (count+1), frame)
count = count + 1
# If there are no more frames left
if (count > (video_length-1)):
# Log the time again
time_end = time.time()
# Release the feed
cap.release()
# Print stats
print ("Done extracting frames.\n%d frames extracted" % count)
print ("It took %d seconds forconversion." % (time_end-time_start))
break
if __name__=="__main__":
files(path_HDD)
好的,我照做了,但我遇到了一个问题。请检查新编辑。