Python 如何使用立体相机创建好的深度贴图?
我在一个项目上学习了很长时间。我的目标是从立体相机的图像中获取深度图,并仅过滤人体,以便计算人体内部的数量 我正在尝试校准我的相机,连续校准1-2个月。然而,当我在校正对上画极线时,结果还不够好(我附上了校正对的结果)。我现在正在工作,我的校准结果非常好,并试图从视差图中获得深度图。我已经录制了一个图像序列,.avi文件,当我试图从这个视频中获取深度贴图时,当我尝试这样做时,我面临着一个不稳定的局面。在前一帧中为白色的斑点在下一帧中可能为黑色。所以我不能仅仅通过过滤差异来计算人数。我使用SGBM从校正图像中获取深度。我仍然被认为是这个项目的业余爱好者。我愿意接受任何建议。(如何进行更好的校准?更好的视差图?更好的深度图?) 这是深度图,并对其进行了校正: 整流对线和极线 我已经校准了我的相机,几乎有600双,并且改进了它。我的总体平均误差是.13像素,35对图像Python 如何使用立体相机创建好的深度贴图?,python,matlab,depth,stereo-3d,disparity-mapping,Python,Matlab,Depth,Stereo 3d,Disparity Mapping,我在一个项目上学习了很长时间。我的目标是从立体相机的图像中获取深度图,并仅过滤人体,以便计算人体内部的数量 我正在尝试校准我的相机,连续校准1-2个月。然而,当我在校正对上画极线时,结果还不够好(我附上了校正对的结果)。我现在正在工作,我的校准结果非常好,并试图从视差图中获得深度图。我已经录制了一个图像序列,.avi文件,当我试图从这个视频中获取深度贴图时,当我尝试这样做时,我面临着一个不稳定的局面。在前一帧中为白色的斑点在下一帧中可能为黑色。所以我不能仅仅通过过滤差异来计算人数。我使用SGBM
minDisparity=-1,
numDisparities=2*16, # max_disp has to be dividable by 16 f. E. HH 192, 256
blockSize=window_size,
P1=8 * 3 * window_size,
# wsize default 3; 5; 7 for SGBM reduced size image; 15 for SGBM full size image (1300px and above); 5 Works nicely
P2=32 * 3 * window_size,
disp12MaxDiff=12,
uniquenessRatio=1,
speckleWindowSize=50,
speckleRange=32,
preFilterCap=63,
mode=cv2.STEREO_SGBM_MODE_SGBM_3WAY
这是我的块匹配参数。要改进视差贴图的结果,可以实施后期过滤,下面是一个教程()。我还使用了一个额外的斑点过滤器和选项来填补缺失的差异。python实现如下所示:
stereoProcessor = cv2.StereoSGBM_create(
minDisparity=0,
numDisparities = max_disparity, # max_disp has to be dividable by 16 f. E. HH 192, 256
blockSize=window_size,
P1 = p1, # 8*number_of_image_channels*SADWindowSize*SADWindowSize
P2 = p2, # 32*number_of_image_channels*SADWindowSize*SADWindowSize
disp12MaxDiff=disp12Maxdiff,
uniquenessRatio= uniquenessRatio,
speckleWindowSize=speckle_window,
speckleRange=speckle_range,
preFilterCap=prefiltercap,
# mode=cv2.STEREO_SGBM_MODE_HH# numDisparities = max_disparity, # max_disp has to be dividable by 16 f. E. HH 192, 256
)
#stereoProcessor = cv2.StereoBM_create(numDisparities=16, blockSize=15)
# set up left to right + right to left left->right + right->left matching +
# weighted least squares filtering (not used by default)
left_matcher = stereoProcessor
right_matcher = cv2.ximgproc.createRightMatcher(left_matcher)
#Image information
height, width, channels = I.shape
frameL= I[:,0:int(width/2),:]
frameR = I[:,int(width/2):width,:]
# remember to convert to grayscale (as the disparity matching works on grayscale)
grayL = cv2.cvtColor(frameL,cv2.COLOR_BGR2GRAY)
grayR = cv2.cvtColor(frameR,cv2.COLOR_BGR2GRAY)
# perform preprocessing - raise to the power, as this subjectively appears
# to improve subsequent disparity calculation
grayL = np.power(grayL, 0.75).astype('uint8')
grayR = np.power(grayR, 0.75).astype('uint8')
# compute disparity image from undistorted and rectified versions
# (which for reasons best known to the OpenCV developers is returned scaled by 16)
if (wls_filter):
wls_filter = cv2.ximgproc.createDisparityWLSFilter(matcher_left=left_matcher)
wls_filter.setLambda(wls_lambda)
wls_filter.setSigmaColor(wls_sigma)
displ = left_matcher.compute(cv2.UMat(grayL),cv2.UMat(grayR)) # .astype(np.float32)/16
dispr = right_matcher.compute(cv2.UMat(grayR),cv2.UMat(grayL)) # .astype(np.float32)/16
displ = np.int16(cv2.UMat.get(displ))
dispr = np.int16(cv2.UMat.get(dispr))
disparity = wls_filter.filter(displ, grayL, None, dispr)
else:
disparity_UMat = stereoProcessor.compute(cv2.UMat(grayL),cv2.UMat(grayR))
disparity = cv2.UMat.get(disparity_UMat)
speckleSize = math.floor((width * height) * 0.0005)
maxSpeckleDiff = (8 * 16) # 128
cv2.filterSpeckles(disparity, 0, speckleSize, maxSpeckleDiff)
# scale the disparity to 8-bit for viewing
# divide by 16 and convert to 8-bit image (then range of values should
# be 0 -> max_disparity) but in fact is (-1 -> max_disparity - 1)
# so we fix this also using a initial threshold between 0 and max_disparity
# as disparity=-1 means no disparity available
_, disparity = cv2.threshold(disparity,0, max_disparity * 16, cv2.THRESH_TOZERO)
disparity_scaled = (disparity / 16.).astype(np.uint8)
# fill disparity if requested
if (fill_missing_disparity):
_, mask = cv2.threshold(disparity_scaled,0, 1, cv2.THRESH_BINARY_INV)
mask[:,0:120] = 0
disparity_scaled = cv2.inpaint(disparity_scaled, mask, 2, cv2.INPAINT_NS)
# display disparity - which ** for display purposes only ** we re-scale to 0 ->255
disparity_to_display = (disparity_scaled * (256. / self.value_NumDisp)).astype(np.uint8)
为什么要使用距离图来检测人类?在我看来,这是一个目标检测问题 无论如何,在目前获取距离图的技术水平下,我推荐基于人工智能的模型 像NeRF这样的模型已经取得了惊人的成果
- 谷歌提供了一个深度图,但基于一个摄像头
- 英伟达酒店
- 基于Nerf从一个视频中实现了三维重建
在几周内,我将致力于此,我想实现一个在TensorFlowLite中运行的模型,该模型通过立体相机实现深度图快速浏览几件事:
- GBM中的P1、P2参数应为平方,计算如下:
P1=8*3*blockSize**2 P2=32*3*块大小**2
- 立体声GBM支持彩色图像,请尝试跳过灰度转换。如果使用灰度,则应删除P1、P2参数中的*3乘数。这是图像通道数,其中灰度为1
- 您使用的是cv2.STEREO\u SGBM\u MODE\u 3WAY,速度更快,但精确度更低。要获得更好的效果,但速度较慢,请尝试使用
(默认值,5个邻居)或cv2.STEREO\u SGBM\u MODE\u SGBM
(8个邻居)cv2.STEREO\u SGBM\u MODE\u HH
- 您的图像具有不同的曝光,如果可能,请尝试固定相机的AWB/增益,以便一致地捕获图像