使用Matlab录制/播放Kinect-倒置，以黑/白而非灰度显示

我正试图编写一个程序来记录和播放xbox kinect的颜色和深度流，以便于测试图像处理程序。目前，我已经完成了批量和颜色流的作品很好。然而，我有麻烦的深度流

目前，深度流正在倒转播放，只有黑白两种颜色。我有三个想法来解释为什么会出现这种情况： 1） 从11位到8位的转换 2） 运动JPEG 2000格式（以前从未使用过此格式） 3） 颜色映射是错误的

下面是我正在使用的代码。我觉得我不可能是唯一一个尝试这样做的人，所以任何指针和帮助都将非常感谢，因为我在网上找不到任何这样具体的东西

    %------------------------------------------------
    %------------------------------------------------
    %Code to record kinect colour and sensor data
    %using code supplied on http://www.mathworks.co.uk/help/imaq/examples/using-the-                kinect-r-for-windows-r-from-image-acquisition-toolbox-tm.html
    %and http://www.mathworks.co.uk/help/imaq/examples/logging-data-to-disk.html
    %------------------------------------------------
    %------------------------------------------------

    dbstop if error

    imaqreset %deletes any image acquisition objects that exsist in memory and uploads         all adaptors loaded by the toolbox. As a result, image acquisition hardware is reset

    %------------------------------------------------
    %setting up video streams
    %------------------------------------------------
    disp('Setting up video streams');

    %Call up dicertory containing utility functions
    utilpath = fullfile(matlabroot, 'toolbox', 'imaq', 'imaqdemos', 'html', 'KinectForWindows');
    addpath(utilpath);

    %Create the videoinput objects for the colour and depth streams
    colourVid = videoinput('kinect', 1, 'RGB_640x480');
    %preview(colourVid);
    depthVid = videoinput('kinect', 2, 'Depth_640x480');

    %set backlight compensation with centre priority
    %set(colourVid, 'BacklightCompensation', 'CentrePriority');

    %Set camera angle to 0
    %set(colourVid, 'CameraElevationAngle', 0);

    disp('Video stream set-up complete');

    %------------------------------------------------
    %setting up record
    %------------------------------------------------

    % set the data streams to logging mode and to disk
    set(colourVid, 'LoggingMode', 'Disk&Memory');
    set(depthVid, 'LoggingMode', 'Disk&Memory');

    %Set a video timeout property limit to 50 seconds from
    %www.mathworks.com/matlabcentral/answers/103543-why-do-i-receive-the-error-getdata-timed-out-before-frames-were-available-when-using-getdata-in-im
    set(colourVid, 'Timeout',50);
    set(depthVid, 'Timeout',50);

    %Creat a VideoReader object
    colourLogfile = VideoWriter('colourTrial5.mj2', 'Motion JPEG 2000');
    depthLogfile = VideoWriter('depthTrial5.mj2', 'Motion JPEG 2000');

    %configure the video input object to use the VideoWriter object
    colourVid.DiskLogger = colourLogfile;
    depthVid.DiskLogger = depthLogfile;

    %set the triggering mode to 'manual'
    triggerconfig([colourVid depthVid], 'manual');

    %set the FramePerTrigger property of the VIDEOINPUT objects to 100 to
    %acquire 100 frames per trigger.
    set([colourVid depthVid], 'FramesPerTrigger', 200);

    disp('Video record set-up complete');

    %------------------------------------------------
    %Initiating the aquisition
    %------------------------------------------------
    disp('Starting Steam');

    %Start the colour and depth device. This begins acquisition, but does not
    %start logging of acquired data
    start([colourVid depthVid]);

    pause(20); %allow time for both streams to start

    %Trigger the devices to start logging of data.
    trigger([colourVid depthVid]);

    %Retrieve the acquired data
    [colourFrameData, colourTimeData, colourMetaData] = getdata(colourVid);
    [depthFrameData, depthTimeData, depthMetaData] = getdata(depthVid);

    stop([colourVid depthVid])

    disp('Recording Complete')

    %------------------------------------------------
    %Play back recordings
    %------------------------------------------------
    disp('Construct playback objects')

    colourPlayback = VideoReader('colourTrial5.mj2');
    depthPlayback = VideoReader('depthTrial5.mj2');

    %Set colour(c) playback parameters
    cFrames = colourPlayback.NumberOfFrames;
    cHeight = colourPlayback.Height;
    cWidth = colourPlayback.Width;

    %Preallocate movie structure
    colourMov(1:cFrames)=struct('cdata', zeros(cHeight,cWidth,3,'uint8'),'colormap',[]);

    disp('Reading colour frames one by one')

    %read one frame at a time
    for k = 1:cFrames
        colourMov(k).cdata=read(colourPlayback,k);
    end

    disp('Sizing figure for colour playback')

    %Size a figure based on the video's width and height
    hf1=figure;
    set(hf1,'position',[150 150 cWidth cHeight])

    disp('Playing Colour recording')

    %play back the movie once at the video's frame rate
    movie(hf1,colourMov,1,colourPlayback.FrameRate);

    %Set depth(d) playback parameters
    dFrames = depthPlayback.NumberOfFrames;
    dHeight = depthPlayback.Height;
    dWidth = depthPlayback.Width;

    %Preallocate movie structure
    depthMov(1:dFrames)=struct('cdata', zeros(dHeight,dWidth,3,'uint8'),'colormap',gray(256));

    disp('Reading depth frames one by one')

    %read one frame at a time
    for k = 1:dFrames
        depthMov(k).cdata=uint8(read(depthPlayback,k));
        %depthMov(k)=imrotate(depthMov(k),180); %tried this to no effect
    end

    disp('Sizing figure for depth playback')

    %Size a figure based on the video's width and height
    hf2=figure;
    set(hf2,'position',[150 150 dWidth dHeight])

    disp('Playing Depth recording')

    %play back the movie once at the video's frame rate
    movie(hf2,depthMov,1,depthPlayback.FrameRate);

    %clear videos from workspace
    delete([colourVid depthVid])
    clear [colourVid depthVid]

---

除了显示深度数据的方式外，脚本编写得很好且正确

Kinect将深度记录为16位无符号整数。深度帧中的像素值是该像素中的任何对象与摄影机平面之间的距离（以毫米为单位）。因此，如果您关心的对象像素的深度值读出为，例如，1723，这意味着该对象的部分距离相机1.723米

那么，这与您损坏的显示功能有什么关系呢？以下是您的旧显示片段：

%read one frame at a time
for k = 1:dFrames
    depthMov(k).cdata=uint8(read(depthPlayback,k));
end

问题就在

uint8

上。我确信很多深度值都超过了255，这意味着场景中的对象距离摄影机的距离超过了0.2米。事实上，Kinect甚至无法感知关闭的数据

下面是解决显示问题的方法：

%read one frame at a time
maxDistFromCamera = 1600; % 1600 millimeters
for k = 1:dFrames
    % Depth frames are int16.
    depthFrame = read(depthPlayback,k); 
    % We'll rescale the image from [0,maxDistFromCamera] to [0,255]
    depthFrame = 255.0*single(depthFrame)/maxDistFromCamera;
    % And then recast it to uint8 for display.
    depthMov(k).cdata=uint8(depthFrame);
end

为了方便起见，下面是我编辑的整个脚本

%------------------------------------------------
%------------------------------------------------
%Code to record kinect colour and sensor data
%using code supplied on http://www.mathworks.co.uk/help/imaq/examples/using-the-                kinect-r-for-windows-r-from-image-acquisition-toolbox-tm.html
%and http://www.mathworks.co.uk/help/imaq/examples/logging-data-to-disk.html
%------------------------------------------------
%------------------------------------------------

imaqreset %deletes any image acquisition objects that exsist in memory and uploads         all adaptors loaded by the toolbox. As a result, image acquisition hardware is reset

%------------------------------------------------
%setting up video streams
%------------------------------------------------
disp('Setting up video streams');

%Call up dicertory containing utility functions
utilpath = fullfile(matlabroot, 'toolbox', 'imaq', 'imaqdemos', 'html', 'KinectForWindows');
addpath(utilpath);

%Create the videoinput objects for the colour and depth streams
colourVid = videoinput('kinect', 1, 'RGB_640x480');
%preview(colourVid);
depthVid = videoinput('kinect', 2, 'Depth_320x240');

% Set the depth mode to near.
srcDepth = getselectedsource(depthVid);
srcColor = getselectedsource(colourVid);
set(srcDepth, 'DepthMode' , 'Near');
set(srcDepth, 'CameraElevationAngle', 0);

%set backlight compensation with centre priority
set(srcColor, 'BacklightCompensation', 'CenterPriority');

disp('Video stream set-up complete');

%------------------------------------------------
%setting up record
%------------------------------------------------

% set the data streams to logging mode and to disk
set(colourVid, 'LoggingMode', 'Disk&Memory');
set(depthVid, 'LoggingMode', 'Disk&Memory');

%Set a video timeout property limit to 50 seconds from
%www.mathworks.com/matlabcentral/answers/103543-why-do-i-receive-the-error-getdata-timed-out-before-frames-were-available-when-using-getdata-in-im
set(colourVid, 'Timeout',50);
set(depthVid, 'Timeout',50);

%Creat a VideoReader object
colourLogfile = VideoWriter('colourTrial5.mj2', 'Motion JPEG 2000');
depthLogfile = VideoWriter('depthTrial5.mj2', 'Archival');

%configure the video input object to use the VideoWriter object
colourVid.DiskLogger = colourLogfile;
depthVid.DiskLogger = depthLogfile;

%set the triggering mode to 'manual'
triggerconfig([colourVid depthVid], 'manual');

%set the FramePerTrigger property of the VIDEOINPUT objects to 100 to
%acquire 100 frames per trigger.
set([colourVid depthVid], 'FramesPerTrigger', 30);

disp('Video record set-up complete');

%------------------------------------------------
%Initiating the aquisition
%------------------------------------------------
disp('Starting Steam');

%Start the colour and depth device. This begins acquisition, but does not
%start logging of acquired data
start([colourVid depthVid]);

pause(20); %allow time for both streams to start

disp('Starting Depth Stream');

%Trigger the devices to start logging of data.
trigger([colourVid depthVid]);

%Retrieve the acquired data
[colourFrameData, colourTimeData, colourMetaData] = getdata(colourVid);
[depthFrameData, depthTimeData, depthMetaData] = getdata(depthVid);

stop([colourVid depthVid])

disp('Recording Complete')

%------------------------------------------------
%Play back recordings
%------------------------------------------------
disp('Construct playback objects')

colourPlayback = VideoReader('colourTrial5.mj2');
depthPlayback = VideoReader('depthTrial5.mj2');

%Set colour(c) playback parameters
cFrames = colourPlayback.NumberOfFrames;
cHeight = colourPlayback.Height;
cWidth = colourPlayback.Width;

%Preallocate movie structure
colourMov(1:cFrames)=struct('cdata', zeros(cHeight,cWidth,3,'uint8'),'colormap',[]);

disp('Reading colour frames one by one')

%read one frame at a time
for k = 1:cFrames
    colourMov(k).cdata=read(colourPlayback,k);
end

disp('Sizing figure for colour playback')

%Size a figure based on the video's width and height
hf1=figure;
set(hf1,'position',[150 150 cWidth cHeight])

disp('Playing Colour recording')

%play back the movie once at the video's frame rate
movie(hf1,colourMov,1,colourPlayback.FrameRate);

%Set depth(d) playback parameters
dFrames = depthPlayback.NumberOfFrames;
dHeight = depthPlayback.Height;
dWidth = depthPlayback.Width;

%Preallocate movie structure
depthMov(1:dFrames)=struct('cdata', zeros(dHeight,dWidth,3,'uint8'),'colormap',gray(256));

disp('Reading depth frames one by one')

%read one frame at a time
maxDistFromCamera = 1600; % 1600 millimeters
for k = 1:dFrames
    % Depth frames are int16.
    depthFrame = read(depthPlayback,k); 
    % We'll rescale the image from [0,maxDistFromCamera] to [0,255]
    depthFrame = 255.0*single(depthFrame)/maxDistFromCamera;
    % And then recast it to uint8 for display.
    depthMov(k).cdata=uint8(depthFrame);
end

disp('Sizing figure for depth playback')

%Size a figure based on the video's width and height
hf2=figure;
set(hf2,'position',[150 150 dWidth dHeight])

disp('Playing Depth recording')

%play back the movie once at the video's frame rate
movie(hf2,depthMov,1,depthPlayback.FrameRate);

%clear videos from workspace
delete([colourVid depthVid])
clear [colourVid depthVid]

close all;

---

有一些小的文本差异，例如，我的Kinect SDK要求一些参数的拼写不同，我只是内联修复了这些参数。我还禁用了顶部的积极调试设置。不重要，只是记下来。最后，我设置了，以便在保存时不会损坏任何深度值（对于精确的深度测量应用程序很重要，这正是我目前所做的）。

除了如何显示深度数据之外，您的脚本编写得很好且正确

Kinect将深度记录为16位无符号整数。深度帧中的像素值是该像素中的任何对象与摄影机平面之间的距离（以毫米为单位）。因此，如果您关心的对象像素的深度值读出为，例如，1723，这意味着该对象的部分距离相机1.723米

那么，这与您损坏的显示功能有什么关系呢？以下是您的旧显示片段：

%read one frame at a time
for k = 1:dFrames
    depthMov(k).cdata=uint8(read(depthPlayback,k));
end

问题就在

uint8

上。我确信很多深度值都超过了255，这意味着场景中的对象距离摄影机的距离超过了0.2米。事实上，Kinect甚至无法感知关闭的数据

下面是解决显示问题的方法：

%read one frame at a time
maxDistFromCamera = 1600; % 1600 millimeters
for k = 1:dFrames
    % Depth frames are int16.
    depthFrame = read(depthPlayback,k); 
    % We'll rescale the image from [0,maxDistFromCamera] to [0,255]
    depthFrame = 255.0*single(depthFrame)/maxDistFromCamera;
    % And then recast it to uint8 for display.
    depthMov(k).cdata=uint8(depthFrame);
end

为了方便起见，下面是我编辑的整个脚本

%------------------------------------------------
%------------------------------------------------
%Code to record kinect colour and sensor data
%using code supplied on http://www.mathworks.co.uk/help/imaq/examples/using-the-                kinect-r-for-windows-r-from-image-acquisition-toolbox-tm.html
%and http://www.mathworks.co.uk/help/imaq/examples/logging-data-to-disk.html
%------------------------------------------------
%------------------------------------------------

imaqreset %deletes any image acquisition objects that exsist in memory and uploads         all adaptors loaded by the toolbox. As a result, image acquisition hardware is reset

%------------------------------------------------
%setting up video streams
%------------------------------------------------
disp('Setting up video streams');

%Call up dicertory containing utility functions
utilpath = fullfile(matlabroot, 'toolbox', 'imaq', 'imaqdemos', 'html', 'KinectForWindows');
addpath(utilpath);

%Create the videoinput objects for the colour and depth streams
colourVid = videoinput('kinect', 1, 'RGB_640x480');
%preview(colourVid);
depthVid = videoinput('kinect', 2, 'Depth_320x240');

% Set the depth mode to near.
srcDepth = getselectedsource(depthVid);
srcColor = getselectedsource(colourVid);
set(srcDepth, 'DepthMode' , 'Near');
set(srcDepth, 'CameraElevationAngle', 0);

%set backlight compensation with centre priority
set(srcColor, 'BacklightCompensation', 'CenterPriority');

disp('Video stream set-up complete');

%------------------------------------------------
%setting up record
%------------------------------------------------

% set the data streams to logging mode and to disk
set(colourVid, 'LoggingMode', 'Disk&Memory');
set(depthVid, 'LoggingMode', 'Disk&Memory');

%Set a video timeout property limit to 50 seconds from
%www.mathworks.com/matlabcentral/answers/103543-why-do-i-receive-the-error-getdata-timed-out-before-frames-were-available-when-using-getdata-in-im
set(colourVid, 'Timeout',50);
set(depthVid, 'Timeout',50);

%Creat a VideoReader object
colourLogfile = VideoWriter('colourTrial5.mj2', 'Motion JPEG 2000');
depthLogfile = VideoWriter('depthTrial5.mj2', 'Archival');

%configure the video input object to use the VideoWriter object
colourVid.DiskLogger = colourLogfile;
depthVid.DiskLogger = depthLogfile;

%set the triggering mode to 'manual'
triggerconfig([colourVid depthVid], 'manual');

%set the FramePerTrigger property of the VIDEOINPUT objects to 100 to
%acquire 100 frames per trigger.
set([colourVid depthVid], 'FramesPerTrigger', 30);

disp('Video record set-up complete');

%------------------------------------------------
%Initiating the aquisition
%------------------------------------------------
disp('Starting Steam');

%Start the colour and depth device. This begins acquisition, but does not
%start logging of acquired data
start([colourVid depthVid]);

pause(20); %allow time for both streams to start

disp('Starting Depth Stream');

%Trigger the devices to start logging of data.
trigger([colourVid depthVid]);

%Retrieve the acquired data
[colourFrameData, colourTimeData, colourMetaData] = getdata(colourVid);
[depthFrameData, depthTimeData, depthMetaData] = getdata(depthVid);

stop([colourVid depthVid])

disp('Recording Complete')

%------------------------------------------------
%Play back recordings
%------------------------------------------------
disp('Construct playback objects')

colourPlayback = VideoReader('colourTrial5.mj2');
depthPlayback = VideoReader('depthTrial5.mj2');

%Set colour(c) playback parameters
cFrames = colourPlayback.NumberOfFrames;
cHeight = colourPlayback.Height;
cWidth = colourPlayback.Width;

%Preallocate movie structure
colourMov(1:cFrames)=struct('cdata', zeros(cHeight,cWidth,3,'uint8'),'colormap',[]);

disp('Reading colour frames one by one')

%read one frame at a time
for k = 1:cFrames
    colourMov(k).cdata=read(colourPlayback,k);
end

disp('Sizing figure for colour playback')

%Size a figure based on the video's width and height
hf1=figure;
set(hf1,'position',[150 150 cWidth cHeight])

disp('Playing Colour recording')

%play back the movie once at the video's frame rate
movie(hf1,colourMov,1,colourPlayback.FrameRate);

%Set depth(d) playback parameters
dFrames = depthPlayback.NumberOfFrames;
dHeight = depthPlayback.Height;
dWidth = depthPlayback.Width;

%Preallocate movie structure
depthMov(1:dFrames)=struct('cdata', zeros(dHeight,dWidth,3,'uint8'),'colormap',gray(256));

disp('Reading depth frames one by one')

%read one frame at a time
maxDistFromCamera = 1600; % 1600 millimeters
for k = 1:dFrames
    % Depth frames are int16.
    depthFrame = read(depthPlayback,k); 
    % We'll rescale the image from [0,maxDistFromCamera] to [0,255]
    depthFrame = 255.0*single(depthFrame)/maxDistFromCamera;
    % And then recast it to uint8 for display.
    depthMov(k).cdata=uint8(depthFrame);
end

disp('Sizing figure for depth playback')

%Size a figure based on the video's width and height
hf2=figure;
set(hf2,'position',[150 150 dWidth dHeight])

disp('Playing Depth recording')

%play back the movie once at the video's frame rate
movie(hf2,depthMov,1,depthPlayback.FrameRate);

%clear videos from workspace
delete([colourVid depthVid])
clear [colourVid depthVid]

close all;

---

有一些小的文本差异，例如，我的Kinect SDK要求一些参数的拼写不同，我只是内联修复了这些参数。我还禁用了顶部的积极调试设置。不重要，只是记下来。最后，我设置了，以便在保存时不会损坏任何深度值（对于精确的深度测量应用程序很重要，这正是我目前所做的）。

您找到解决此问题的方法了吗？我可以将它保存在其他扩展中而不是mj2吗？有些人使用自定义记录器，例如：。然而，Matlab的内置视频记录功能似乎要求它写入某种类型的视频文件。至于深度数据的不同视频选项，由于数据以16位整数的形式出现（请参见下面的回答），因此唯一支持的能够理解16位整数的视频格式是Motion JPEG 2000。因此，除非您在写入磁盘之前提前进行一些打字，否则如果您想要视频写入功能，MJ2是您唯一的选择。您找到解决此问题的方法了吗？我可以将它保存在其他扩展中而不是mj2吗？有些人使用自定义记录器，例如：。然而，Matlab的内置视频记录功能似乎要求它写入某种类型的视频文件。至于深度数据的不同视频选项，由于数据以16位整数的形式出现（请参见下面的回答），因此唯一支持的能够理解16位整数的视频格式是Motion JPEG 2000。因此，除非您在写入磁盘之前提前进行一些打字，否则如果您想要视频写入功能，MJ2是您唯一的选择。