Matlab 加速代码：内存与速度

我必须在MATLAB中创建和存储几个矩阵。如果我选择使用多维数组，那么我有一个内存问题。如果我选择一个单元格，那么代码会非常慢。如何提高计算速度和内存使用率

这里是我的代码的简化版本

%%%%% MAIN FILE %%%%%%
rng default;  %for reproducibility

%% Paramaters
N=20; 
M=400;
R=M/2; 
B=M/2;

%% Generate the matrix data of dimension NM x(1+1+N +(N-1)+(N-1))
data=[kron((1:1:M)', ones(N,1)) repmat((1:1:N)', M,1) randn(M*N, N +(N-1)+(N-1))]; 

%% Generate the matrix unob of dimension NMx(1+1+N-1)xR
unob=[repmat(data(:,1:2),1,1,R)  randn(M*N,N-1,R)]; 

%% Option 1: MEMORY PROBLEM
%bootdata1 and bootunob1 have respectively dimension NMx(1+1+N +(N-1)+(N-1))xB and NMx(1+1+N-1)xRxB
[bootdata1, bootunob1]=boot1(N,M,B,R,data, unob);

%% Option 2: SLOW, NEVER ENDING
%bootdata is a matrix of dimension NMx(1+1+N +(N-1)+(N-1))xB
%bootunob is a cell of dimension Bx1 with bootunob{b} that is a matrix of dimension NMx(1+1+N-1)
[bootdata, bootunob]=boot(N,M,B,R,data, unob) ; 

功能

boot1

function [bootdata, bootunob]=boot1(N,M,B,R,data, unob)    

         %Allocate space
         bootdata=zeros(N*M,1+1+N+(N-1)+(N-1), B);
         bootunob=zeros(N*M,(1+1+N-1),R,B); 

         for b=1:B   

             %Draw uniformly at random with replacement M integers from {1,...,M} and 
             %store them into the vector networkindices of dimension 1xM
             networkindices =randi([1 M],M,1); 

             %Fill bootdata(:,:,b) and bootunob(:,:,:,b)
             for m=1:M
                 bootdata((m-1)*N+1:m*N, :,b)=data(data(:,1)==networkindices(m),:);
                 bootdata((m-1)*N+1:m*N,1,b)=m*(ones(N,1)); %change indices

                 for r=1:R
                     bootunob((m-1)*N+1:m*N, :,r,b)=unob(unob(:,1,r)==networkindices(m),:,r); 
                     bootunob((m-1)*N+1:m*N,1,r,b)=m*(ones(N,1));
                 end
             end


         end
end

功能

启动

：

function [bootdata, bootunob]=boot(N,M,B,R,data, unob)    

         %Allocate space
         bootdata=zeros(N*M,1+1+N+(N-1)+(N-1), B);
         bootunob=cell(B,1);

         for b=1:B
             bootunob{b}=zeros(N*M,(1+1+N-1),R) ;

             %Draw uniformly at random with replacement M integers from {1,...,M} and 
             %store them into the vector networkindices of dimension 1xM
             networkindices =randi([1 M],M,1); 

             %Fill bootdata(:,:,b) and bootunob{b}
             for m=1:M
                 bootdata((m-1)*N+1:m*N, :,b)=data(data(:,1)==networkindices(m),:);
                 bootdata((m-1)*N+1:m*N,1,b)=m*(ones(N,1)); %change indices

                 for r=1:R
                     bootunob{b}((m-1)*N+1:m*N, :,r)=unob(unob(:,1,r)==networkindices(m),:,r); 
                     bootunob{b}((m-1)*N+1:m*N,1,r)=m*(ones(N,1));
                 end
             end
         end

end

---

新方法

速度明显加快，在我的计算机上，写速度大约为100-300MB/s，尽管瓶颈已经成为代码的一部分，请参阅本节末尾的我的评论

新的

引导速度
方法：

function [bootDataFiles, bootUnobFiles] = bootFaster(N, M, B, R, data, unob)
    % Temporary Variables
    % Based on some simple math, these slices should be OKish. This will
    % crash and burn if you try to use the save with files greater than
    % 2^31 bytes, so just be careful with that.
    bootDataTemp = zeros(N*M, 1+1+N+(N-1)+(N-1), 1);
    bootUnobTemp = zeros(N*M, 1+1+(N-1), R, 1);

    % Matricies containing the file names for the .mat files. There are B
    % rows.
    bootDataFiles = zeros(B, 17);
    bootUnobFiles = zeros(B, 17);

    for b = 1:B
        networkIndices = randi([1 M], M, 1);

        for m = 1:M
            bootDataTemp((m-1)*N+1:m*N, :) = data(data(:, 1) == networkIndices(m), :);
            bootDataTemp((m-1)*N+1:m*N, 1) = m * (ones(N, 1)); %change indices

            for r = 1:R
                bootUnobTemp((m-1)*N+1:m*N, :, r) = unob(unob(:, 1, r) == networkIndices(m), :, r); 
                bootUnobTemp((m-1)*N+1:m*N, 1, r) = m * (ones(N, 1));
            end
        end

        % Creates the file name for the bth matrix.
        % NOTE: if you change the 5 it will change the length of each file
        % name, you will have to change the number of columns in
        % bootDataFiles and bootUnobFiles accordingly.
        bootDataFileB = sprintf('bootData%5.i.mat', b);
        bootUnobFileB = sprintf('bootUnob%5.i.mat', b);

        % Writes the contents of the tempoary variables to the file
        save(bootDataFileB, 'bootDataTemp', '-v6');
        save(bootUnobFileB, 'bootUnobTemp', '-v6');

        % Storing the file names.
        bootDataFiles(b, :) = bootDataFileB;
        bootUnobFiles(b, :) = bootUnobFileB;
    end

    % Convert the values back to chars (each row will not be a "string")
    bootDataFiles = char(bootDataFiles);
    bootUnobFiles = char(bootUnobFiles);
end

要使用此文件中的数据，现在可以利用matfile方法，如下所示

[bootDataFiles, bootUnobFiles] = bootFaster(N,M,B,R,data, unob);

bootDataAtbFile = matfile(bootDataFiles(b, :));
% Note the use of "bootDataTemp" to access the data, you have to use
% the name of the temporary variable that you stored the data in inside
% bootFaster. E.g. to access the bootUnob data you would have to use
% bootUnobTemp, or whatever you choose to rename them to.
bootDataAtb = bootDataAtbFile.bootDataTemp;

最后一点。这一行：

bootUnobTemp((m-1)*N+1:m*N, :, r) = unob(unob(:, 1, r) == networkIndices(m), :, r);

速度非常慢。例如N=30，M=200。该行占用了执行时间的2/3。强烈建议您看看是否可以重构它


旧方法-由于压缩，2015b速度太慢

使用
matfile
函数和写入文件的功能，这变得非常容易实现。您不需要对代码进行太多更改。请参阅下面的代码，以使其正常工作

在主文件中，您需要执行以下操作：

file = bootFast(N,M,B,R,data, unob);

bootdata1F = file.bootdata;
bootunob1F = file.bootunob;

和
快速启动
：

function file = bootFast(N, M, B, R, data, unob)
% Opens the file, or creates it if it doesn't exsit
file = matfile('output', 'Writable', true);

% Sets the sizes of the variables
file.bootdata(N*M, 1+1+N+(N-1)+(N-1), B) = 0;
file.bootunob(N*M, 1+1+(N-1), R, B) = 0;

bootdataTemp = file.bootdata(:, :, 1);
bootunobTemp = file.bootunob(:, :, :, 1);

for b = 1:B
    networkindices = randi([1 M], M, 1);

    for m = 1:M
        bootdataTemp((m-1)*N+1:m*N, :) = data(data(:, 1) == networkindices(m), :);
        bootdataTemp((m-1)*N+1:m*N, 1) = m * (ones(N, 1)); %change indices

        for r = 1:R
            bootunobTemp((m-1)*N+1:m*N, :, r) = unob(unob(:, 1, r) == networkindices(m), :, r); 
            bootunobTemp((m-1)*N+1:m*N, 1, r) = m * (ones(N, 1));
        end
    end
    file.bootdata(:, :, b) = bootdataTemp;
    file.bootunob(:, :, :, b) = bootunobTemp;
end
end

为了加快速度，我将磁盘写入限制为每b只发生一次，但您可能需要根据内存大小调整确切的位置

另外，我希望您有一个SSD。
我也尝试过使用结构，但没有任何帮助，例如，在调用的函数中使用
for r=1:r bootunbo.b（（m-1）*N+1:m*N，：，r）=unbo（unbo（：，1，r）=networkindex（m），：，r）；bootunbo.b（（m-1）*N+1:m*N，1，r）=m*（一个（N，1））；结束
R=M/2；B=M/2
不应该是
R
吗
N/2
？@rahnema1不，为什么？两个相等的对象和不同的名称我明白了，在这个例子中它们是相等的，但它们可能不同。对于索引，你可以使用
：
而不是
1:end
@SardarUsama True。当我将打印语句从内部for循环中取出时，我忘记将更改为。哦！Fixed@user9985谢谢这些行
file.bootdata（：，：，b）=bootdataTemp；file.bootunbo（：，：，：，b）=bootunbotemp永远。你知道为什么吗？是的。这些是写入文件的行。实际上，我一直在玩它，看看是否有任何方法可以让它运行得更快，方法是在文件中包含B 2-D bootdata和B 3-D BootUnbo条目，而不是1个3D bootdata和1个4-D BootUnbo条目。@user3285148。它们速度如此之慢的原因之一是matlab在默认情况下启用了压缩功能，以便写入如此大的MAT文件。你使用什么版本的matlab？