Warning: file_get_contents(/data/phpspider/zhask/data//catemap/8/mysql/62.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/1/database/9.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/7/arduino/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
MySql数据库的Matlab结构_Mysql_Database_Matlab_Structure - Fatal编程技术网

MySql数据库的Matlab结构

MySql数据库的Matlab结构,mysql,database,matlab,structure,Mysql,Database,Matlab,Structure,我在Matlab中有一个结构。该结构包含210万行,混合了双精度、整数和字符。该结构称为TaqQ TaqQ.time 2100000x1 uint32, TaqQ.bid 2100000x1 double, TaqQ.ex 2100000x4 char, 如何快速地将该结构转移到MySql 可能通过将结构保存到csv文件,然后将其导入mysql。我试过: csvwrite('test.csv',[TaqQ.time TaqQ.bid TaqQ.ex]) %this is very slow

我在Matlab中有一个结构。该结构包含210万行,混合了双精度、整数和字符。该结构称为TaqQ

TaqQ.time 2100000x1 uint32,
TaqQ.bid 2100000x1 double,
TaqQ.ex 2100000x4 char,
如何快速地将该结构转移到MySql

可能通过将结构保存到csv文件,然后将其导入mysql。我试过:

csvwrite('test.csv',[TaqQ.time TaqQ.bid TaqQ.ex]) %this is very slow
csvwrite('test.csv',[TaqQ.time';TaqQ.bid';TaqQ.ex'])速度很快,但不知道如何在MySql中处理它

我也试过使用fastinsert,但我的速度太慢了

我还尝试:

 connHandle = conn.Handle;
 stmt = connHandle.createStatement;
 for i= 1:2100000
 stmt.addBatch('INSERT INTO Quotes (time,bid,ex) VALUES (TaqQ.time(i),
 TaqQ.bid(i), TaqQ.ex(i))')
结束 stmt.executeBatch; stmt.close; `

但我也是慢下来的


有人能帮我吗?

这将占用大量RAM,但它将在内存中执行操作,最大限度地减少函数调用的数量,并将一次写入所有内容,因此速度应该很快:

% Test struct
TaqQ = struct( ...
   'time', [1;2;3;4;5], ...
   'bid',  [110.1;120.1;130.1;140.1;150.1], ...
   'ex',   ['ABCD';'EFGH';'IJKL';'MNOP';'QRST'] ...
);

% Writing the file
NL = 10;           % ASCII new line
DQ = double('"');  % ASCII double quote

f = fopen('test.csv', 'wb');
fwrite(f, [ ...
   reshape(sprintf('% 10u,',   TaqQ.time), 11, []); ...  % 10 digits
   reshape(sprintf('% 10.3f,', TaqQ.bid),  11, []); ...  % 10 digits + point + sign 
          char(DQ*(ones(1,size(TaqQ.ex,1))));       ...  % open quotes
                               TaqQ.ex.';           ...  % already char
          char(DQ*(ones(1,size(TaqQ.ex,1))));       ...  % end quotes
          char(NL*(ones(1,size(TaqQ.ex,1))))        ...  % newlines
]);
fclose(f);
事实上,我很好奇它是否快,我无法在巨大的数据上进行测试。请让我知道

如果要挤出所有空格,则这是一个(较慢的)变体:


你如何定义fast?另外,您的机器上有多少可用内存?(在Windows安装上执行命令
memory
)我有8GB内存,运行Mac osx yosemite。我试过的东西大约需要10分钟。我希望它只需要大约30秒,它很快,但它没有给出正确的答案。它将所有内容写入一行。其中一些行是4466666777777053334400001233307459770395069025214400304533079053604604517,,,,,,,,,,,,,,,没有逗号?真奇怪。。。你能在你的问题上发布你每个数据的前5条记录吗?我用TaqQ.time(1:15)替换了TaqQ.time(1:15)和TaqQ.bid(1:15)
% Preparing content
NL = 10;           % ASCII new line
DQ = double('"');  % ASCII double quote
bytes =  [ ...
   reshape(sprintf('% 10u,',   TaqQ.time), 11, []); ...  % 10 digits
   reshape(sprintf('% 10.3f,', TaqQ.bid),  11, []); ...  % 10 digits + point + sign 
          char(DQ*(ones(1,size(TaqQ.ex,1))));       ...  % open quotes
                               TaqQ.ex.';           ...  % already char
          char(DQ*(ones(1,size(TaqQ.ex,1))));       ...  % end quotes
          char(NL*(ones(1,size(TaqQ.ex,1))))        ...  % newlines
];

% Writing content without spaces
f = fopen('test.csv', 'wb');
fwrite(f,bytes(bytes~=' '));
fclose(f);