Memory 对fgetcsv()的单次调用会耗尽PHP中的大内存限制

Memory 对fgetcsv()的单次调用会耗尽PHP中的大内存限制,memory,nginx,php,fgetcsv,Memory,Nginx,Php,Fgetcsv,我试图从一个平均大小的CSV(10MB)中读取两行,标题和第一行。我将PHP的内存限制设置为64 MB。然而,在第二次调用fgetcsv()时,我得到了一个内存耗尽的错误 我已经删除了脚本中所有不必要的部分以进行调试,并一步一步地跟踪内存使用情况,但我仍然无法理解它。在第一次调用之后,内存使用率低于1MB,但在第二次调用时,内存耗尽错误发生 代码如下: <?php function trace($msg) { echo "<br>" . date("Y-m-d H:i:

我试图从一个平均大小的CSV(10MB)中读取两行,标题和第一行。我将PHP的内存限制设置为64 MB。然而,在第二次调用fgetcsv()时,我得到了一个内存耗尽的错误

我已经删除了脚本中所有不必要的部分以进行调试,并一步一步地跟踪内存使用情况,但我仍然无法理解它。在第一次调用之后,内存使用率低于1MB,但在第二次调用时,内存耗尽错误发生

代码如下:

<?php
function trace($msg) {
    echo "<br>" . date("Y-m-d H:i:s") . " - " . memory_get_usage() . " bytes - {$msg}";
}

ini_set('display_errors', 'on');
error_reporting(E_ALL);
ob_implicit_flush(true);

trace("CSV Memory Limit test");

$path = "../data/uploaded/1405712684_base_leitores_july2014.csv";
trace("path = {$path}");

$fp = fopen($path, "r");
trace("file pointer opened");

trace("getting header...");
$header = fgetcsv($fp, 1024);
trace("header fetched");
var_dump($header);
trace("header displayed");

trace("fetching another line");
$l1 = fgetcsv($fp, 1024);
trace("line fetched");
var_dump($header);
trace("line displayed");

我在Ubuntu14.04 LTS上运行php5 fpm(5.5.9)和nginx(1.4.6)。

最后,在经历了大量的混乱之后,我发现了问题:行结束

不知为什么,Mac的Excel正在导出一个CSV,标题上有
\r\n
,所有其他行上都有
\n
。标题可以很好地输入,但是
fgetcsv
知道第二行是整个文档

修复方法是在fopen之前添加以下内容:

ini\u set('auto\u detect\u line\u endings',TRUE)


这将使
fgetcsv
在解析CSV文件时正确检测行尾。

是否有任何行超过1024个字符?大多数行的长度约为300380个字符。我还没有对整个数据集进行测试,但他们可能会这样做。
2014-07-18 16:06:26 - 249544 bytes - CSV Memory Limit test
2014-07-18 16:06:26 - 249800 bytes - path = ../data/uploaded/1405712684_base_leitores_july2014.csv
2014-07-18 16:06:26 - 250840 bytes - file pointer opened
2014-07-18 16:06:26 - 250840 bytes - getting header...
2014-07-18 16:06:26 - 279280 bytes - header fetched
array (size=111)
  0 => string 'razao social' (length=12)
  1 => string 'Identificaao' (length=13)
  2 => string 'save' (length=4)
  3 => string 'id' (length=2)
  4 => string 'categoria' (length=9)
  5 => string 'nome fantasia' (length=13)
  6 => string 'email' (length=5)
  7 => string 'website' (length=7)
  8 => string 'logradouro' (length=10)
  9 => string 'endereco' (length=8)
  10 => string 'numero' (length=6)
  11 => string 'complemento' (length=11)
  12 => string 'bairro' (length=6)
  13 => string 'cidade' (length=6)
  14 => string 'estado' (length=6)
  15 => string 'cep' (length=3)
  16 => string 'localidade' (length=10)
  17 => string 'ddd' (length=3)
  18 => string 'telefone' (length=8)
  19 => string 'fax' (length=3)
  20 => string 'contato' (length=7)
  21 => string 'cargo' (length=5)
  22 => string 'departamento' (length=12)
  23 => string 'ramo de atividade' (length=17)
  24 => string 'data de nascimento' (length=18)
  25 => string 'especialidade' (length=13)
  26 => string 'linha de atuacao' (length=16)
  27 => string 'data de fundacao' (length=16)
  28 => string 'classificacao de leitor' (length=23)
  29 => string 'observacoes' (length=11)
  30 => string 'is_protocolada' (length=14)
  31 => string 'is_assinante' (length=12)
  32 => string 'qtd_reclamacoes' (length=15)
  33 => string 'error' (length=5)
  34 => string 'errorReport' (length=11)
  35 => string 'saved' (length=5)
  36 => string 'onTrash' (length=7)
  37 => string 'SugestAo de matEria' (length=19)
  38 => string 'Campo2
#redacted#' (length=31)
  39 => string '1' (length=1)
  40 => string '0' (length=1)
  41 => string '141657' (length=6)
  42 => string '2000000041' (length=10)
  43 => string '' (length=0)
  44 => string '' (length=0)
  45 => string '' (length=0)
  46 => string 'RUA' (length=3)
  47 => string '#redacted#' (length=15)
  48 => string '#redacted#' (length=3)
  49 => string '' (length=0)
  50 => string 'JABAQUARA' (length=9)
  51 => string 'PARI' (length=4)
  52 => string 'SP' (length=2)
  53 => string '#redacted#' (length=7)
  54 => string '' (length=0)
  55 => string '11' (length=2)
  56 => string '#redacted#' (length=9)
  57 => string '#redacted#' (length=9)
  58 => string '#redacted#' (length=5)
  59 => string '#redacted#' (length=12)
  60 => string '' (length=0)
  61 => string 'Centros Automotivos, Mec‰nicas e Oficinas - Leves' (length=49)
  62 => string '' (length=0)
  63 => string '' (length=0)
  64 => string '' (length=0)
  65 => string '' (length=0)
  66 => string '' (length=0)
  67 => string 'Centros Automotivos, MecAnicas e Oficinas - Leves ' (length=50)
  68 => string 'N' (length=1)
  69 => string '0' (length=1)
  70 => string '0' (length=1)
  71 => string '0' (length=1)
  72 => string '' (length=0)
  73 => string '1' (length=1)
  74 => string '0' (length=1)
  75 => string '' (length=0)
  76 => string '
#redacted#' (length=16)
  77 => string '2' (length=1)
  78 => string '0' (length=1)
  79 => string '228109' (length=6)
  80 => string '2000000041' (length=10)
  81 => string '0800 AUTOMOTIVE' (length=15)
  82 => string '#redacted#@hotmail.com' (length=32)
  83 => string '' (length=0)
  84 => string 'RUA' (length=3)
  85 => string '#redacted#' (length=7)
  86 => string '322' (length=3)
  87 => string '' (length=0)
  88 => string '' (length=0)
  89 => string 'SAO PAULO' (length=9)
  90 => string 'SP' (length=2)
  91 => string '#redacted#' (length=7)
  92 => string '' (length=0)
  93 => string '11' (length=2)
  94 => string '#redacted#' (length=9)
  95 => string '#redacted#' (length=9)
  96 => string '#redacted#' (length=40)
  97 => string 'PROPRIETARIO' (length=12)
  98 => string '' (length=0)
  99 => string 'Centros Automotivos, Mec‰nicas e Oficinas - Leves' (length=49)
  100 => string '' (length=0)
  101 => string '' (length=0)
  102 => string '' (length=0)
  103 => string '' (length=0)
  104 => string '' (length=0)
  105 => string 'Centros Automotivos, MecAnicas e Oficinas - Leves ' (length=50)
  106 => string '' (length=0)
  107 => string '0' (length=1)
  108 => string '0' (length=1)
  109 => string '0' (length=1)
  110 => string 'Registro criado' (length=15)

2014-07-18 16:06:26 - 287608 bytes - header displayed
2014-07-18 16:06:26 - 287608 bytes - fetching another line
( ! ) Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 72 bytes) in /home/lqdi/quantum.pranaeditora.com.br/sandbox/test_csv_read.php on line 25
Call Stack
#   Time    Memory  Function    Location
1   0.0011  249048  {main}( )   ../test_csv_read.php:0
2   0.0093  287520  fgetcsv ( ) ../test_csv_read.php:25