C++ 在C+中拆分字符串+；当线条的长度大致相同时，所需的时间越来越多

对于我的项目，我需要读取和处理一个包含地震接收器能量的大文件。出于通用性目的，它必须能够处理.dat和.segy文件。我的问题是

.dat

文件。我当前的实现在

'\t'

字符处拆分字符串，将匹配项放入子字符串中，并将值作为浮点值推送到

std:：vector

。然后从行中删除子字符串和制表符，并搜索下一个值。见下文：

std::vector<float> parseLine(std::string& number, std::ifstream& file)
{
    getline(file, number); // read the number
    std::vector<float> datalist = selectData(number);

    //for (auto y : datalist) std::cout << y << " ";
    //std::cout << std::endl;
    return datalist;

}


std::vector<float> selectData(std::string& line)
{
    std::vector<float> returnVec;
    //auto parsing_start = std::chrono::high_resolution_clock::now();

    // The question is about this part
    while (true)
    {
        int index = line.find_first_of("\t");
        std::string match = line.substr(0, index);
        if (!line.empty()) {
            returnVec.push_back(std::stof(match));
            line.erase(0, match.length());
        }
        if (line[0] == '\t') line.erase(0,1);
        if (line.empty()) {
            //std::cout << "line is empty" << std::endl; 
            break;
        }

    }   
    return returnVec;
}

当然是4000列，而不是6列

以下是主要功能，以及我如何测量时间和

\include

s：

#include <stdio.h>
#include <fstream>
#include <string>
#include <iostream>
#define _SILENCE_EXPERIMENTAL_FILESYSTEM_DEPRECATION_WARNING
#include <experimental/filesystem>
#include <regex>
#include <iterator>
#include <chrono>
#include <Eigen/Dense>
#include "readSeis.h"

MatrixXf extractSeismics(std::string file)
{
    MatrixXf M;

    auto start = std::chrono::high_resolution_clock::now();
    auto interstart = std::chrono::high_resolution_clock::now();
    checkExistence(file);
    std::ifstream myfile(file);
    if (!myfile)
    {
        std::cout << "Could not open file " << file << std::endl;
        exit(1);
    }
    int skipCols = 2; // I don't need the coordinates now
    size_t linecount = 0;
    size_t colcount = 0;
    while (!myfile.eof()) // while not at End Of File (eof)
    {
        std::string number;
        std::vector<float> data = parseLine(number, myfile);
        if (linecount == 0)  colcount = data.size() - skipCols;
        //auto resize_start = std::chrono::high_resolution_clock::now();
        M.conservativeResize(linecount + 1, colcount); // preserves old values :)
        //printElapsedTime(resize_start);
        for (int i = skipCols; i < data.size(); i++)
        {
            M(linecount, i - skipCols) = data[i];

        }
        linecount++;
        // Measure interval time
        if (linecount % 100 == 0)
        {
            std::cout << "Parsing line " << linecount << ", ";
            printElapsedTime(interstart);
            interstart = std::chrono::high_resolution_clock::now();
        }
    }
    myfile.close();
    printElapsedTime(start);
    return M;

}

---

有人知道为什么会这样吗？非常感谢。：）

下面是一些代码，可以大致按照您描述的方式读取文件。它一次读取一行，解析出一行中的浮点数，跳过前N列，并将其余列放入

向量中。main函数将每一行存储到一个
向量中，并且（为了确保其余的不会被优化），将它读取的所有值相加，最后打印出来

#include <iostream>
#include <sstream>
#include <vector>
#include <iterator>
#include <numeric>
#include <fstream>

std::vector<float> selectData(std::string const &line, int skip_count) { 
    std::istringstream buffer(line);

    float ignore;
    for (int i=0; i<skip_count; i++)
        buffer >> ignore;

    return std::vector<float>{std::istream_iterator<float>(buffer), {}};
}

int main(int argc, char **argv) { 
    std::string line;
    float total = 0.0f;

    if (argc != 2) {
        std::cerr << "Usage: accum <infile>\n";
        return EXIT_FAILURE;
    }

    std::vector<std::vector<float>> matrix;

    std::ifstream infile(argv[1]);
    while (std::getline(infile, line)) {
        auto vals = selectData(line, 2);
        matrix.push_back(vals);
        total += std::accumulate(vals.begin(), vals.end(), 0.0f);
    }
    std::cout << "total: " <<total << "\n";
}

#包括
#包括
#包括
#包括
#包括
#包括
std:：vector selectData（std:：string const&line，int skip_count）{
std:：istringstream缓冲区（行）；
浮动忽略；
对于（int i=0；i>ignore；
返回std:：vector{std:：istream_迭代器（缓冲区），{}；
}
intmain（intargc，字符**argv）{
std：：字符串行；
总浮点数=0.0f；
如果（argc！=2）{
std:：cerr听起来像是在那里的某个地方。我还猜测，将值添加到M中才是真正的罪魁祸首。您可能应该使用所选的探查器进行查看，或者在extractSeismics
中注释掉for循环，而（！myfile.eof（））
。其次，如果调用
erase（）
执行解析。为什么要擦除部分行？（请注意，这样做的目的是：字符串的整个部分必须“向下”移动，以掩盖erase（）调用的漏洞）。最后，请发布您的编译器设置，更重要的是，您在构建应用程序时使用的优化设置。如果我不得不猜测，我会怀疑
M.conservativeResize
call。我打赌它在矩阵的当前大小中是线性的。这与字符串解析无关；每一行的大小可能相似，so当你做得效率低下时，整个过程都应该是同样的效率低下。我和@IgorTandetnik一起认为，可能的罪魁祸首是随着你的发展而不断增长的东西，比如
m.conservativeResize
call；名字像“conservative”，它可能是调整所需的最小金额，而不是超额分配；超额分配倍数将在每次扩展中摊销
O（1）
，最小调整大小将
O（n）
每次扩展。好的，我知道它是如何工作的。这将执行时间从38分钟减少到73.4秒，已经好得多了！我将看看是否可以改进更多。它可能会在我的工作电脑上运行得更快（73秒在我的中等规格笔记本电脑上）
testSeis1500_1510_290_832.dat exists, continuing program
Parsing line 100, Execution time : 1204968 Microseconds
Parsing line 200, Execution time : 1971723 Microseconds
Parsing line 300, Execution time : 2727474 Microseconds
Parsing line 400, Execution time : 3640131 Microseconds
Parsing line 500, Execution time : 4392584 Microseconds
Parsing line 600, Execution time : 5150465 Microseconds
Parsing line 700, Execution time : 5944256 Microseconds
Parsing line 800, Execution time : 6680841 Microseconds
Parsing line 900, Execution time : 7456237 Microseconds
Parsing line 1000, Execution time : 8201579 Microseconds
Parsing line 1100, Execution time : 8999075 Microseconds
Parsing line 1200, Execution time : 9860883 Microseconds
Parsing line 1300, Execution time : 10524525 Microseconds
Parsing line 1400, Execution time : 11286452 Microseconds
Parsing line 1500, Execution time : 12134566 Microseconds
Parsing line 1600, Execution time : 12872876 Microseconds
Parsing line 1700, Execution time : 13815265 Microseconds
Parsing line 1800, Execution time : 14528233 Microseconds
Parsing line 1900, Execution time : 15221609 Microseconds
Parsing line 2000, Execution time : 15989419 Microseconds
Parsing line 2100, Execution time : 16850944 Microseconds
Parsing line 2200, Execution time : 17717721 Microseconds
Parsing line 2300, Execution time : 18318276 Microseconds
Parsing line 2400, Execution time : 19286148 Microseconds
Parsing line 2500, Execution time : 19828358 Microseconds
Parsing line 2600, Execution time : 20678683 Microseconds
Parsing line 2700, Execution time : 21648089 Microseconds
Parsing line 2800, Execution time : 22229266 Microseconds
Parsing line 2900, Execution time : 23398151 Microseconds
Parsing line 3000, Execution time : 23915173 Microseconds
Parsing line 3100, Execution time : 24523879 Microseconds
Parsing line 3200, Execution time : 25547811 Microseconds
Parsing line 3300, Execution time : 26087140 Microseconds
Parsing line 3400, Execution time : 26991734 Microseconds
Parsing line 3500, Execution time : 27795577 Microseconds
Parsing line 3600, Execution time : 28367321 Microseconds
Parsing line 3700, Execution time : 29127089 Microseconds
Parsing line 3800, Execution time : 29998775 Microseconds
Parsing line 3900, Execution time : 30788170 Microseconds
Parsing line 4000, Execution time : 31456488 Microseconds
Parsing line 4100, Execution time : 32458102 Microseconds
Parsing line 4200, Execution time : 33345031 Microseconds
Parsing line 4300, Execution time : 33853183 Microseconds
Parsing line 4400, Execution time : 34676522 Microseconds
Parsing line 4500, Execution time : 35593187 Microseconds
Parsing line 4600, Execution time : 37059032 Microseconds
Parsing line 4700, Execution time : 37118954 Microseconds
Parsing line 4800, Execution time : 37824417 Microseconds
Parsing line 4900, Execution time : 38756924 Microseconds
Parsing line 5000, Execution time : 39446184 Microseconds
Parsing line 5100, Execution time : 40194553 Microseconds
Parsing line 5200, Execution time : 41051359 Microseconds
Parsing line 5300, Execution time : 41498345 Microseconds
Parsing line 5400, Execution time : 42524946 Microseconds
Parsing line 5500, Execution time : 43252436 Microseconds
Parsing line 5600, Execution time : 44145627 Microseconds
Parsing line 5700, Execution time : 45081208 Microseconds
Parsing line 5800, Execution time : 46072319 Microseconds
Parsing line 5900, Execution time : 46603417 Microseconds
Execution time : 1442777428 Microseconds

#include <iostream>
#include <sstream>
#include <vector>
#include <iterator>
#include <numeric>
#include <fstream>

std::vector<float> selectData(std::string const &line, int skip_count) { 
    std::istringstream buffer(line);

    float ignore;
    for (int i=0; i<skip_count; i++)
        buffer >> ignore;

    return std::vector<float>{std::istream_iterator<float>(buffer), {}};
}

int main(int argc, char **argv) { 
    std::string line;
    float total = 0.0f;

    if (argc != 2) {
        std::cerr << "Usage: accum <infile>\n";
        return EXIT_FAILURE;
    }

    std::vector<std::vector<float>> matrix;

    std::ifstream infile(argv[1]);
    while (std::getline(infile, line)) {
        auto vals = selectData(line, 2);
        matrix.push_back(vals);
        total += std::accumulate(vals.begin(), vals.end(), 0.0f);
    }
    std::cout << "total: " <<total << "\n";
}