Python 熊猫读取\u csv忽略最后一列中的分隔符

我有一个具有以下结构的文件（第一行是标题，文件名是

test.dat

）：

如何指示pandas将整个站点名称（在本例中为加拿大努纳武特的Alert）作为单个元素读取？我在代码中使用了

delim_whitespace=True

，但这不起作用，因为电台名称包含空格字符

跑步：

import pandas as pd
test = pd.read_csv('./test.dat', delim_whitespace=True, header=1)
print(test.to_string())

                   ID_OBS    LAT LON  ALT   TP    TO  LT_min    LT_max STATIONNAME
ALT_NOA_000 82.45  -62.52  210.0  FM    0  0.0  24.0  Alert,  Nunavut,      Canada

import re

def parse_my_file(filename):
    with open(filename) as f:
        for line in f:
            yield re.split(r'\s+', line.strip(), 8)

# build the generator        
my_parser = parse_my_file('test.dat')

# first element returned is the columns
columns = next(my_parser)

# build the data frame
df = pd.DataFrame(my_parser, columns=columns)
print(df)

        ID_OBS    LAT     LON    ALT  TP TO LT_min LT_max  \
0  ALT_NOA_000  82.45  -62.52  210.0  FM  0    0.0   24.0   

              STATIONNAME  
0  Alert, Nunavut, Canada 

产生：

import pandas as pd
test = pd.read_csv('./test.dat', delim_whitespace=True, header=1)
print(test.to_string())

                   ID_OBS    LAT LON  ALT   TP    TO  LT_min    LT_max STATIONNAME
ALT_NOA_000 82.45  -62.52  210.0  FM    0  0.0  24.0  Alert,  Nunavut,      Canada

import re

def parse_my_file(filename):
    with open(filename) as f:
        for line in f:
            yield re.split(r'\s+', line.strip(), 8)

# build the generator        
my_parser = parse_my_file('test.dat')

# first element returned is the columns
columns = next(my_parser)

# build the data frame
df = pd.DataFrame(my_parser, columns=columns)
print(df)

        ID_OBS    LAT     LON    ALT  TP TO LT_min LT_max  \
0  ALT_NOA_000  82.45  -62.52  210.0  FM  0    0.0   24.0   

              STATIONNAME  
0  Alert, Nunavut, Canada 

---

快速阅读教程没有帮助。我在这里遗漏了什么？

我经常通过编写自己的小解析器来解决这些问题。总的来说，有很多方法可以让熊猫服从你的意愿，但我发现这种方法通常更容易：

代码：

import pandas as pd
test = pd.read_csv('./test.dat', delim_whitespace=True, header=1)
print(test.to_string())

                   ID_OBS    LAT LON  ALT   TP    TO  LT_min    LT_max STATIONNAME
ALT_NOA_000 82.45  -62.52  210.0  FM    0  0.0  24.0  Alert,  Nunavut,      Canada

import re

def parse_my_file(filename):
    with open(filename) as f:
        for line in f:
            yield re.split(r'\s+', line.strip(), 8)

# build the generator        
my_parser = parse_my_file('test.dat')

# first element returned is the columns
columns = next(my_parser)

# build the data frame
df = pd.DataFrame(my_parser, columns=columns)
print(df)

        ID_OBS    LAT     LON    ALT  TP TO LT_min LT_max  \
0  ALT_NOA_000  82.45  -62.52  210.0  FM  0    0.0   24.0   

              STATIONNAME  
0  Alert, Nunavut, Canada 

结果：

import pandas as pd
test = pd.read_csv('./test.dat', delim_whitespace=True, header=1)
print(test.to_string())

                   ID_OBS    LAT LON  ALT   TP    TO  LT_min    LT_max STATIONNAME
ALT_NOA_000 82.45  -62.52  210.0  FM    0  0.0  24.0  Alert,  Nunavut,      Canada

import re

def parse_my_file(filename):
    with open(filename) as f:
        for line in f:
            yield re.split(r'\s+', line.strip(), 8)

# build the generator        
my_parser = parse_my_file('test.dat')

# first element returned is the columns
columns = next(my_parser)

# build the data frame
df = pd.DataFrame(my_parser, columns=columns)
print(df)

        ID_OBS    LAT     LON    ALT  TP TO LT_min LT_max  \
0  ALT_NOA_000  82.45  -62.52  210.0  FM  0    0.0   24.0   

              STATIONNAME  
0  Alert, Nunavut, Canada 

---

粘贴的示例文件有点模棱两可：例如，无法通过肉眼判断看起来像几个空格的内容是否是选项卡

不过，一般来说，请注意，普通的老Python比Pandas或CSV模块（Pandas的优势在于Elsewhere）更具表现力。例如，甚至还有用于递归下降解析器的Python模块，熊猫显然缺乏这些模块。您可以使用常规Python将文件处理成更容易解析的形式。例如：

import re
>>> ['@'.join(re.split(r'[ \t]+', l.strip(), maxsplit=8)) for l in open('stuff.tsv') if l.strip()]
['ID_OBS@LAT@LON@ALT@TP@TO@LT_min@LT_max@STATIONNAME',
 'ALT_NOA_000@82.45@-62.52@210.0@FM@0@0.0@24.0@Alert, Nunavut, Canada']

将分隔符更改为

'@'

，例如，如果您回写到文件，则可以使用

delimiter='@'

进行分析