从结构不良的XML文件中提取数据

我正在处理生理数据，并且有非常大、格式很差的XML文件，其中包含分析所需的所有原始数据。我们以前在autohotkey中使用过regex风格的方法来提取数据，但它的性能很差。我一直在寻找解析或提取数据的方法 有效地从这些文件中删除

我遇到了XML文件结构无效或无法构建一个好的sed命令来提取数据并按照我们需要的方式将其拼凑在一起的问题。也就是说，数据结构的总体可变性最小，因此我认为可以使用regex有效地从文件中提取数据。到目前为止，我一直在尝试使用grep/sed和各种XML解析工具

Input: XML (poorly structured, 500-1500 mb)

Desired Output: CSV

VitalSigns:

1 VitalSigns, Date, Time, Par, Value
2 VitalSigns, Date, Time, Par, Value
...
end

Waveforms:

1 Waveforms, Date, Time, Waveform Channel, value 1, value 2, ... , value 480
2 Waveforms, Date, Time, Waveform Channel, value 1, value 2, ... , value 480
...
end

以下是XML文件的示例：

---

正如其他评论者所指出的，XML工具更适合此任务。但是它也可以在

sed

中完成，下面是GNU-sed脚本，它可以实现您想要的功能：

script.sed

# concatenate VitalSigns or Waveforms, the date, the time into the holdspace
# on finding a Par, Value or WaveformData, further append the info to the holdspace
/<VitalSigns|Waveforms /  { s/^[^<]*<([^ ]+) Time="([^ ]+) ([^"]+)".*$/\1, \2, \3, /; h; } 
/<Par>.*<\/Par>/          { s/^[^>]+>([^<]*).*$/\1, /; H; }
/<Value.*<\/Value>/       { s/^[^>]+>([^<]*).*$/\1/; 
                            H; 
                            # now print Holdspace after removing newlines
                            x;
                            s/, \n/, /g;
                            p;
                            # now put only the first three columns back into Hold space
                            s/^(([^,]+, ){3}).*$/\1/;
                            x;
                          }

/<WaveformData /          { s/^.*WaveformData Channel="([^"]+)[^>]*>(.*)<\/WaveformData>.*$/\1, \2/; 
                            H;
                            # now print Holdspace after removing newlines
                            x;
                            s/, \n/, /g;
                            p;
                            # now put only the first three columns back into Hold space
                            s/^(([^,]+, ){3}).*$/\1/;
                            x;
                          }

#将生命信号或波形、日期和时间连接到保持空间中
在找到PAR、Valm或ValfDATA数据时，进一步将信息附加到HOLDDACE中。
/为什么不使用专门的工具来解析xml或使用xslt？@JoseRicardoBustosM。这个问题确实表明该文件不是有效的XML。除了缺少根元素之外，您的示例看起来还不错。我建议先解决XML的任何问题，然后使用合适的工具，如
xmlstarlet
或
xsltproc
。
VitalSigns, 01/27/2017, 10:33:04 PM, PVC, 0    
VitalSigns, 01/27/2017, 10:33:04 PM, RESP, 68    
VitalSigns, 01/27/2017, 10:33:04 PM, SPO2, 100    
Waveforms, 01/27/2017, 10:33:04 PM, I, 347, 342, ..., end    
Waveforms, 01/27/2017, 10:33:04 PM, II, 347, 342, ..., end    
Waveforms, 01/27/2017, 10:33:04 PM, SPO2, 370, 741, ..., end    
VitalSigns, 01/27/2017, 10:33:06 PM, PVC, 0    
VitalSigns, 01/27/2017, 10:33:06 PM, RESP, 55
VitalSigns, 01/27/2017, 10:33:06 PM, SPO2, 99
...
end of file

# concatenate VitalSigns or Waveforms, the date, the time into the holdspace
# on finding a Par, Value or WaveformData, further append the info to the holdspace
/<VitalSigns|Waveforms /  { s/^[^<]*<([^ ]+) Time="([^ ]+) ([^"]+)".*$/\1, \2, \3, /; h; } 
/<Par>.*<\/Par>/          { s/^[^>]+>([^<]*).*$/\1, /; H; }
/<Value.*<\/Value>/       { s/^[^>]+>([^<]*).*$/\1/; 
                            H; 
                            # now print Holdspace after removing newlines
                            x;
                            s/, \n/, /g;
                            p;
                            # now put only the first three columns back into Hold space
                            s/^(([^,]+, ){3}).*$/\1/;
                            x;
                          }

/<WaveformData /          { s/^.*WaveformData Channel="([^"]+)[^>]*>(.*)<\/WaveformData>.*$/\1, \2/; 
                            H;
                            # now print Holdspace after removing newlines
                            x;
                            s/, \n/, /g;
                            p;
                            # now put only the first three columns back into Hold space
                            s/^(([^,]+, ){3}).*$/\1/;
                            x;
                          }