使用各种分隔符分析R中的文本文件_R_Parsing

使用各种分隔符分析R中的文本文件

r parsing

使用各种分隔符分析R中的文本文件,r,parsing,R,Parsing,我有一个非常可怕的HTML格式的文本文件： A<b>Metabolism</b> B B <b>Overview</b> C 01200 Carbon metabolism [PATH:bpe01200] D BP3142 pgi; glucose-6-phosphate isomerase K01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9] D BP197

我有一个非常可怕的HTML格式的文本文件：

A<b>Metabolism</b>
B
B  <b>Overview</b>
C    01200 Carbon metabolism [PATH:bpe01200]
D      BP3142 pgi; glucose-6-phosphate isomerase    K01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]
D      BP1971 pgi; glucose-6-phosphate isomerase    K01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]
D      BP1519 fba; fructose-1,6-bisphosphate aldolase   K01624 FBA; fructose-bisphosphate aldolase, class II [EC:4.1.2.13]
D      BP0801 tpiA; triosephosphate isomerase   K01803 TPI; triosephosphate isomerase (TIM) [EC:5.3.1.1]
D      BP1000 gap; glyceraldehyde-3-phosphate dehydrogenase K00134 GAPDH; glyceraldehyde 3-phosphate dehydrogenase [EC:1.2.1.12]

问题是分隔符在行的每个部分都会发生变化。它似乎遵循这种模式 e、 g

我可以想出一种不太聪明的方法，一次解析一个分隔符。但有人有什么聪明的解决方案吗？或者知道一个工具可以很好地解释这一点

我非常感谢您的帮助：

谢谢，这是一个更简单的版本，只在一个匹配中使用相同的正则表达式字符串提取细节

text <- "
A<b>Metabolism</b>
B
B  <b>Overview</b>
C    01200 Carbon metabolism [PATH:bpe01200]
D      BP3142 pgi; glucose-6-phosphate isomerase    K01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]
D      BP1971 pgi; glucose-6-phosphate isomerase    K01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]
D      BP1519 fba; fructose-1,6-bisphosphate aldolase   K01624 FBA; fructose-bisphosphate aldolase, class II [EC:4.1.2.13]
D      BP0801 tpiA; triosephosphate isomerase   K01803 TPI; triosephosphate isomerase (TIM) [EC:5.3.1.1]
D      BP1000 gap; glyceraldehyde-3-phosphate dehydrogenase K00134 GAPDH; glyceraldehyde 3-phosphate dehydrogenase [EC:1.2.1.12]
"

library(stringr)

# get the detail items (liens beginning with D blank)
details <- str_match_all(text, "D\\s+(.+)\n")[[1]][,2]

details
pattern <- "([^\\s]+)\\s([^\\s]+);(.*)\\s([^\\s]+)\\s([^\\s]+);\\s(.*)\\s([^\\s]+)$"
trimws(str_match(details, pattern)[,-1])

#[,1]     [,2]   [,3]                                       [,4]     [,5]   
#[1,] "BP3142" "pgi"  "glucose-6-phosphate isomerase"            "K01810" "GPI"  
#[2,] "BP1971" "pgi"  "glucose-6-phosphate isomerase"            "K01810" "GPI"  
#[3,] "BP1519" "fba"  "fructose-1,6-bisphosphate aldolase"       "K01624" "FBA"  
#[4,] "BP0801" "tpiA" "triosephosphate isomerase"                "K01803" "TPI"  
#[5,] "BP1000" "gap"  "glyceraldehyde-3-phosphate dehydrogenase" "K00134" "GAPDH"
#               [,6]                                       [,7]           
#[1,] "glucose-6-phosphate isomerase"            "[EC:5.3.1.9]" 
#[2,] "glucose-6-phosphate isomerase"            "[EC:5.3.1.9]" 
#[3,] "fructose-bisphosphate aldolase, class II" "[EC:4.1.2.13]"
#[4,] "triosephosphate isomerase (TIM)"          "[EC:5.3.1.1]" 
#[5,] "glyceraldehyde 3-phosphate dehydrogenase" "[EC:1.2.1.12]"

还有一个更简单的版本，只在一个匹配中使用相同的正则表达式字符串来提取细节

text <- "
A<b>Metabolism</b>
B
B  <b>Overview</b>
C    01200 Carbon metabolism [PATH:bpe01200]
D      BP3142 pgi; glucose-6-phosphate isomerase    K01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]
D      BP1971 pgi; glucose-6-phosphate isomerase    K01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]
D      BP1519 fba; fructose-1,6-bisphosphate aldolase   K01624 FBA; fructose-bisphosphate aldolase, class II [EC:4.1.2.13]
D      BP0801 tpiA; triosephosphate isomerase   K01803 TPI; triosephosphate isomerase (TIM) [EC:5.3.1.1]
D      BP1000 gap; glyceraldehyde-3-phosphate dehydrogenase K00134 GAPDH; glyceraldehyde 3-phosphate dehydrogenase [EC:1.2.1.12]
"

library(stringr)

# get the detail items (liens beginning with D blank)
details <- str_match_all(text, "D\\s+(.+)\n")[[1]][,2]

details
pattern <- "([^\\s]+)\\s([^\\s]+);(.*)\\s([^\\s]+)\\s([^\\s]+);\\s(.*)\\s([^\\s]+)$"
trimws(str_match(details, pattern)[,-1])

#[,1]     [,2]   [,3]                                       [,4]     [,5]   
#[1,] "BP3142" "pgi"  "glucose-6-phosphate isomerase"            "K01810" "GPI"  
#[2,] "BP1971" "pgi"  "glucose-6-phosphate isomerase"            "K01810" "GPI"  
#[3,] "BP1519" "fba"  "fructose-1,6-bisphosphate aldolase"       "K01624" "FBA"  
#[4,] "BP0801" "tpiA" "triosephosphate isomerase"                "K01803" "TPI"  
#[5,] "BP1000" "gap"  "glyceraldehyde-3-phosphate dehydrogenase" "K00134" "GAPDH"
#               [,6]                                       [,7]           
#[1,] "glucose-6-phosphate isomerase"            "[EC:5.3.1.9]" 
#[2,] "glucose-6-phosphate isomerase"            "[EC:5.3.1.9]" 
#[3,] "fructose-bisphosphate aldolase, class II" "[EC:4.1.2.13]"
#[4,] "triosephosphate isomerase (TIM)"          "[EC:5.3.1.1]" 
#[5,] "glyceraldehyde 3-phosphate dehydrogenase" "[EC:1.2.1.12]"

如果将一行中的所有分隔符转换为同一个分隔符，例如，将制表符、空格和分号转换为具有全局查找和替换的tab，则只需处理一个分隔符，这应该很容易。注意：我从未使用过R，但这种技术非常简单，可以在许多系统中使用。由于我不使用R，所以我不会将其作为答案发布，但如果您测试了它，并且它有效，并且您希望将其作为答案发布，请告诉我，我会将其作为答案发布。我会使用正则表达式。反馈良好。然后更改分隔符，使带空格的值用引号括起来。用于起始分隔符；更改为制表符和结束分隔符将制表符更改为制表符。这应该在引用的文本中用空格将文本括起来，并将它们放在一起。您还可以将数据转换为带有分隔符的格式，并围绕所有值。如果您计划经常这样做，那么我同意Apom的观点，即使用正则表达式是一种可行的方法。将这些步骤视为概念证明，然后一旦知道它按预期工作，就转换为正则表达式。我发现这样做更快，因为您不必尝试调试正则表达式，同时还可以了解数据是否可以转换。编辑早期注释。事实上，我经常这样做，也会遇到同样的问题。不过，为了避免这种情况，我在本地计算机上使用MicrosoftWord，而不是在服务器上使用，因为我使用的文件相当大。我所做的是如上所述和您所做的转换，但在Word中，将数据转换为具有选定分隔符的表，这会将具有问题空间的所有数据放在同一列中。Word允许我仅对该列执行查找和替换操作，我只需将空格更改为制表符，然后再转换回文本。如果将一行中的所有分隔符转换为同一分隔符，例如制表符、空格和分号，则只需使用全局查找和替换的制表符，然后，您只需要处理一个分隔符，这应该很容易。注意：我从未使用过R，但这种技术非常简单，可以在许多系统中使用。由于我不使用R，所以我不会将其作为答案发布，但如果您测试了它，并且它有效，并且您希望将其作为答案发布，请告诉我，我会将其作为答案发布。我会使用正则表达式。反馈良好。然后更改分隔符，使带空格的值用引号括起来。用于起始分隔符；更改为制表符和结束分隔符将制表符更改为制表符。这应该在引用的文本中用空格将文本括起来，并将它们放在一起。您还可以将数据转换为带有分隔符的格式，并围绕所有值。如果您计划经常这样做，那么我同意Apom的观点，即使用正则表达式是一种可行的方法。将这些步骤视为概念证明，然后一旦知道它按预期工作，就转换为正则表达式。我发现这样做更快，因为您不必尝试调试正则表达式，同时还可以了解数据是否可以转换。编辑早期注释。事实上，我经常这样做，也会遇到同样的问题。不过，为了避免这种情况，我在本地计算机上使用MicrosoftWord，而不是在服务器上使用，因为我使用的文件相当大。我所做的是如上所述和您所做的转换，但在Word中，将数据转换为具有选定分隔符的表，这会将具有问题空间的所有数据放在同一列中。Word允许我在该列中进行查找和替换，我只需将空格更改为tab，然后再转换回文本。我将添加注释。这个例子应该有用。至少在我的机器上是这样。哇！完全震惊于你的承诺水平！我来看看！你有一个我可以用来测试正则表达式的网站吗？特定于R？我找不到一个具体的答案：我在答案中添加了最终输出的标题。一个人应该能够在自己的机器上以复制粘贴的方式运行我的代码，如果它不起作用，请告诉我谢谢您的示例。我这样问是因为我必须安装R才能看到它的样子，在看到示例后，它澄清了我对代码的期望。我将添加注释。这个例子应该有用。至少在我的机器上是这样。哇！完全震惊于你的承诺水平！我来看看！你有一个我可以用来测试正则表达式的网站吗

在…上特定于R？我找不到一个具体的答案：我在答案中添加了最终输出的标题。一个人应该能够在自己的机器上以复制粘贴的方式运行我的代码，如果它不起作用，请告诉我谢谢您的示例。我这样问是因为我必须安装R才能看到它的样子，在看到示例后，它澄清了我对代码的期望。

library(stringr)
library(purrr)
file <- "A<b>Metabolism</b>
B
B  <b>Overview</b>
C\t01200 Carbon metabolism [PATH:bpe01200]
D\tBP3142 pgi; glucose-6-phosphate isomerase\tK01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]
D\tBP1971 pgi; glucose-6-phosphate isomerase\tK01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]
D\tBP1519 fba; fructose-1,6-bisphosphate aldolase\tK01624 FBA; fructose-bisphosphate aldolase, class II [EC:4.1.2.13]
D\tBP0801 tpiA; triosephosphate isomerase\tK01803 TPI; triosephosphate isomerase (TIM) [EC:5.3.1.1]
D\tBP1000 gap; glyceraldehyde-3-phosphate dehydrogenase\tK00134 GAPDH; glyceraldehyde 3-phosphate dehydrogenase [EC:1.2.1.12]
This line is to check behavior when parsing fails."
cat(file)
data <- readLines(con = textConnection(file))
# Pattern to capture "A<b>Metabolism</b>" for instance
pattern_1 <- "^(\\w+)\\h*<b>\\h*(\\w+)\\h*</b>\\h*$"
# Pattern to capture "B" for instance
pattern_2 <- "^(\\w+)$"
# Pattern to capture "C\t01200 Carbon metabolism [PATH:bpe01200]" for instance
pattern_3 <- "^(\\w+)\\t+(\\w+)\\s+([^\\[\\t;]*)\\h*(\\[[^\\]]*\\])$"
# Pattern to capture "D\tBP3142 pgi; glucose-6-phosphate isomerase\tK01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]" for instance
pattern_4 <- "^(\\w+)\\t+(\\w+)\\s+(\\w+);\\h*([^\\t]*)\\t+(\\w+)\\s+(\\w+);\\h*([^\\[]*)\\h*(\\[[^\\]]*\\])$"
# Some more explanations:
# Parens wrap groups to extract
# "\\w+" matches words
# "\\t+", "\\s+" or ";\\h*" are specific separators of OP's original data
# "([^\\t]*)" matches anything until the next tab separator
# Convoluted patterns such as "(\\[[^\\]]*\\])" extract whatever is inside brackets
patterns <- mget(paste0("pattern_", 1:4))
# A list of the data parsed 4 times, once for each pattern:
patterns %>% 
  map(~ {
    extraction <- str_match(data, .x)
    cbind(match = !is.na(extraction[, 1]), extraction[, - 1])
  })
# This is closer to your desired output: a list of [un]parsed rows:
data %>%
  map(~ {
    # Find the first pattern that matches. 0 if none does
    pattern_index <- detect_index(patterns, grepl, .x, perl = TRUE)
    # If failed to parse, return original row as length 1 character vector. Else return parsed row as character vector
    if (pattern_index == 0L) .x else str_match(.x, get(paste0("pattern_", pattern_index)))[- 1]
  })

list(c("A", "Metabolism"), "B", c("B", "Overview"), c("C", "01200", 
"Carbon metabolism ", "[PATH:bpe01200]"), c("D", "BP3142", "pgi", 
"glucose-6-phosphate isomerase", "K01810", "GPI", "glucose-6-phosphate isomerase ", 
"[EC:5.3.1.9]"))

text <- "
A<b>Metabolism</b>
B
B  <b>Overview</b>
C    01200 Carbon metabolism [PATH:bpe01200]
D      BP3142 pgi; glucose-6-phosphate isomerase    K01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]
D      BP1971 pgi; glucose-6-phosphate isomerase    K01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]
D      BP1519 fba; fructose-1,6-bisphosphate aldolase   K01624 FBA; fructose-bisphosphate aldolase, class II [EC:4.1.2.13]
D      BP0801 tpiA; triosephosphate isomerase   K01803 TPI; triosephosphate isomerase (TIM) [EC:5.3.1.1]
D      BP1000 gap; glyceraldehyde-3-phosphate dehydrogenase K00134 GAPDH; glyceraldehyde 3-phosphate dehydrogenase [EC:1.2.1.12]
"
library(stringr)

# get the header items (beginning with C blank) 
headers <- str_match(text, "C\\s+(.+)\n")[,2]
header_items <- trimws(str_match(headers, "(\\d+)\\s+([^\\[]+)(.+)")[2:4]) 

# get the detail items (liens beginning with D blank)
details <- str_match_all(text, "D\\s+(.+)\n")[[1]][,2]

# parse each item within detail 

# split on ";" and organize into dataframe
items <- as.data.frame(t(data.frame(
  str_split(details,";\\s")
)), row.names = 1:length(details), stringsAsFactors = FALSE)

# parse each part using pattern matches

# capture () beginning of string ^ and all characters not whitespace [^\\s]+
items$V1A <- str_match(items$V1,"(^[^\\s]+)")[,2]

# capture () end of string $ and a non-whitespace sequence [^\\s]+
items$V1B <- str_match(items$V1,"([^\\s]+)$")[,2]

# capture () beginning of string exluding two non-whitespace sequences [^\\s]+ at end $
items$V2A <- str_match(items$V2,"^(.+)\\s[^\\s]+\\s[^\\s]+$")[,2]

# capture () non-whitespace sequence [^\\s]+ at end of string $
items$V2C <- str_match(items$V2,"([^\\s]+)$")[,2]

# capture () second to last non-whitespace sequence [^\\s]+ at end of string $ 
items$V2B <- str_match(items$V2,"([^\\s]+)\\s[^\\s]+$")[,2]

# capture () begining of string ^ excluding last non-whitespace sequence [^\\s]+
items$V3A <- str_match(items$V3,"^(.+)\\s[^\\s]+$")[,2]

# capture () non-whitespace sequence at end $
items$V3B <- str_match(items$V3,"([^\\s]+)$")[,2]

select & reorder
items <- items[, c("V1A", "V1B", "V2A", "V2B", "V2C", "V3A", "V3B")]

#     V1A  V1B                                      V2A    V2B   V2C                                      V3A           V3B
#1 BP3142  pgi         glucose-6-phosphate isomerase    K01810   GPI            glucose-6-phosphate isomerase  [EC:5.3.1.9]
#2 BP1971  pgi         glucose-6-phosphate isomerase    K01810   GPI            glucose-6-phosphate isomerase  [EC:5.3.1.9]
#3 BP1519  fba     fructose-1,6-bisphosphate aldolase   K01624   FBA fructose-bisphosphate aldolase, class II [EC:4.1.2.13]
#4 BP0801 tpiA              triosephosphate isomerase   K01803   TPI          triosephosphate isomerase (TIM)  [EC:5.3.1.1]
#5 BP1000  gap glyceraldehyde-3-phosphate dehydrogenase K00134 GAPDH glyceraldehyde 3-phosphate dehydrogenase [EC:1.2.1.12]

text <- "
A<b>Metabolism</b>
B
B  <b>Overview</b>
C    01200 Carbon metabolism [PATH:bpe01200]
D      BP3142 pgi; glucose-6-phosphate isomerase    K01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]
D      BP1971 pgi; glucose-6-phosphate isomerase    K01810 GPI; glucose-6-phosphate isomerase [EC:5.3.1.9]
D      BP1519 fba; fructose-1,6-bisphosphate aldolase   K01624 FBA; fructose-bisphosphate aldolase, class II [EC:4.1.2.13]
D      BP0801 tpiA; triosephosphate isomerase   K01803 TPI; triosephosphate isomerase (TIM) [EC:5.3.1.1]
D      BP1000 gap; glyceraldehyde-3-phosphate dehydrogenase K00134 GAPDH; glyceraldehyde 3-phosphate dehydrogenase [EC:1.2.1.12]
"

library(stringr)

# get the detail items (liens beginning with D blank)
details <- str_match_all(text, "D\\s+(.+)\n")[[1]][,2]

details
pattern <- "([^\\s]+)\\s([^\\s]+);(.*)\\s([^\\s]+)\\s([^\\s]+);\\s(.*)\\s([^\\s]+)$"
trimws(str_match(details, pattern)[,-1])

#[,1]     [,2]   [,3]                                       [,4]     [,5]   
#[1,] "BP3142" "pgi"  "glucose-6-phosphate isomerase"            "K01810" "GPI"  
#[2,] "BP1971" "pgi"  "glucose-6-phosphate isomerase"            "K01810" "GPI"  
#[3,] "BP1519" "fba"  "fructose-1,6-bisphosphate aldolase"       "K01624" "FBA"  
#[4,] "BP0801" "tpiA" "triosephosphate isomerase"                "K01803" "TPI"  
#[5,] "BP1000" "gap"  "glyceraldehyde-3-phosphate dehydrogenase" "K00134" "GAPDH"
#               [,6]                                       [,7]           
#[1,] "glucose-6-phosphate isomerase"            "[EC:5.3.1.9]" 
#[2,] "glucose-6-phosphate isomerase"            "[EC:5.3.1.9]" 
#[3,] "fructose-bisphosphate aldolase, class II" "[EC:4.1.2.13]"
#[4,] "triosephosphate isomerase (TIM)"          "[EC:5.3.1.1]" 
#[5,] "glyceraldehyde 3-phosphate dehydrogenase" "[EC:1.2.1.12]"