R 使用div标记获取级别指示器_R_Selenium_Web Scraping

R 使用div标记获取级别指示器

r selenium web-scraping

R 使用div标记获取级别指示器,r,selenium,web-scraping,R,Selenium,Web Scraping,我正在尝试从中提取文本（单击“打开所有级别”以便更好地理解）我期望的输出是您在一列中看到的表中的所有文本，以及它由多少子级别组成的计数我尝试的动态xpath treeView <- rawdata %>% html_nodes(xpath = '//div//p[contains(@class, "Mui")]') %>% html_text(trim = TRUE) treeView% html_节点（xpath='//div//p[con

我正在尝试从中提取文本（单击“打开所有级别”以便更好地理解）我期望的输出是您在一列中看到的表中的所有文本，以及它由多少子级别组成的计数

我尝试的动态xpath

treeView <- rawdata %>%
  html_nodes(xpath = '//div//p[contains(@class, "Mui")]') %>%
  html_text(trim = TRUE)

treeView%
html_节点（xpath='//div//p[contains（@class，“Mui”）]'）%>%
html_文本（trim=TRUE）

给出了我需要的文本，但没有给出里面有多少层（或行）的指标。有什么解决办法使之成为可能吗

编辑：

本例中的第一个文本“电动机和发电机（不包括发电机组）”将为零（表示它没有任何父级）
第二个文本；（输出功率不超过37.5 W的电机）有一个父级，因此其子级为1
第三级和第四级的情况也是如此（它们只有一个父级，因此其子级为1）
让我们看看第五个（输出功率不超过750 W），它有一个父级（第四个文本，inturn有一个父级）。这使得第5个文本子级别为2。这也适用于从第6页到第8页的所有文本。（1个父项+1个父项表示父项=2）
第9级和第10级有一个家长，因此他们的子级别为1
第11级和第12级有一个父级，父级有一个父级，因此其子级为2
第13个文本（带齿轮或速度转换器的交流电机：）有1个父级（以及随后的2个父级）。所以它的子级是3

library(httr)
json_list <- content(GET("https://tolltariffen.toll.no/api/search/headings/85.01"))

recur_pluck(json_list)

   sublevel                                                         text
1         6  Electric motors and generators (excluding generating sets).
2         1                     Motors of an output not exceeding 37.5 W
3         1         Universal AC/DC motors of an output exceeding 37.5 W
4         4                             Other DC motors; DC generators :
5         1                       Of an output not exceeding 750 W&nbsp;
6         1   Of an output exceeding 750 W but not exceeding 75 kW&nbsp;
7         1   Of an output exceeding 75 kW but not exceeding \n\n 375 kW
8         1                                Of an output exceeding 375 kW
9         1                                Other AC motors, single-phase
10        3                               Other AC motors, multi-phase :
11        1                       Of an output not exceeding 750 W&nbsp;
12        2  Of an output exceeding 750 W but not exceeding \n\n 75 kW :
13        2                    AC motors with gear or speed converters :
14        1         Of an output exceeding 750 W but not exceeding 15 kW
15        1         Of an output exceeding 15 kW but not exceeding 75 kW
16        3                                                      Other :
17        1        Of an output exceeding 750 W but not exceeding 7.5 kW
18        1        Of an output exceeding 7.5 kW but not exceeding 22 kW
19        1         Of an output exceeding 22 kW but not exceeding 75 kW
20        1                           Of an output exceeding 75 kW&nbsp;
21        4                                AC generators (alternators) :
22        1                            Of an output not exceeding 75 kVA
23        1 Of an output exceeding 75 kVA but not exceeding \n\n 375 kVA
24        1     Of an output exceeding 375 kVA but not exceeding 750 kVA
25        1                               Of an output exceeding 750 kVA

更新

recur_pluck <- function(x) {
  sublevel <- integer()
  text <- character()
  recur_ <- function(x, depth = 0L) {
    text <<- c(text, trimws(gsub("<[^<>]+>", "", x$description$en)))
    sublevel <<- c(sublevel, depth)
    if (is.null(x$headingItems)) {
      return(NULL)
    }
    lapply(x$headingItems, recur_, depth + 1L)
  }
  recur_(x)
  data.frame(sublevel, text)
}

recur\uu

对于每个元素，函数将目标数据附加到相应的存储变量
```
text
```
、
```
subvel
```
和
```
code
```
。在向
```
code
```
追加数据时，我们使用
```
%| |%
```
函数，因为所需的8位代码可能有不同的名称。该行简单地转换为首先获取“commodityCode”，如果没有“commodityCode”，则获取“headingCode”，或者如果找不到任何内容，则返回空字符串
```
 text <<- c(text, trimws(gsub("<[^<>]+>", "", x$description$en)))
 sublevel <<- c(sublevel, depth)
 code <<- c(code, x$commodityCode %||% x$headingCode %||% "")
```
如果内部有其他
```
headingItems
```
，它将以深度计数加1的方式进行更深的遍历
```
 lapply(x$headingItems, recur_, depth + 1L)
```

   sublevel       code                                                         text
1         0      85.01  Electric motors and generators (excluding generating sets).
2         1 85.01.1000                     Motors of an output not exceeding 37.5 W
3         1 85.01.2000         Universal AC/DC motors of an output exceeding 37.5 W
4         1                                        Other DC motors; DC generators :
5         2 85.01.3100                       Of an output not exceeding 750 W&nbsp;
6         2 85.01.3200   Of an output exceeding 750 W but not exceeding 75 kW&nbsp;
7         2 85.01.3300   Of an output exceeding 75 kW but not exceeding \n\n 375 kW
8         2 85.01.3400                                Of an output exceeding 375 kW
9         1 85.01.4000                                Other AC motors, single-phase
10        1                                          Other AC motors, multi-phase :
11        2 85.01.5100                       Of an output not exceeding 750 W&nbsp;
12        2             Of an output exceeding 750 W but not exceeding \n\n 75 kW :
13        3                               AC motors with gear or speed converters :
14        4 85.01.5201         Of an output exceeding 750 W but not exceeding 15 kW
15        4 85.01.5202         Of an output exceeding 15 kW but not exceeding 75 kW
16        3                                                                 Other :
17        4 85.01.5203        Of an output exceeding 750 W but not exceeding 7.5 kW
18        4 85.01.5204        Of an output exceeding 7.5 kW but not exceeding 22 kW
19        4 85.01.5205         Of an output exceeding 22 kW but not exceeding 75 kW
20        2 85.01.5300                           Of an output exceeding 75 kW&nbsp;
21        1                                           AC generators (alternators) :
22        2 85.01.6100                            Of an output not exceeding 75 kVA
23        2 85.01.6200 Of an output exceeding 75 kVA but not exceeding \n\n 375 kVA
24        2 85.01.6300     Of an output exceeding 375 kVA but not exceeding 750 kVA
25        2 85.01.6400                               Of an output exceeding 750 kVA

description$en

 text <<- c(text, trimws(gsub("<[^<>]+>", "", x$description$en)))
 sublevel <<- c(sublevel, depth)
 code <<- c(code, x$commodityCode %||% x$headingCode %||% "")

 if (is.null(x$headingItems)) {
   return(NULL)
 }

 lapply(x$headingItems, recur_, depth + 1L)

   sublevel       code                                                         text
1         0      85.01  Electric motors and generators (excluding generating sets).
2         1 85.01.1000                     Motors of an output not exceeding 37.5 W
3         1 85.01.2000         Universal AC/DC motors of an output exceeding 37.5 W
4         1                                        Other DC motors; DC generators :
5         2 85.01.3100                       Of an output not exceeding 750 W&nbsp;
6         2 85.01.3200   Of an output exceeding 750 W but not exceeding 75 kW&nbsp;
7         2 85.01.3300   Of an output exceeding 75 kW but not exceeding \n\n 375 kW
8         2 85.01.3400                                Of an output exceeding 375 kW
9         1 85.01.4000                                Other AC motors, single-phase
10        1                                          Other AC motors, multi-phase :
11        2 85.01.5100                       Of an output not exceeding 750 W&nbsp;
12        2             Of an output exceeding 750 W but not exceeding \n\n 75 kW :
13        3                               AC motors with gear or speed converters :
14        4 85.01.5201         Of an output exceeding 750 W but not exceeding 15 kW
15        4 85.01.5202         Of an output exceeding 15 kW but not exceeding 75 kW
16        3                                                                 Other :
17        4 85.01.5203        Of an output exceeding 750 W but not exceeding 7.5 kW
18        4 85.01.5204        Of an output exceeding 7.5 kW but not exceeding 22 kW
19        4 85.01.5205         Of an output exceeding 22 kW but not exceeding 75 kW
20        2 85.01.5300                           Of an output exceeding 75 kW&nbsp;
21        1                                           AC generators (alternators) :
22        2 85.01.6100                            Of an output not exceeding 75 kVA
23        2 85.01.6200 Of an output exceeding 75 kVA but not exceeding \n\n 375 kVA
24        2 85.01.6300     Of an output exceeding 375 kVA but not exceeding 750 kVA
25        2 85.01.6400                               Of an output exceeding 750 kVA