R 在课文中找到相似的句子
我有一个问题,我正在努力寻找解决方案或方法来解决它 我有一些例句,例如R 在课文中找到相似的句子,r,text,similarity,R,Text,Similarity,我有一个问题,我正在努力寻找解决方案或方法来解决它 我有一些例句,例如 model_sentences = data.frame("model_id" = c("model_id_1", "model_id_2"), "model_text" = c("Company x had 3000 employees in 2016.",
model_sentences = data.frame("model_id" = c("model_id_1", "model_id_2"), "model_text" = c("Company x had 3000 employees in 2016.",
"Google makes 300 dollar in revenue in 2018."))
还有一些文本
data = data.frame("id" = c("id1", "id2"), "text" = c("Company y is expected to employ 2000 employees in 2020. This is an increase of 10%. Some stupid sentences.",
"Amazon´s revenue is 400 dollar in 2020. That is twice as much as last year."))
我想从这些文本中提取与模型句相似的句子
像这样的东西将是我想要的解决方案
result = data.frame("id" = c("id1", "id2"), "model_id" = c("model_id_1", "model_id_2"), "sentence_from_data" = c("Company y is expected to employ 2000 employees in 2020.", "Amazon´s revenue is 400 dollar in 2020."), "score" = c(0.5, 0.4))
也许有可能找到某种“相似性分数”
我使用此功能按句子分割文本:
split_by_sentence <- function (text) {
result <-unlist(strsplit(text, "(?<=[[:alnum:]]{4}[?!.])\\s+", perl=TRUE))
result <- stri_trim_both(result)
result <- result [nchar (result) > 0]
if (length (result) == 0)
result <- ""
return (result)
}
但我不知道如何将每个句子与示范句进行比较。
我很高兴有任何建议。请查看此软件包stringdist
例如:
library(stringdist)
mysent = "This is a sentence"
apply(model_sentences, 1, function(row) {
stringdist(row['model_text'], mysent, method="jaccard")
})
它将返回从mycent到model_文本变量的jaccard距离。值越小,就给定的距离度量而言,句子越相似