Python 使用n_gram的文本摘要(如何编写函数对句子进行评分)
我有如下生成n_gram的代码:Python 使用n_gram的文本摘要(如何编写函数对句子进行评分),python,python-3.x,Python,Python 3.x,我有如下生成n_gram的代码: import nltk from nltk.util import ngrams def extract_ngrams(data, num): n_grams = ngrams(nltk.word_tokenize(data.lower()),num) return [' '.join(grams) for grams in n_grams] gram_num = 3 n_grams = extract_ngrams(filtered_se
import nltk
from nltk.util import ngrams
def extract_ngrams(data, num):
n_grams = ngrams(nltk.word_tokenize(data.lower()),num)
return [' '.join(grams) for grams in n_grams]
gram_num = 3
n_grams = extract_ngrams(filtered_sentence, gram_num)
from nltk import FreqDist
ngram_freqs = FreqDist(n_grams)
sent_tokens = nltk.sent_tokenize(text)
我使用fdist获得了n-gram频率的代码,如下所示:
import nltk
from nltk.util import ngrams
def extract_ngrams(data, num):
n_grams = ngrams(nltk.word_tokenize(data.lower()),num)
return [' '.join(grams) for grams in n_grams]
gram_num = 3
n_grams = extract_ngrams(filtered_sentence, gram_num)
from nltk import FreqDist
ngram_freqs = FreqDist(n_grams)
sent_tokens = nltk.sent_tokenize(text)
但我不知道如何定义给句子打分的函数
我想得到如下函数:
calculate_sentence_scores_ngram(sent_tokens, ngram_freqs, n_grams)