Nlp 创建具有节的Spacy文档
我想知道当人们想把一个文档分成不同的跨度时,他们为Spacy做了什么?例如,假设我创建了一个doc对象的语料库如下。但是对于我正在执行的任务,我希望在维护原始对象的同时为不同的部分创建索引Nlp 创建具有节的Spacy文档,nlp,spacy,Nlp,Spacy,我想知道当人们想把一个文档分成不同的跨度时,他们为Spacy做了什么?例如,假设我创建了一个doc对象的语料库如下。但是对于我正在执行的任务,我希望在维护原始对象的同时为不同的部分创建索引 doc = nlp(""" Patient History: This is paragraph 1. Assessment: This is paragraph 2. Signature: This is paragraph 3. """) 然后对其进行解析,类似于: doc.se
doc = nlp("""
Patient History:
This is paragraph 1.
Assessment:
This is paragraph 2.
Signature:
This is paragraph 3.
""")
然后对其进行解析,类似于:
doc.sections_
会让步
["Patient History", "Assessment", "Signature"]
SpaCy不支持“部分”——它们不是文档的通用功能,如何定义它们取决于你是在处理小说、学术论文还是报纸等 最简单的方法是在将文档馈送给spacy之前自己拆分文档。如果它的格式与您的示例类似,那么使用缩进应该很容易做到
如果您真的希望只有一个Doc对象,那么您应该能够通过spaCy的管道扩展来管理它。请参阅文档。显然,这必须在文件步骤中进行,并且没有针对管道进行优化,但这是我的稍微有点粗糙的解决方案
class ParsedNoteSections(object):
"""
Pars notes into sections based on entity-tags. All sections are return as newly
created doc objects.
"""
def __init__(self,doc):
self.doc = doc
def get_section_titles(self):
"""Return the section header titles."""
return [(e,e.start, e.end) for e in self.doc.ents if e.label_ == 'NOTESECTION']
def original(self,doc):
"""Retrieve oringal doc object."""
return self.doc
def __repr__(self):
return repr(self.doc)
def parse_note_sections(self):
""" Use entity sections as break-points to split original doc.
Input:
None
Output:
List of section of objects stored in dictionary.
"""
section_titles = self.get_section_titles()
# stopgap for possible errors
assert len(section_titles) > 0
doc_section_spans = []
for idx,section in enumerate(section_titles):
section_label_new = section[0]
label_start_new = section[1]
label_end_new = section[2]
# store first label
if idx == 0:
section_label_old = section_label_new
continue
# store last section
elif idx == 1:
section_label = section_label_old
section_doc = self.doc[:label_start_new]
# if on the last section
elif idx == len(section_titles) - 1:
section_label = section_label_old
section_doc = self.doc[label_start_old:label_start_new]
doc_section_spans.append({'section_label':section_label, 'section_doc':section_doc})
section_label = section_label_new
section_doc = self.doc[label_start_new:]
# if not storing first or last section
else:
section_label = section_label_old
section_doc = self.doc[label_start_old:label_start_new]
label_start_old = label_start_new
section_label_old = section_label_new
doc_section_spans.append({'section_label':section_label, 'section_doc':section_doc})
assert len(doc_section_spans) == len(section_titles)
return doc_section_spans
谢谢,这也是我解决方案的基本思路。