Nlp 创建具有节的Spacy文档

Nlp 创建具有节的Spacy文档,nlp,spacy,Nlp,Spacy,我想知道当人们想把一个文档分成不同的跨度时,他们为Spacy做了什么?例如,假设我创建了一个doc对象的语料库如下。但是对于我正在执行的任务,我希望在维护原始对象的同时为不同的部分创建索引 doc = nlp(""" Patient History: This is paragraph 1. Assessment: This is paragraph 2. Signature: This is paragraph 3. """) 然后对其进行解析,类似于: doc.se

我想知道当人们想把一个文档分成不同的跨度时,他们为Spacy做了什么?例如,假设我创建了一个doc对象的语料库如下。但是对于我正在执行的任务,我希望在维护原始对象的同时为不同的部分创建索引

doc = nlp("""
Patient History:
    This is paragraph 1.
Assessment:
    This is paragraph 2.
Signature:
    This is paragraph 3.
""")
然后对其进行解析,类似于:

doc.sections_ 
会让步

["Patient History", "Assessment", "Signature"]

SpaCy不支持“部分”——它们不是文档的通用功能,如何定义它们取决于你是在处理小说、学术论文还是报纸等

最简单的方法是在将文档馈送给spacy之前自己拆分文档。如果它的格式与您的示例类似,那么使用缩进应该很容易做到


如果您真的希望只有一个Doc对象,那么您应该能够通过spaCy的管道扩展来管理它。请参阅文档。

显然,这必须在文件步骤中进行,并且没有针对管道进行优化,但这是我的稍微有点粗糙的解决方案

  class ParsedNoteSections(object):
    """
        Pars notes into sections based on entity-tags. All sections are return as newly
        created doc objects.
    """



    def __init__(self,doc):
        self.doc = doc

    def get_section_titles(self):
    """Return the section header titles."""
    return [(e,e.start, e.end) for e in self.doc.ents if e.label_ == 'NOTESECTION']

    def original(self,doc):
        """Retrieve oringal doc object."""
        return self.doc

    def __repr__(self):
        return repr(self.doc)

    def parse_note_sections(self):
        """ Use entity sections as break-points to split original doc.

        Input: 
            None
        Output:
            List of section of objects stored in dictionary.
        """
        section_titles = self.get_section_titles()

        # stopgap for possible errors
        assert len(section_titles) > 0

        doc_section_spans = []
        for idx,section in enumerate(section_titles):
            section_label_new = section[0]
            label_start_new = section[1]
            label_end_new = section[2]

            # store first label
            if idx == 0:
                section_label_old = section_label_new
                continue

            # store last section
            elif idx == 1:
                section_label = section_label_old
                section_doc = self.doc[:label_start_new]

            # if on the last section
            elif idx == len(section_titles) - 1:
                section_label = section_label_old
                section_doc = self.doc[label_start_old:label_start_new]
                doc_section_spans.append({'section_label':section_label, 'section_doc':section_doc})

                section_label = section_label_new
                section_doc = self.doc[label_start_new:]

            # if not storing first or last section
            else:
                section_label = section_label_old
                section_doc = self.doc[label_start_old:label_start_new]

            label_start_old = label_start_new
            section_label_old = section_label_new

            doc_section_spans.append({'section_label':section_label, 'section_doc':section_doc})

        assert len(doc_section_spans) == len(section_titles)

        return doc_section_spans

谢谢,这也是我解决方案的基本思路。