从pdf-Python中提取引用
在我的python项目中，我需要从pdf研究论文中提取从pdf-Python中提取引用,python,text-extraction,Python,Text Extraction,在我的python项目中，我需要从pdf研究论文中提取参考资料。我正在使用PyPDF2阅读pdf并像这样从中提取文本 import PyPDF2 pdfFileObj = open('fileName.pdf','rb') pdfReader = PyPDF2.PdfFileReader(pdfFileObj) pageCount = pdfReader.numPages count = 0 text = '' while count < pageCount: pageObj
参考资料。我正在使用PyPDF2
阅读pdf并像这样从中提取文本
import PyPDF2
pdfFileObj = open('fileName.pdf','rb')
pdfReader = PyPDF2.PdfFileReader(pdfFileObj)
pageCount = pdfReader.numPages
count = 0
text = ''
while count < pageCount:
pageObj = pdfReader.getPage(count)
count +=1
text += pageObj.extractText()
如何将这些引用字符串解析为pdf中提到的多个引用？任何帮助都将不胜感激。PDF
非常复杂，我不是专家，但我从源代码中了解了它的工作原理，并使用打印（'>>'，运算符，操作数）
我可以看到它在PDF中找到了哪些值
在本文档中，它使用“Tm”
将位置移动到新行，因此在extractText（）
中更改了原始代码，我使用“Tm”
添加了\n
，我得到了行中的文本
Arto Anttila. 1995. How to recognise subjects in
English. In Karlsson et al., chapt. 9, pp. 315-358.
Dekang Lin. 1996. Evaluation of Principar with the
Susanne corpus. In John Carroll, editor, Work-
shop on Robust Parsing, pages 54-69, Prague.
Jason M. Eisner. 1996. Three new probabilistic
models for dependency parsing: An exploration.
In The 16th International Conference on Compu-
tational Linguistics, pages 340-345. Copenhagen.
David G. Hays. 1964. Dependency theory: A
formalism and some observations. Language,
40(4):511-525.
或者在行间使用--
---
Arto Anttila. 1995. How to recognise subjects in
---
English. In Karlsson et al., chapt. 9, pp. 315-358.
---
Dekang Lin. 1996. Evaluation of Principar with the
---
Susanne corpus. In John Carroll, editor, Work-
---
shop on Robust Parsing, pages 54-69, Prague.
---
Jason M. Eisner. 1996. Three new probabilistic
---
models for dependency parsing: An exploration.
---
In The 16th International Conference on Compu-
---
tational Linguistics, pages 340-345. Copenhagen.
---
David G. Hays. 1964. Dependency theory: A
---
formalism and some observations. Language,
---
40(4):511-525.
但它仍然没有那么有用，但现在我使用的代码得到了这个结果
import PyPDF2
from PyPDF2.pdf import * # to import function used in origimal `extractText`
# --- functions ---
def myExtractText(self):
# code from original `extractText()`
# https://github.com/mstamy2/PyPDF2/blob/d7b8d3e0f471530267827511cdffaa2ab48bc1ad/PyPDF2/pdf.py#L2645
text = u_("")
content = self["/Contents"].getObject()
if not isinstance(content, ContentStream):
content = ContentStream(content, self.pdf)
for operands, operator in content.operations:
# used only for test to see values in variables
#print('>>>', operator, operands)
if operator == b_("Tj"):
_text = operands[0]
if isinstance(_text, TextStringObject):
text += _text
elif operator == b_("T*"):
text += "\n"
elif operator == b_("'"):
text += "\n"
_text = operands[0]
if isinstance(_text, TextStringObject):
text += operands[0]
elif operator == b_('"'):
_text = operands[2]
if isinstance(_text, TextStringObject):
text += "\n"
text += _text
elif operator == b_("TJ"):
for i in operands[0]:
if isinstance(i, TextStringObject):
text += i
text += "\n"
# new code to add `\n` when text moves to new line
elif operator == b_("Tm"):
text += '\n'
return text
# --- main ---
pdfFileObj = open('A97-1011.pdf', 'rb')
pdfReader = PyPDF2.PdfFileReader(pdfFileObj)
text = ''
for page in pdfReader.pages:
#text += page.extractText() # original function
text += myExtractText(page) # modified function
# get only text after word `References`
pos = text.lower().find('references')
text = text[pos+len('references '):]
# print all at once
print(text)
# print line by line
for line in text.split('\n'):
print(line)
print('---')
挖掘之后，似乎Tm
也有值，并且有一个新的位置x，y
，我用来计算文本行之间的距离，当距离大于某个值时，我添加\n
。我测试了不同的值，从值17
中得到了预期的结果
---
Arto Anttila. 1995. How to recognise subjects in English. In Karlsson et al., chapt. 9, pp. 315-358.
---
Dekang Lin. 1996. Evaluation of Principar with the Susanne corpus. In John Carroll, editor, Work- shop on Robust Parsing, pages 54-69, Prague.
---
Jason M. Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In The 16th International Conference on Compu- tational Linguistics, pages 340-345. Copenhagen.
---
David G. Hays. 1964. Dependency theory: A formalism and some observations. Language, 40(4):511-525.
---
这里是代码
import PyPDF2
from PyPDF2.pdf import * # to import function used in origimal `extractText`
# --- functions ---
def myExtractText2(self):
# original code from `page.extractText()`
# https://github.com/mstamy2/PyPDF2/blob/d7b8d3e0f471530267827511cdffaa2ab48bc1ad/PyPDF2/pdf.py#L2645
text = u_("")
content = self["/Contents"].getObject()
if not isinstance(content, ContentStream):
content = ContentStream(content, self.pdf)
prev_x = 0
prev_y = 0
for operands, operator in content.operations:
# used only for test to see values in variables
#print('>>>', operator, operands)
if operator == b_("Tj"):
_text = operands[0]
if isinstance(_text, TextStringObject):
text += _text
elif operator == b_("T*"):
text += "\n"
elif operator == b_("'"):
text += "\n"
_text = operands[0]
if isinstance(_text, TextStringObject):
text += operands[0]
elif operator == b_('"'):
_text = operands[2]
if isinstance(_text, TextStringObject):
text += "\n"
text += _text
elif operator == b_("TJ"):
for i in operands[0]:
if isinstance(i, TextStringObject):
text += i
text += "\n"
elif operator == b_("Tm"):
x = operands[-2]
y = operands[-1]
diff_x = prev_x - x
diff_y = prev_y - y
#print('>>>', diff_x, diff_y - y)
#text += f'| {diff_x}, {diff_y - y} |'
if diff_y > 17 or diff_y < 0: # (bigger margin) or (move to top in next column)
text += '\n'
#text += '\n' # to add empty line between elements
prev_x = x
prev_y = y
return text
# --- main ---
pdfFileObj = open('A97-1011.pdf', 'rb')
pdfReader = PyPDF2.PdfFileReader(pdfFileObj)
text = ''
for page in pdfReader.pages:
#text += page.extractText() # original function
text += myExtractText(page) # modified function
# get only text after word `References`
pos = text.lower().find('references')
text = text[pos+len('references '):]
# print all at once
print(text)
# print line by line
for line in text.split('\n'):
print(line)
print('---')
然后，它就像原始的extractText（）
一样工作，您可以在一个字符串中获得所有内容
如果第二个参数为True
text += myExtractText(page, True)
然后它在每个Tm
之后添加新行，就像我的第一个版本一样
如果第二个参数是整数-即17
text += myExtractText(page, 17)
然后，当距离大于17
-就像我的第二个版本一样，它会添加新行
import PyPDF2
from PyPDF2.pdf import * # to import function used in origimal `extractText`
# --- functions ---
def myExtractText(self, distance=None):
# original code from `page.extractText()`
# https://github.com/mstamy2/PyPDF2/blob/d7b8d3e0f471530267827511cdffaa2ab48bc1ad/PyPDF2/pdf.py#L2645
text = u_("")
content = self["/Contents"].getObject()
if not isinstance(content, ContentStream):
content = ContentStream(content, self.pdf)
prev_x = 0
prev_y = 0
for operands, operator in content.operations:
# used only for test to see values in variables
#print('>>>', operator, operands)
if operator == b_("Tj"):
_text = operands[0]
if isinstance(_text, TextStringObject):
text += _text
elif operator == b_("T*"):
text += "\n"
elif operator == b_("'"):
text += "\n"
_text = operands[0]
if isinstance(_text, TextStringObject):
text += operands[0]
elif operator == b_('"'):
_text = operands[2]
if isinstance(_text, TextStringObject):
text += "\n"
text += _text
elif operator == b_("TJ"):
for i in operands[0]:
if isinstance(i, TextStringObject):
text += i
text += "\n"
if operator == b_("Tm"):
if distance is True:
text += '\n'
elif isinstance(distance, int):
x = operands[-2]
y = operands[-1]
diff_x = prev_x - x
diff_y = prev_y - y
#print('>>>', diff_x, diff_y - y)
#text += f'| {diff_x}, {diff_y - y} |'
if diff_y > distance or diff_y < 0: # (bigger margin) or (move to top in next column)
text += '\n'
#text += '\n' # to add empty line between elements
prev_x = x
prev_y = y
return text
# --- main ---
pdfFileObj = open('A97-1011.pdf', 'rb')
pdfReader = PyPDF2.PdfFileReader(pdfFileObj)
text = ''
for page in pdfReader.pages:
#text += page.extractText() # original function
#text += myExtractText(page) # modified function (works like original version)
#text += myExtractText(page, True) # modified function (add `\n` after every `Tm`)
text += myExtractText(page, 17) # modified function (add `\n` only if distance is bigger then `17`)
# get only text after word `References`
pos = text.lower().find('references')
text = text[pos+len('references '):]
# print all at once
print(text)
# print line by line
for line in text.split('\n'):
print(line)
print('---')
PDF
非常复杂，我不是专家，但我拿了源代码来看看它是如何工作的，并使用print（'>>>，运算符，操作数）
我可以看到它在PDF中找到了什么值
在本文档中，它使用“Tm”
将位置移动到新行，因此在extractText（）
中更改了原始代码，我使用“Tm”
添加了\n
，我得到了行中的文本
Arto Anttila. 1995. How to recognise subjects in
English. In Karlsson et al., chapt. 9, pp. 315-358.
Dekang Lin. 1996. Evaluation of Principar with the
Susanne corpus. In John Carroll, editor, Work-
shop on Robust Parsing, pages 54-69, Prague.
Jason M. Eisner. 1996. Three new probabilistic
models for dependency parsing: An exploration.
In The 16th International Conference on Compu-
tational Linguistics, pages 340-345. Copenhagen.
David G. Hays. 1964. Dependency theory: A
formalism and some observations. Language,
40(4):511-525.
或者在行间使用--
---
Arto Anttila. 1995. How to recognise subjects in
---
English. In Karlsson et al., chapt. 9, pp. 315-358.
---
Dekang Lin. 1996. Evaluation of Principar with the
---
Susanne corpus. In John Carroll, editor, Work-
---
shop on Robust Parsing, pages 54-69, Prague.
---
Jason M. Eisner. 1996. Three new probabilistic
---
models for dependency parsing: An exploration.
---
In The 16th International Conference on Compu-
---
tational Linguistics, pages 340-345. Copenhagen.
---
David G. Hays. 1964. Dependency theory: A
---
formalism and some observations. Language,
---
40(4):511-525.
但它仍然没有那么有用，但现在我使用的代码得到了这个结果
import PyPDF2
from PyPDF2.pdf import * # to import function used in origimal `extractText`
# --- functions ---
def myExtractText(self):
# code from original `extractText()`
# https://github.com/mstamy2/PyPDF2/blob/d7b8d3e0f471530267827511cdffaa2ab48bc1ad/PyPDF2/pdf.py#L2645
text = u_("")
content = self["/Contents"].getObject()
if not isinstance(content, ContentStream):
content = ContentStream(content, self.pdf)
for operands, operator in content.operations:
# used only for test to see values in variables
#print('>>>', operator, operands)
if operator == b_("Tj"):
_text = operands[0]
if isinstance(_text, TextStringObject):
text += _text
elif operator == b_("T*"):
text += "\n"
elif operator == b_("'"):
text += "\n"
_text = operands[0]
if isinstance(_text, TextStringObject):
text += operands[0]
elif operator == b_('"'):
_text = operands[2]
if isinstance(_text, TextStringObject):
text += "\n"
text += _text
elif operator == b_("TJ"):
for i in operands[0]:
if isinstance(i, TextStringObject):
text += i
text += "\n"
# new code to add `\n` when text moves to new line
elif operator == b_("Tm"):
text += '\n'
return text
# --- main ---
pdfFileObj = open('A97-1011.pdf', 'rb')
pdfReader = PyPDF2.PdfFileReader(pdfFileObj)
text = ''
for page in pdfReader.pages:
#text += page.extractText() # original function
text += myExtractText(page) # modified function
# get only text after word `References`
pos = text.lower().find('references')
text = text[pos+len('references '):]
# print all at once
print(text)
# print line by line
for line in text.split('\n'):
print(line)
print('---')
挖掘之后，似乎Tm
也有值，并且有一个新的位置x，y
，我用来计算文本行之间的距离，当距离大于某个值时，我添加\n
。我测试了不同的值，从值17
中得到了预期的结果
---
Arto Anttila. 1995. How to recognise subjects in English. In Karlsson et al., chapt. 9, pp. 315-358.
---
Dekang Lin. 1996. Evaluation of Principar with the Susanne corpus. In John Carroll, editor, Work- shop on Robust Parsing, pages 54-69, Prague.
---
Jason M. Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In The 16th International Conference on Compu- tational Linguistics, pages 340-345. Copenhagen.
---
David G. Hays. 1964. Dependency theory: A formalism and some observations. Language, 40(4):511-525.
---
这里是代码
import PyPDF2
from PyPDF2.pdf import * # to import function used in origimal `extractText`
# --- functions ---
def myExtractText2(self):
# original code from `page.extractText()`
# https://github.com/mstamy2/PyPDF2/blob/d7b8d3e0f471530267827511cdffaa2ab48bc1ad/PyPDF2/pdf.py#L2645
text = u_("")
content = self["/Contents"].getObject()
if not isinstance(content, ContentStream):
content = ContentStream(content, self.pdf)
prev_x = 0
prev_y = 0
for operands, operator in content.operations:
# used only for test to see values in variables
#print('>>>', operator, operands)
if operator == b_("Tj"):
_text = operands[0]
if isinstance(_text, TextStringObject):
text += _text
elif operator == b_("T*"):
text += "\n"
elif operator == b_("'"):
text += "\n"
_text = operands[0]
if isinstance(_text, TextStringObject):
text += operands[0]
elif operator == b_('"'):
_text = operands[2]
if isinstance(_text, TextStringObject):
text += "\n"
text += _text
elif operator == b_("TJ"):
for i in operands[0]:
if isinstance(i, TextStringObject):
text += i
text += "\n"
elif operator == b_("Tm"):
x = operands[-2]
y = operands[-1]
diff_x = prev_x - x
diff_y = prev_y - y
#print('>>>', diff_x, diff_y - y)
#text += f'| {diff_x}, {diff_y - y} |'
if diff_y > 17 or diff_y < 0: # (bigger margin) or (move to top in next column)
text += '\n'
#text += '\n' # to add empty line between elements
prev_x = x
prev_y = y
return text
# --- main ---
pdfFileObj = open('A97-1011.pdf', 'rb')
pdfReader = PyPDF2.PdfFileReader(pdfFileObj)
text = ''
for page in pdfReader.pages:
#text += page.extractText() # original function
text += myExtractText(page) # modified function
# get only text after word `References`
pos = text.lower().find('references')
text = text[pos+len('references '):]
# print all at once
print(text)
# print line by line
for line in text.split('\n'):
print(line)
print('---')
然后，它就像原始的extractText（）
一样工作，您可以在一个字符串中获得所有内容
如果第二个参数为True
text += myExtractText(page, True)
然后它在每个Tm
之后添加新行，就像我的第一个版本一样
如果第二个参数是整数-即17
text += myExtractText(page, 17)
然后，当距离大于17
-就像我的第二个版本一样，它会添加新行
import PyPDF2
from PyPDF2.pdf import * # to import function used in origimal `extractText`
# --- functions ---
def myExtractText(self, distance=None):
# original code from `page.extractText()`
# https://github.com/mstamy2/PyPDF2/blob/d7b8d3e0f471530267827511cdffaa2ab48bc1ad/PyPDF2/pdf.py#L2645
text = u_("")
content = self["/Contents"].getObject()
if not isinstance(content, ContentStream):
content = ContentStream(content, self.pdf)
prev_x = 0
prev_y = 0
for operands, operator in content.operations:
# used only for test to see values in variables
#print('>>>', operator, operands)
if operator == b_("Tj"):
_text = operands[0]
if isinstance(_text, TextStringObject):
text += _text
elif operator == b_("T*"):
text += "\n"
elif operator == b_("'"):
text += "\n"
_text = operands[0]
if isinstance(_text, TextStringObject):
text += operands[0]
elif operator == b_('"'):
_text = operands[2]
if isinstance(_text, TextStringObject):
text += "\n"
text += _text
elif operator == b_("TJ"):
for i in operands[0]:
if isinstance(i, TextStringObject):
text += i
text += "\n"
if operator == b_("Tm"):
if distance is True:
text += '\n'
elif isinstance(distance, int):
x = operands[-2]
y = operands[-1]
diff_x = prev_x - x
diff_y = prev_y - y
#print('>>>', diff_x, diff_y - y)
#text += f'| {diff_x}, {diff_y - y} |'
if diff_y > distance or diff_y < 0: # (bigger margin) or (move to top in next column)
text += '\n'
#text += '\n' # to add empty line between elements
prev_x = x
prev_y = y
return text
# --- main ---
pdfFileObj = open('A97-1011.pdf', 'rb')
pdfReader = PyPDF2.PdfFileReader(pdfFileObj)
text = ''
for page in pdfReader.pages:
#text += page.extractText() # original function
#text += myExtractText(page) # modified function (works like original version)
#text += myExtractText(page, True) # modified function (add `\n` after every `Tm`)
text += myExtractText(page, 17) # modified function (add `\n` only if distance is bigger then `17`)
# get only text after word `References`
pos = text.lower().find('references')
text = text[pos+len('references '):]
# print all at once
print(text)
# print line by line
for line in text.split('\n'):
print(line)
print('---')
您应该添加PDF，这会造成问题。没有PDF的问题是无用的。添加了所有必需的数据…你应该添加PDF，这会造成问题。没有PDF的问题是无用的。添加了所有必需的数据…你实际上救了我。。在过去的一周里，我一直被这个问题困扰着。。尝试不同的方法和库。感谢您为我提供解决方案，并向我展示另一种解决问题的方法。非常感谢..顺便说一句：它不仅可以用于引用
，还可以用于拆分其余文本-我在您的PDF中添加了开始文本的示例。这只适用于一个PDF，我有50个，没有一个适用于此代码。。你知道我如何制作一个通用函数来处理所有PDF吗？几乎不可能为每个PDF检查/更新extractText函数。有人猜测吗？PDF非常复杂，当你不得不从PDF中刮取数据/文本时，这是一场噩梦。没有通用的方法。如果每个引用之间都有一定距离，则可以尝试获取引用中的所有距离，并获取最大值以拆分它们。但它可能需要在提取文本中进行许多更改-首先，它必须获取所有距离，然后再次读取，并使用此距离来分离引用。或者它会将它作为成对的列表（距离，文本）
来搜索最大的距离并将其拆分。你实际上救了我。。在过去的一周里，我一直被这个问题困扰着。。尝试不同的方法和库。感谢您为我提供解决方案，并向我展示另一种解决问题的方法。非常感谢..顺便说一句：它不仅可以用于引用
，还可以用于拆分其余文本-我在您的PDF中添加了开始文本的示例。这只适用于一个PDF，我有50个，没有一个适用于此代码。。你知道我如何制作一个通用函数来处理所有PDF吗？几乎不可能为每个PDF检查/更新extractText函数。有人猜测吗？PDF非常复杂，当你不得不从PDF中刮取数据/文本时，这是一场噩梦。没有通用的方法。如果每个引用之间都有一定距离，则可以尝试获取引用中的所有距离，并获取最大值以拆分它们。但它可能需要在提取文本中进行许多更改-首先，它必须获取所有距离，然后再次读取，并使用此距离来分离引用。或者，它会将其作为成对的列表（距离，文本）
，以搜索最大距离并将其拆分。