Python 如何使用GCP Vision或OpenCV提取图像中的自动文本和手写文本_Python_Opencv_Image Processing_Google Cloud Platform_Deep Learning

Python 如何使用GCP Vision或OpenCV提取图像中的自动文本和手写文本

python opencv image-processing google-cloud-platform deep-learning

Python 如何使用GCP Vision或OpenCV提取图像中的自动文本和手写文本,python,opencv,image-processing,google-cloud-platform,deep-learning,Python,Opencv,Image Processing,Google Cloud Platform,Deep Learning,我已经写了一段代码，其中自动文本和书面文本出现在图像中。GCP vision API无法给我正确的响应。有人能建议我做些什么吗这是我的形象这是我的代码： import os import io from google.cloud import vision from google.cloud.vision import types import pandas as pd os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = r'key.json'

我已经写了一段代码，其中自动文本和书面文本出现在图像中。GCP vision API无法给我正确的响应。有人能建议我做些什么吗

这是我的形象

这是我的代码：

import os
import io
from google.cloud import vision
from google.cloud.vision import types
import pandas as pd

os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = r'key.json'
client = vision.ImageAnnotatorClient()

IMAGE_FILE = 'datasetone.png'
FILE_PATH = os.path.join(IMAGE_FILE)

with io.open(FILE_PATH, 'rb') as image_file:
    content = image_file.read()

image = vision.types.Image(content=content)
response = client.document_text_detection(image=image)

docText = response.full_text_annotation.text
print(docText)


pages = response.full_text_annotation.pages
for page in pages:
    for block in page.blocks:
        print('block confidence:', block.confidence)

        for paragraph in block.paragraphs:
            print('paragraph confidence:', paragraph.confidence)

            for word in paragraph.words:
                word_text = ''.join([symbol.text for symbol in word.symbols])

                print('Word text: {0} (confidence: {1}'.format(
                    word_text, word.confidence))

                for symbol in word.symbols:
                    print('\tSymbol: {0} (confidence: {1}'.format(
                        symbol.text, symbol.confidence))

输出是

Worksheet
Car number: - 1002110021
Data:- Endter
Shell
Qilchange
Phone: 982838238823
email:xyz@xyz.com
name
pod
pos
Technician
name
desc
code
112121223 Benz
10-10-19
10-10-20
Jhondoe
nil
12244334
Baleno
20-10-10
| 20-20-10
Machel
nil
D901.27
'RIOR

block confidence: 0.9800000190734863
paragraph confidence: 0.9800000190734863
Word text: Worksheet (confidence: 0.9900000095367432
        Symbol: W (confidence: 0.9900000095367432
        Symbol: o (confidence: 0.9900000095367432
        Symbol: r (confidence: 1.0
        Symbol: k (confidence: 0.9900000095367432
        Symbol: s (confidence: 1.0
        Symbol: h (confidence: 1.0
        Symbol: e (confidence: 0.9900000095367432
        Symbol: e (confidence: 1.0
        Symbol: t (confidence: 1.0
Word text: Car (confidence: 0.9800000190734863
        Symbol: C (confidence: 0.9800000190734863
        Symbol: a (confidence: 0.9900000095367432
        Symbol: r (confidence: 0.9900000095367432
Word text: number (confidence: 0.9800000190734863
        Symbol: n (confidence: 0.9599999785423279
        Symbol: u (confidence: 0.9800000190734863
        Symbol: m (confidence: 0.9800000190734863
        Symbol: b (confidence: 0.9800000190734863
        Symbol: e (confidence: 1.0
        Symbol: r (confidence: 1.0
Word text: : (confidence: 0.9900000095367432
        Symbol: : (confidence: 0.9900000095367432
Word text: - (confidence: 0.949999988079071
        Symbol: - (confidence: 0.949999988079071
Word text: 1002110021 (confidence: 0.9700000286102295
        Symbol: 1 (confidence: 0.9599999785423279
        Symbol: 0 (confidence: 0.9800000190734863
        Symbol: 0 (confidence: 0.9800000190734863
        Symbol: 2 (confidence: 0.9399999976158142
        Symbol: 1 (confidence: 0.9599999785423279
        Symbol: 1 (confidence: 0.9900000095367432
        Symbol: 0 (confidence: 0.9599999785423279
        Symbol: 0 (confidence: 0.9900000095367432
        Symbol: 2 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 0.9900000095367432
Word text: Data (confidence: 0.9900000095367432
        Symbol: D (confidence: 0.9900000095367432
        Symbol: a (confidence: 0.9900000095367432
        Symbol: t (confidence: 0.9900000095367432
        Symbol: a (confidence: 1.0
Word text: : (confidence: 0.9900000095367432
        Symbol: : (confidence: 0.9900000095367432
Word text: - (confidence: 0.9900000095367432
        Symbol: - (confidence: 0.9900000095367432
Word text: Endter (confidence: 0.9900000095367432
        Symbol: E (confidence: 0.9900000095367432
        Symbol: n (confidence: 1.0
        Symbol: d (confidence: 0.9900000095367432
        Symbol: t (confidence: 0.9900000095367432
        Symbol: e (confidence: 1.0
        Symbol: r (confidence: 1.0
block confidence: 0.9700000286102295
paragraph confidence: 0.9700000286102295
Word text: Shell (confidence: 0.9900000095367432
        Symbol: S (confidence: 0.9900000095367432
        Symbol: h (confidence: 0.9900000095367432
        Symbol: e (confidence: 0.9900000095367432
        Symbol: l (confidence: 1.0
        Symbol: l (confidence: 1.0
Word text: Qilchange (confidence: 0.8999999761581421
        Symbol: Q (confidence: 0.6100000143051147
        Symbol: i (confidence: 0.9100000262260437
        Symbol: l (confidence: 0.9100000262260437
        Symbol: c (confidence: 0.8500000238418579
        Symbol: h (confidence: 0.9200000166893005
        Symbol: a (confidence: 0.9900000095367432
        Symbol: n (confidence: 0.9900000095367432
        Symbol: g (confidence: 1.0
        Symbol: e (confidence: 1.0
Word text: Phone (confidence: 0.9900000095367432
        Symbol: P (confidence: 0.9900000095367432
        Symbol: h (confidence: 0.9900000095367432
        Symbol: o (confidence: 1.0
        Symbol: n (confidence: 1.0
        Symbol: e (confidence: 1.0
Word text: : (confidence: 1.0
        Symbol: : (confidence: 1.0
Word text: 982838238823 (confidence: 0.9800000190734863
        Symbol: 9 (confidence: 0.9599999785423279
        Symbol: 8 (confidence: 0.9900000095367432
        Symbol: 2 (confidence: 0.9900000095367432
        Symbol: 8 (confidence: 0.9900000095367432
        Symbol: 3 (confidence: 0.9900000095367432
        Symbol: 8 (confidence: 0.9900000095367432
        Symbol: 2 (confidence: 0.9900000095367432
        Symbol: 3 (confidence: 0.9900000095367432
        Symbol: 8 (confidence: 0.9900000095367432
        Symbol: 8 (confidence: 0.9900000095367432
        Symbol: 2 (confidence: 0.9900000095367432
        Symbol: 3 (confidence: 0.9900000095367432
Word text: email (confidence: 0.9900000095367432
        Symbol: e (confidence: 0.9900000095367432
        Symbol: m (confidence: 0.9900000095367432
        Symbol: a (confidence: 0.9900000095367432
        Symbol: i (confidence: 1.0
        Symbol: l (confidence: 0.9900000095367432
Word text: : (confidence: 0.9900000095367432
        Symbol: : (confidence: 0.9900000095367432
Word text: xyz (confidence: 0.9900000095367432
        Symbol: x (confidence: 0.9900000095367432
        Symbol: y (confidence: 0.9900000095367432
        Symbol: z (confidence: 0.9900000095367432
Word text: @ (confidence: 0.9900000095367432
        Symbol: @ (confidence: 0.9900000095367432
Word text: xyz (confidence: 0.9700000286102295
        Symbol: x (confidence: 0.949999988079071
        Symbol: y (confidence: 0.9900000095367432
        Symbol: z (confidence: 0.9900000095367432
Word text: . (confidence: 0.9900000095367432
        Symbol: . (confidence: 0.9900000095367432
Word text: com (confidence: 0.9900000095367432
        Symbol: c (confidence: 0.9900000095367432
        Symbol: o (confidence: 1.0
        Symbol: m (confidence: 1.0
block confidence: 0.9900000095367432
paragraph confidence: 0.9900000095367432
Word text: name (confidence: 0.9900000095367432
        Symbol: n (confidence: 0.9900000095367432
        Symbol: a (confidence: 1.0
        Symbol: m (confidence: 0.9900000095367432
        Symbol: e (confidence: 1.0
block confidence: 0.9900000095367432
paragraph confidence: 0.9900000095367432
Word text: pod (confidence: 0.9900000095367432
        Symbol: p (confidence: 0.9900000095367432
        Symbol: o (confidence: 1.0
        Symbol: d (confidence: 0.9900000095367432
block confidence: 0.5699999928474426
paragraph confidence: 0.5699999928474426
Word text: pos (confidence: 0.5699999928474426
        Symbol: p (confidence: 0.20000000298023224
        Symbol: o (confidence: 0.9900000095367432
        Symbol: s (confidence: 0.5400000214576721
block confidence: 0.9900000095367432
paragraph confidence: 0.9900000095367432
Word text: Technician (confidence: 0.9900000095367432
        Symbol: T (confidence: 0.9900000095367432
        Symbol: e (confidence: 1.0
        Symbol: c (confidence: 0.9900000095367432
        Symbol: h (confidence: 1.0
        Symbol: n (confidence: 1.0
        Symbol: i (confidence: 1.0
        Symbol: c (confidence: 1.0
        Symbol: i (confidence: 0.9900000095367432
        Symbol: a (confidence: 1.0
        Symbol: n (confidence: 1.0
Word text: name (confidence: 0.9900000095367432
        Symbol: n (confidence: 0.9900000095367432
        Symbol: a (confidence: 1.0
        Symbol: m (confidence: 0.9900000095367432
        Symbol: e (confidence: 1.0
block confidence: 0.9900000095367432
paragraph confidence: 0.9900000095367432
Word text: desc (confidence: 0.9900000095367432
        Symbol: d (confidence: 0.9900000095367432
        Symbol: e (confidence: 1.0
        Symbol: s (confidence: 1.0
        Symbol: c (confidence: 1.0
block confidence: 0.9900000095367432
paragraph confidence: 0.9900000095367432
Word text: code (confidence: 0.9900000095367432
        Symbol: c (confidence: 0.9900000095367432
        Symbol: o (confidence: 1.0
        Symbol: d (confidence: 0.9900000095367432
        Symbol: e (confidence: 1.0
block confidence: 0.9900000095367432
paragraph confidence: 0.9900000095367432
Word text: 112121223 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 1.0
        Symbol: 2 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 0.9900000095367432
        Symbol: 2 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 0.9900000095367432
        Symbol: 2 (confidence: 1.0
        Symbol: 2 (confidence: 0.9900000095367432
        Symbol: 3 (confidence: 0.9900000095367432
Word text: Benz (confidence: 0.9900000095367432
        Symbol: B (confidence: 0.9900000095367432
        Symbol: e (confidence: 0.9900000095367432
        Symbol: n (confidence: 0.9900000095367432
        Symbol: z (confidence: 0.9900000095367432
block confidence: 0.9900000095367432
paragraph confidence: 0.9900000095367432
Word text: 10 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 0.9900000095367432
        Symbol: 0 (confidence: 0.9900000095367432
Word text: - (confidence: 0.9900000095367432
        Symbol: - (confidence: 0.9900000095367432
Word text: 10 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 1.0
        Symbol: 0 (confidence: 0.9900000095367432
Word text: - (confidence: 0.9900000095367432
        Symbol: - (confidence: 0.9900000095367432
Word text: 19 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 0.9900000095367432
        Symbol: 9 (confidence: 0.9900000095367432
block confidence: 0.9800000190734863
paragraph confidence: 0.9800000190734863
Word text: 10 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 0.9900000095367432
        Symbol: 0 (confidence: 0.9900000095367432
Word text: - (confidence: 0.9900000095367432
        Symbol: - (confidence: 0.9900000095367432
Word text: 10 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 0.9900000095367432
        Symbol: 0 (confidence: 0.9900000095367432
Word text: - (confidence: 0.9900000095367432
        Symbol: - (confidence: 0.9900000095367432
Word text: 20 (confidence: 0.9800000190734863
        Symbol: 2 (confidence: 0.9900000095367432
        Symbol: 0 (confidence: 0.9800000190734863
block confidence: 0.9599999785423279
paragraph confidence: 0.9599999785423279
Word text: Jhondoe (confidence: 0.9599999785423279
        Symbol: J (confidence: 0.8100000023841858
        Symbol: h (confidence: 0.9900000095367432
        Symbol: o (confidence: 1.0
        Symbol: n (confidence: 0.9900000095367432
        Symbol: d (confidence: 0.9900000095367432
        Symbol: o (confidence: 0.9900000095367432
        Symbol: e (confidence: 0.9800000190734863
block confidence: 0.9700000286102295
paragraph confidence: 0.9700000286102295
Word text: nil (confidence: 0.9700000286102295
        Symbol: n (confidence: 0.9900000095367432
        Symbol: i (confidence: 0.9900000095367432
        Symbol: l (confidence: 0.9399999976158142
block confidence: 0.9900000095367432
paragraph confidence: 0.9900000095367432
Word text: 12244334 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 0.9900000095367432
        Symbol: 2 (confidence: 1.0
        Symbol: 2 (confidence: 0.9900000095367432
        Symbol: 4 (confidence: 0.9900000095367432
        Symbol: 4 (confidence: 0.9900000095367432
        Symbol: 3 (confidence: 0.9900000095367432
        Symbol: 3 (confidence: 0.9900000095367432
        Symbol: 4 (confidence: 0.9900000095367432
block confidence: 0.9900000095367432
paragraph confidence: 0.9900000095367432
Word text: Baleno (confidence: 0.9900000095367432
        Symbol: B (confidence: 0.9900000095367432
        Symbol: a (confidence: 1.0
        Symbol: l (confidence: 0.9900000095367432
        Symbol: e (confidence: 0.9900000095367432
        Symbol: n (confidence: 0.9900000095367432
        Symbol: o (confidence: 0.9900000095367432
block confidence: 0.9900000095367432
paragraph confidence: 0.9900000095367432
Word text: 20 (confidence: 0.9900000095367432
        Symbol: 2 (confidence: 0.9900000095367432
        Symbol: 0 (confidence: 0.9900000095367432
Word text: - (confidence: 0.9900000095367432
        Symbol: - (confidence: 0.9900000095367432
Word text: 10 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 0.9900000095367432
        Symbol: 0 (confidence: 0.9900000095367432
Word text: - (confidence: 0.9900000095367432
        Symbol: - (confidence: 0.9900000095367432
Word text: 10 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 0.9900000095367432
        Symbol: 0 (confidence: 0.9900000095367432
block confidence: 0.8899999856948853
paragraph confidence: 0.8899999856948853
Word text: | (confidence: 0.4300000071525574
        Symbol: | (confidence: 0.4300000071525574
Word text: 20 (confidence: 0.8399999737739563
        Symbol: 2 (confidence: 0.699999988079071
        Symbol: 0 (confidence: 0.9800000190734863
Word text: - (confidence: 0.949999988079071
        Symbol: - (confidence: 0.949999988079071
Word text: 20 (confidence: 0.9800000190734863
        Symbol: 2 (confidence: 0.9900000095367432
        Symbol: 0 (confidence: 0.9800000190734863
Word text: - (confidence: 1.0
        Symbol: - (confidence: 1.0
Word text: 10 (confidence: 0.9900000095367432
        Symbol: 1 (confidence: 0.9900000095367432
        Symbol: 0 (confidence: 1.0
block confidence: 0.9300000071525574
paragraph confidence: 0.9300000071525574
Word text: Machel (confidence: 0.9300000071525574
        Symbol: M (confidence: 0.9200000166893005
        Symbol: a (confidence: 0.949999988079071
        Symbol: c (confidence: 0.9200000166893005
        Symbol: h (confidence: 0.9599999785423279
        Symbol: e (confidence: 0.949999988079071
        Symbol: l (confidence: 0.9200000166893005
block confidence: 0.9900000095367432
paragraph confidence: 0.9900000095367432
Word text: nil (confidence: 0.9900000095367432
        Symbol: n (confidence: 0.9900000095367432
        Symbol: i (confidence: 1.0
        Symbol: l (confidence: 0.9900000095367432
block confidence: 0.7599999904632568
paragraph confidence: 0.7599999904632568
Word text: D901 (confidence: 0.7200000286102295
        Symbol: D (confidence: 0.6499999761581421
        Symbol: 9 (confidence: 0.8700000047683716
        Symbol: 0 (confidence: 0.49000000953674316
        Symbol: 1 (confidence: 0.8999999761581421
Word text: . (confidence: 0.6399999856948853
        Symbol: . (confidence: 0.6399999856948853
Word text: 27 (confidence: 0.8600000143051147
        Symbol: 2 (confidence: 0.9399999976158142
        Symbol: 7 (confidence: 0.7799999713897705
Word text: ' (confidence: 0.9300000071525574
        Symbol: ' (confidence: 0.9300000071525574
Word text: RIOR (confidence: 0.7400000095367432
        Symbol: R (confidence: 0.9599999785423279
        Symbol: I (confidence: 0.7799999713897705
        Symbol: O (confidence: 0.8399999737739563
        Symbol: R (confidence: 0.4000000059604645

数据集中存在代码的最后一列是手写的，所以有人能帮助我在存在手写文本和自动文本时如何识别文本吗

如果不是google cloud vision，是否有任何方法可以根据opencv中的每一行查找并打印数据？

您可以尝试通过删除水平线和垂直线来预处理图像，这样可以获得更好的输出效果

方法#1：水平/垂直内核和“填充”轮廓以删除线条

使用PyteSeract OCR的结果

Worksheet
Car number - 1002110021
Data - Endter
Shell
Qilchange
Phone 982838238823
email xvz@xyz com
id name pod pos Technician desc code

name
112121223 Benz 10-10-19 10-10-20 Jhondoe nil

ve O \

12244334 Baleno 20-10-10 20-20-10 Machel nil \ \

它似乎无法捕获手写文本，您可能需要使用机器/深度学习来训练您自己的分类器，但它对于自动文本非常有效。我们使用

--psm 6

配置选项来假定一个统一的文本块。查看更多配置选项

import cv2
import pytesseract

pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe"

image = cv2.imread('result.png')
data = pytesseract.image_to_string(image, lang='eng',config='--psm 6')
print(data)

为了提取表格结构，amazon提供了一个名为textract的API。你可以探索一下。可以很好地处理简单的表。这里是链接：它也适用于手写文本吗？是的，但不如它适用于打印文本。Azure API最适用于手写文本。甚至比GCP更好。哪一种是手写文本的最佳深度学习算法？请查看并。手写文本识别是一个活跃的研究领域，所以我不确定是否有一个“最佳”的算法。我见过它们，但准确率并不令人印象深刻。这可能是你能找到的最好的开源结果，你可以尝试自己训练

Worksheet
Car number - 1002110021
Data - Endter
Shell
Qilchange
Phone 982838238823
email xvz@xyz com
id name pod pos Technician desc code

name
112121223 Benz 10-10-19 10-10-20 Jhondoe nil

ve O \

12244334 Baleno 20-10-10 20-20-10 Machel nil \ \

import cv2
import pytesseract

pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe"

image = cv2.imread('result.png')
data = pytesseract.image_to_string(image, lang='eng',config='--psm 6')
print(data)