简介： 本文介绍通过ModelScope来完成光学字符识别（OCR）这一应用，该应用使用两个模型：
- 文本检测（ocr_detection）
- 文本识别（ocr_recognition）
操作步骤
参考快速开始
环境准备
为了更快的体验产品，这里选择了使用ModelScope提供的远程环境，即Notebook进行开发，更加便捷。
- “快速开始”文档中，并未给出进入notebook的链接，需要从个人中心进入，https://modelscope.cn/#/my/mynotebook
模型调试：文本检测
参考：https://www.modelscope.cn/models/damo/cv_resnet18_ocr-detection-line-level_damo/summary
- 准备图像文件。
- 将图像文件上传至notebook（可拖拽），或使用url。
- 输入下列代码。
from modelscope.pipelines import pipeline from modelscope.utils.constant import Tasks import cv2 ocr_detection = pipeline(Tasks.ocr_detection, model='damo/cv_resnet18_ocr-detection-line-level_damo') # ocr_detection = pipeline(Tasks.ocr_detection, model='damo/cv_resnet18_ocr-detection-word-level_damo') # read file img_path = 'ocr_detection.jpg' img = cv2.imread(img_path) result = ocr_detection(img) print(result) # or read url img_url = 'https://modelscope.oss-cn-beijing.aliyuncs.com/test/images/ocr_detection.jpg' result_url = ocr_detection(img_url) print(result_url)
上面展示的是文本行检测模型的使用方法。
如需使用单词检测模型，请替换为第6行注释的模型，并参考https://www.modelscope.cn/models/damo/cv_resnet18_ocr-detection-word-level_damo/summary。
模型调试：文本识别
参考：https://www.modelscope.cn/models/damo/cv_convnextTiny_ocr-recognition-general_damo/summary
- 准备图像文件。
- 将图像文件上传至notebook（可拖拽），或使用url。
- 输入下列代码。
from modelscope.pipelines import pipeline from modelscope.utils.constant import Tasks import cv2 ocr_recognition = pipeline(Tasks.ocr_recognition, model='damo/cv_convnextTiny_ocr-recognition-general_damo') # ocr_recognition = pipeline(Tasks.ocr_recognition, model='damo/cv_convnextTiny_ocr-recognition-scene_damo') # ocr_recognition = pipeline(Tasks.ocr_recognition, model='damo/cv_convnextTiny_ocr-recognition-document_damo') # ocr_recognition = pipeline(Tasks.ocr_recognition, model='damo/cv_convnextTiny_ocr-recognition-handwritten_damo') # read file img_path = 'ocr_recognition.jpg' img = cv2.imread(img_path) result = ocr_recognition(img) print(result) # or read url img_url = 'http://duguang-labelling.oss-cn-shanghai.aliyuncs.com/mass_img_tmp_20220922/ocr_recognition.jpg' result_url = ocr_recognition(img_url) print(result_url)
上面展示的是通用文本识别模型的使用方法。
如需使用自然场景文本识别模型，请替换为第6行注释的模型，并参考https://www.modelscope.cn/models/damo/cv_convnextTiny_ocr-recognition-scene_damo/summary。
如需使用印刷文档文本识别模型，请替换为第7行注释的模型，并参考https://www.modelscope.cn/models/damo/cv_convnextTiny_ocr-recognition-document_damo/summary。
如需使用手写文本识别模型，请替换为第8行注释的模型，并参考https://www.modelscope.cn/models/damo/cv_convnextTiny_ocr-recognition-handwritten_damo/summary。
模型调试：检测识别串联
有了上述的基础，我们串联文本检测和文本识别模型，以实现完整的OCR功能，输入下列代码。
from modelscope.pipelines import pipeline from modelscope.utils.constant import Tasks import numpy as np import cv2 import math # scripts for crop images def crop_image(img, position): def distance(x1,y1,x2,y2): return math.sqrt(pow(x1 - x2, 2) + pow(y1 - y2, 2)) position = position.tolist() for i in range(4): for j in range(i+1, 4): if(position[i][0] > position[j][0]): tmp = position[j] position[j] = position[i] position[i] = tmp if position[0][1] > position[1][1]: tmp = position[0] position[0] = position[1] position[1] = tmp if position[2][1] > position[3][1]: tmp = position[2] position[2] = position[3] position[3] = tmp x1, y1 = position[0][0], position[0][1] x2, y2 = position[2][0], position[2][1] x3, y3 = position[3][0], position[3][1] x4, y4 = position[1][0], position[1][1] corners = np.zeros((4,2), np.float32) corners[0] = [x1, y1] corners[1] = [x2, y2] corners[2] = [x4, y4] corners[3] = [x3, y3] img_width = distance((x1+x4)/2, (y1+y4)/2, (x2+x3)/2, (y2+y3)/2) img_height = distance((x1+x2)/2, (y1+y2)/2, (x4+x3)/2, (y4+y3)/2) corners_trans = np.zeros((4,2), np.float32) corners_trans[0] = [0, 0] corners_trans[1] = [img_width - 1, 0] corners_trans[2] = [0, img_height - 1] corners_trans[3] = [img_width - 1, img_height - 1] transform = cv2.getPerspectiveTransform(corners, corners_trans) dst = cv2.warpPerspective(img, transform, (int(img_width), int(img_height))) return dst def order_point(coor): arr = np.array(coor).reshape([4, 2]) sum_ = np.sum(arr, 0) centroid = sum_ / arr.shape[0] theta = np.arctan2(arr[:, 1] - centroid[1], arr[:, 0] - centroid[0]) sort_points = arr[np.argsort(theta)] sort_points = sort_points.reshape([4, -1]) if sort_points[0][0] > centroid[0]: sort_points = np.concatenate([sort_points[3:], sort_points[:3]]) sort_points = sort_points.reshape([4, 2]).astype('float32') return sort_points ocr_detection = pipeline(Tasks.ocr_detection, model='damo/cv_resnet18_ocr-detection-line-level_damo') ocr_recognition = pipeline(Tasks.ocr_recognition, model='damo/cv_convnextTiny_ocr-recognition-general_damo') img_path = 'ocr_detection.jpg' image_full = cv2.imread(img_path) det_result = ocr_detection(image_full) det_result = det_result['polygons'] for i in range(det_result.shape[0]): pts = order_point(det_result[i]) image_crop = crop_image(image_full, pts) result = ocr_recognition(image_crop) print("box: %s" % ','.join([str(e) for e in list(pts.reshape(-1))])) print("text: %s" % result['text'])
至此，我们已经在代码层面完成了两个模型的调试和串联，并实现了完整的OCR功能。
