最近打省电赛，与双目立体视觉相关。要实现双目测距首先要进行摄像头的标定，单目标定主要是为了测定摄像头的内参矩阵和畸变矩阵。这方面有大量的博客和论文可以参考，以下贴一下《opencv计算机视觉编程攻略（第三版）》一书中的标定程序。
好几个月过去，没想到也有几百人看过这篇文章了。现在看来感觉自己的认识还是很粗浅的，只是单纯地调用下API，很多概念只是糊弄过去了。最近我看了《Learning OpenCV3》一书，上面关于标定和立体视觉方面的原理讲得很好，所以原理性的东西可以看看我的这篇读书笔记，当然不打算深入原理也无所谓了。
—— Jacob杨帮帮 10/29/2018
标定用的棋盘格
1.获取标定板
淘宝上有高精度的标定板，但是价格感人。精确度要求不高完全可以自己打印。这里我使用的是一位前辈的python程序自制一个棋盘格标定板 - 简书
"""
Created on Fri Jan 5 10:57:34 2018
@author: 晚晴风
"""
import cv2
import numpy as np
width = 450
height = 350
length = 50
image = np.zeros((width,height),dtype = np.uint8)
print(image.shape[0],image.shape[1])
for j in range(height):
for i in range(width):
if((int)(i/length) + (int)(j/length))%2:
image[i,j] = 255;
cv2.imwrite("pic/chess.jpg",image)
cv2.imshow("chess",image)
cv2.waitKey(0)
打印时根据需要缩放大小即可
2.opencv3标定程序
使用的是书中的程序。作者 Robert Laganiere 将一些关键操作封装起来，如寻找棋盘内角。
bool findChessboardCorners( InputArray image, Size patternSize, OutputArray corners,
int flags = CALIB_CB_ADAPTIVE_THRESH
+ CALIB_CB_NORMALIZE_IMAGE );
精确角点到亚像素（sub-pixel）级。
void cornerSubPix( InputArray image, InputOutputArray corners,
Size winSize, Size zeroZone,
TermCriteria criteria );
根据自己生成的 objectPoints 二维向量和由图片测得的 imagePoints 二维向量来进行标定，并生成内参矩阵（cameraMatrix），畸变矩阵（distCoeffs），旋转矢量（rvecs），平移矢量（tvecs）等参数。
double calibrateCamera( InputArrayOfArrays objectPoints,
InputArrayOfArrays imagePoints, Size imageSize,
InputOutputArray cameraMatrix, InputOutputArray distCoeffs,
OutputArrayOfArrays rvecs, OutputArrayOfArrays tvecs,
int flags = 0, TermCriteria criteria = TermCriteria(
TermCriteria::COUNT + TermCriteria::EPS, 30, DBL_EPSILON) );
用内参矩阵和畸变矩阵生成用于矫正图片的map，分x轴和y轴
void initUndistortRectifyMap( InputArray cameraMatrix, InputArray distCoeffs,
InputArray R, InputArray newCameraMatrix,
Size size, int m1type, OutputArray map1,
OutputArray map2 );
使用上面的map1和map2进行矫正
void remap( InputArray src, OutputArray dst,
InputArray map1, InputArray map2,
int interpolation, int borderMode = BORDER_CONSTANT,
const Scalar& borderValue = Scalar());
头文件如下。虽然不是很懂设计模式，但可以看到作者的c++功底还是可以的
/*------------------------------------------------------------------------------------------*\
This file contains material supporting chapter 11 of the book:
OpenCV3 Computer Vision Application Programming Cookbook
Third Edition
by Robert Laganiere, Packt Publishing, 2016.
This program is free software; permission is hereby granted to use, copy, modify,
and distribute this source code, or portions thereof, for any purpose, without fee,
subject to the restriction that the copyright notice may not be removed
or altered from any source or altered source distribution.
The software is released on an as-is basis and without any warranties of any kind.
In particular, the software is not guaranteed to be fault-tolerant or free from failure.
The author disclaims all warranties with regard to this software, any use,
and any consequent failure, is purely the responsibility of the user.
Copyright (C) 2016 Robert Laganiere, www.laganiere.name
\*------------------------------------------------------------------------------------------*/
#ifndef CAMERACALIBRATOR_H
#define CAMERACALIBRATOR_H
#include <vector>
#include <iostream>
#include <opencv2/core.hpp>
#include "opencv2/imgproc.hpp"
#include "opencv2/calib3d.hpp"
#include <opencv2/highgui.hpp>
class CameraCalibrator {
// input points:
// the points in world coordinates
// (each square is one unit)
std::vector<std::vector<cv::Point3f> > objectPoints;
// the image point positions in pixels
std::vector<std::vector<cv::Point2f> > imagePoints;
// output Matrices
cv::Mat cameraMatrix;
cv::Mat distCoeffs;
// flag to specify how calibration is done
int flag;
// used in image undistortion
cv::Mat map1,map2;
bool mustInitUndistort;
public:
CameraCalibrator() : flag(0), mustInitUndistort(true) {}
// Open the chessboard images and extract corner points
int addChessboardPoints(const std::vector<std::string>& filelist, cv::Size & boardSize, std::string windowName="");
// Add scene points and corresponding image points
void addPoints(const std::vector<cv::Point2f>& imageCorners, const std::vector<cv::Point3f>& objectCorners);
// Calibrate the camera
double calibrate(const cv::Size imageSize);
// Set the calibration flag
void setCalibrationFlag(bool radial8CoeffEnabled=false, bool tangentialParamEnabled=false);
// Remove distortion in an image (after calibration)
cv::Mat remap(const cv::Mat &image, cv::Size &outputSize );
// Getters
cv::Mat getCameraMatrix() { return cameraMatrix; }
cv::Mat getDistCoeffs() { return distCoeffs; }
};
#endif // CAMERACALIBRATOR_H
具体实现如下
/*------------------------------------------------------------------------------------------*\
This file contains material supporting chapter 11 of the book:
OpenCV3 Computer Vision Application Programming Cookbook
Third Edition
by Robert Laganiere, Packt Publishing, 2016.
This program is free software; permission is hereby granted to use, copy, modify,
and distribute this source code, or portions thereof, for any purpose, without fee,
subject to the restriction that the copyright notice may not be removed
or altered from any source or altered source distribution.
The software is released on an as-is basis and without any warranties of any kind.
In particular, the software is not guaranteed to be fault-tolerant or free from failure.
The author disclaims all warranties with regard to this software, any use,
and any consequent failure, is purely the responsibility of the user.
Copyright (C) 2016 Robert Laganiere, www.laganiere.name
\*------------------------------------------------------------------------------------------*/
#include "CameraCalibrator.h"
// Open chessboard images and extract corner points
int CameraCalibrator::addChessboardPoints(
const std::vector<std::string>& filelist, // list of filenames containing board images
cv::Size & boardSize, // size of the board
std::string windowName) { // name of window to display results
// if null, no display shown
// the points on the chessboard
std::vector<cv::Point2f> imageCorners;
std::vector<cv::Point3f> objectCorners;
// 3D Scene Points:
// Initialize the chessboard corners
// in the chessboard reference frame
// The corners are at 3D location (X,Y,Z)= (i,j,0)
for (int i=0; i<boardSize.height; i++) {
for (int j=0; j<boardSize.width; j++) {
objectCorners.push_back(cv::Point3f(i, j, 0.0f));
}
}
// 2D Image points:
cv::Mat image; // to contain chessboard image
int successes = 0;
// for all viewpoints
for (int i=0; i<filelist.size(); i++) {
// Open the image
image = cv::imread(filelist[i],0);
// Get the chessboard corners
bool found = cv::findChessboardCorners(image, // image of chessboard pattern
boardSize, // size of pattern
imageCorners); // list of detected corners
// Get subpixel accuracy on the corners
if (found) {
cv::cornerSubPix(image, imageCorners,
cv::Size(5, 5), // half size of serach window
cv::Size(-1, -1),
cv::TermCriteria(cv::TermCriteria::MAX_ITER +
cv::TermCriteria::EPS,
30, // max number of iterations
0.1)); // min accuracy
// If we have a good board, add it to our data
if (imageCorners.size() == boardSize.area()) {
// Add image and scene points from one view
addPoints(imageCorners, objectCorners);
successes++;
}
}
if (windowName.length()>0 && imageCorners.size() == boardSize.area()) {
//Draw the corners
cv::drawChessboardCorners(image, boardSize, imageCorners, found);
cv::imshow(windowName, image);
cv::waitKey(500);
}
}
return successes;
}
// Add scene points and corresponding image points
void CameraCalibrator::addPoints(const std::vector<cv::Point2f>& imageCorners, const std::vector<cv::Point3f>& objectCorners) {
// 2D image points from one view
imagePoints.push_back(imageCorners);
// corresponding 3D scene points
objectPoints.push_back(objectCorners);
}
// Calibrate the camera
// returns the re-projection error
double CameraCalibrator::calibrate(const cv::Size imageSize)
{
// undistorter must be reinitialized
mustInitUndistort= true;
//Output rotations and translations
std::vector<cv::Mat> rvecs, tvecs;
// start calibration
return
cv::calibrateCamera(objectPoints, // the 3D points
imagePoints, // the image points
imageSize, // image size
cameraMatrix, // output camera matrix
distCoeffs, // output distortion matrix
rvecs, tvecs, // Rs, Ts
CV_CALIB_USE_INTRINSIC_GUESS ); // set options
// ,CV_CALIB_USE_INTRINSIC_GUESS);
}
// remove distortion in an image (after calibration)
cv::Mat CameraCalibrator::remap(const cv::Mat &image, cv::Size &outputSize) {
cv::Mat undistorted;
if (outputSize.height == -1)
outputSize = image.size();
if (mustInitUndistort) { // called once per calibration
// cv::Mat dist = (cv::Mat_<double>(1, 14) << -107.8067046233813, 3606.394522865697, -0.003208480233032964, 0.00212257161649465, -706.3301131023582, -107.3590793091425, 3559.830030448713, 855.5629043718579, 0, 0, 0, 0, 0, 0);
// dist = distCoeffs;
cv::initUndistortRectifyMap(
cameraMatrix, // computed camera matrix
distCoeffs, // computed distortion matrix
cv::Mat(), // optional rectification (none)
cv::Mat(), // camera matrix to generate undistorted
outputSize, // size of undistorted
CV_32FC1, // type of output map
map1, map2); // the x and y mapping functions
mustInitUndistort= false;
}
// Apply mapping functions
cv::remap(image, undistorted, map1, map2,
cv::INTER_LINEAR); // interpolation type
return undistorted;
}
// Set the calibration options
// 8radialCoeffEnabled should be true if 8 radial coefficients are required (5 is default)
// tangentialParamEnabled should be true if tangeantial distortion is present
void CameraCalibrator::setCalibrationFlag(bool radial8CoeffEnabled, bool tangentialParamEnabled) {
// Set the flag used in cv::calibrateCamera()
flag = 0;
if (!tangentialParamEnabled) flag += CV_CALIB_ZERO_TANGENT_DIST;
if (radial8CoeffEnabled) flag += CV_CALIB_RATIONAL_MODEL;
}
main函数如下。将PATH宏改为自己的路径,PICS_NUM改为照片数量即可
#include <iostream>
#include <iomanip>
#include <vector>
#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/features2d.hpp>
#include "CameraCalibrator.h"
#define PATH "F:/QtProjects/stero/stereo4/right"
#define PICS_NUM 17
int main()
{
cv::Mat image;
std::vector<std::string> filelist;
// generate list of chessboard image filename
// named chessboard01 to chessboard27 in chessboard sub-dir
for (int i=1; i<=PICS_NUM; i++) {
std::stringstream str;
str << PATH << std::setw(2) << std::setfill('0') << i << ".png";
std::cout << str.str() << std::endl;
filelist.push_back(str.str());
image= cv::imread(str.str(),0);
// cv::imshow("Board Image",image);
// cv::waitKey(500);
}
// Create calibrator object
CameraCalibrator cameraCalibrator;
// add the corners from the chessboard
cv::Size boardSize(8,6);
cameraCalibrator.addChessboardPoints(
filelist, // filenames of chessboard image
boardSize, "Detected points"); // size of chessboard
// calibrate the camera
cameraCalibrator.setCalibrationFlag(true,true);
cameraCalibrator.calibrate(image.size());
// Exampple of Image Undistortion
std::cout << filelist[5] << std::endl;
image = cv::imread(filelist[5],0);
cv::Size newSize(static_cast<int>(image.cols*1.5), static_cast<int>(image.rows*1.5));
cv::Mat uImage= cameraCalibrator.remap(image, newSize);
// display camera matrix
cv::Mat cameraMatrix= cameraCalibrator.getCameraMatrix();
cv::Mat distCoeffs=cameraCalibrator.getDistCoeffs();
std::cout << " Camera intrinsic: " << cameraMatrix.rows << "x" << cameraMatrix.cols << std::endl;
std::cout << cameraMatrix.at<double>(0,0) << " " << cameraMatrix.at<double>(0,1) << " " << cameraMatrix.at<double>(0,2) << std::endl;
std::cout << cameraMatrix.at<double>(1,0) << " " << cameraMatrix.at<double>(1,1) << " " << cameraMatrix.at<double>(1,2) << std::endl;
std::cout << cameraMatrix.at<double>(2,0) << " " << cameraMatrix.at<double>(2,1) << " " << cameraMatrix.at<double>(2,2) << std::endl;
std::cout << distCoeffs.rows << "x" <<distCoeffs.cols << std::endl;
std::cout << distCoeffs << std::endl;
for(int i = 0;i < distCoeffs.cols;i++)
{
std::cout << distCoeffs.at<double>(0,i) << " " ;
}
std::cout <<std::endl;
cv::namedWindow("Original Image");
cv::imshow("Original Image", image);
cv::namedWindow("Undistorted Image");
cv::imshow("Undistorted Image", uImage);
// Store everything in a xml file
cv::FileStorage fs("calib.xml", cv::FileStorage::WRITE);
fs << "Intrinsic" << cameraMatrix;
fs << "Distortion" << cameraCalibrator.getDistCoeffs();
cv::waitKey();
return 0;
}
效果如下
image.png
3.经验总结
- 照片要不同角度和距离拍摄，最好15张以上，不要偷懒。
- 出现矫正效果不好的情况，可以改变一下函数calibrateCamera中的参数flag。使用CV_CALIB_FIX_K3可以使畸变矩阵输出四个参数。根据网上博客的经验对于低端摄像头来说需要的参数数量一般使用5个或者4个比较合适。
