近年来，随着立体视觉技术的发展和相机成像质量的提升，基于双目或多目相机的高精度3D视觉检测技术得到了长足的发展及广泛的应用。3D视觉检测中，高精度、灵活的相机标定是实现准确且快速测量的基础。然而，现有的相机标定技术应用于实际的大视场测量环境时，存在大型标定靶标制造困难、不易摆放与移动等问题，使标定过程在现场复杂环境条件下难以实施。本项目提出了一种近景摄影测量与全站仪相结合的大视场相机标定方法，在此基础上，发展并实现了一种大视场3D视觉检测技术，可在现场复杂工况下实现大型工件/结构三维变形的实时检测。具体的研究内容及成果如下：

（1）提出了一种近景摄影测量与全站仪相结合的大视场双目相机标定方法。该方法首先利用单个相机近距离采集多张不同姿态摆放的标定靶标图像，并基于近景摄影测量技术，实现图像的定向、标志点三维坐标重建及单个相机内参数的求解；其次，在测量视场中布置多个标志点，控制双目相机同步拍摄一张标志点图像，并利用全站仪观测出视场中任意两个相距较远的标志点的距离作为近景摄影测量中的比例尺，解算出标志点的三维坐标及双目相机的外参数。本文提出的标定方法兼顾了相机标定的精度及灵活性，具有高精度、高柔性的特点。

（2）提出一种卡尔曼滤波与极线约束相结合的序列图像中标志点三维坐标重建方法。该方法首先利用卡尔曼滤波器对历史帧图像中标志点集合的运动状态进行估计，并通过全局最邻近方法完成与当前帧标志点集的关联匹配；之后，利用极线约束实现立体像对中同名标志点的匹配；最后，基于标定阶段获得的投影变换矩阵实现序列图中标志点的三维坐标重建。实验结果表明该方法能快速稳定地实现序列图像中同名标志点的正确匹配及三维坐标精确重建。

（3）研制了一套大视场3D视觉检测系统，即光学三维动态变形监测系统（Dynamic Measurement System，DMS）。该系统硬件部分主要包括工业相机组、全站仪、标定靶标及高性能计算机。系统软件主要利用C++、MFC及CUDA编程语言进行开发。该系统可实现大型工件表面关键点三维坐标、位移、轨迹、速度、加速度、频率等的实时检测。

实验结果表明，对于30 m × 20 m的测量视场，本文提出的标定方法的重投影误差约为0.791个像素，提出的大视场3D视觉检测技术的重复性标准偏差约为0.796 mm，依据国标《GB/T 27663-2011》获得的平均测距标准偏差约为4.072 mm。

Recently, with the development of stereo vision technology and the improvement of camera imaging quality, high-precision 3D visual inspection based on binocular or multiple cameras has been considerably developed and widely used. In 3D visual inspection tasks, a high-precision and flexible camera calibration is the basis to achieve accurate and rapid measurement. However, when the existing camera calibration technologies is applied in the actual large field of view (FOV) measurement environment, there are some difficulties in manufacturing, placement and movement of large calibration target, which makes the calibration process hard to implement under complex on-site environment conditions. A large FOV camera calibration method combing close-range photogrammetry and a total station is proposed in this project. According to this, a kind of large FOV visual inspection technique is developed and completed, which can realize the real-time 3D deformation inspection of large workpieces/structures under complex on-site working condition. The specific research contents and results are as follows:

 

(1) A binocular camera calibration method combing close-range photogrammetry and a total station in large FOV is proposed. In this method, every camera is used to capture multiple calibration target images with different poses at close range. Then the orientation of image, the 3D coordinates reconstruction of reference points and the camera’s intrinsic parameters will be solved based on close-range photogrammetry technology. Besides, multiple reference points are arranged in the measurement field, whose image will be shot by binocular camera simultaneously. Moreover, the distance between any two far apart reference points in the FOV observed by total station is used as the scale of close-range photogrammetry to solve the 3D coordinates of calibration points and the camera’s extrinsic parameters. The calibration method proposed in this thesis takes into account both precision and flexibility, and has the characteristic of high-precision and high-flexibility.

 

(2) A 3D reconstruction method for reference points in sequential images combining Kalman filter with epipolar constraint is presented. Firstly, Kalman filter is used in this method to estimate the motion state of the reference point sets in previous frames which will be matched with the point set in current frame through Global Nearest Neighbor (GNN) method. Afterwards, the matching task of corresponding points between stereo images is realized by means of epipolar constraints. Finally, the 3D reconstruction of reference points in sequence images will be completed based on the projection transformation matrix obtained in the calibration stage. Experimental results show that this method can realize the correct matching and accurate 3D reconstruction of the corresponding points in the sequence of images quickly and stably.

 

(3) A large FOV 3D visual inspection system, named optical 3D dynamic deformation measurement system (DMS), is developed. The hardware part of this system mainly includes an industrial camera group, a total station, a calibration target and a high-performance computer. The software part of this system is mainly developed by C++, MFC and CUDA programming language. The system can realize real-time detection of 3D coordinates, displacement, trajectory, speed, acceleration, frequency, etc. of the key points on large workpieces’ surfaces.

 

The experimental results indicate that for 30 m × 20 m measurement FOV, the reprojection error of the calibration method proposed is about 0.791 pixel, the repeatability standard deviation of the large FOV 3D visual inspection technique proposed is about 0.796 mm，the average standard deviation of distance measurement is about 4.072 mm according to the national standard “GB/T 27663-2011”.

[1] 陈芮. 近景摄影测量在车辆静力变形测量中的应用和研究[D]. 重庆: 重庆交通大学, 2017.
[2] 冯文灏. V-STARS型工业摄像测量系统介绍[J]. 测绘信息与工程, 2000, 25(4): 42-47.
[3] 徐巧玉. 大型装备在线三维视觉测量系统关键技术研究[D]. 哈尔滨: 哈尔滨工业大学, 2007.
[4] 梁晋, 郭翔, 胡浩, 等. 机械与材料力学性能的三维全场变形与应变快速检测研究[J]. 中国工程科学, 2013, 15(1): 51-56+105.
[5] Zhang Z. A Flexible New Technique for Camera Calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334.
[6] Tsai R Y. A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses[J]. IEEE Journal on Robotics & Automation, 1987, 3(4): 323-344.
[7] Tsai R Y. An efficient and accurate camera calibration technique for 3D machine vision[C]. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, USA, June 22-26 1986, pp. 364-374.
[8] Ma S. A self-calibration technique for active vision systems[J]. IEEE Transactions on Robotics and Automation, 1996, 12(1): 114-120.
[9] Hartley R I. Self-Calibration of Stationary Cameras[J]. International Journal of Computer Vision, 1997, 22(1): 5-23.
[10] 姜广文, 晁志超, 伏思华, 等. 基于可控旋转的像机高精度标定技术[J]. 光学学报, 2010, 30(5): 1308-1314.
[11] Maybank S J, Faugeras O D. A theory of self-calibration of a moving camera[J]. International Journal of Computer Vision, 1992, 8(2): 123-151.
[12] Faugeras O D, Luong Q T, Maybank S J. Camera Self-Calibration: Theory and Experiments[C]. Proceedings of the Second European Conference on Computer Vision, Santa Margherita Ligure, Italy, May 19-22 1992, pp. 321-334.
[13] 苑云, 张小虎, 朱肇昆, 等.大型风力发电叶片变形的摄像测量方法研究[J]. 计算机应用, 2012, 32(1): 114-117.
[14] 胡浩, 梁晋, 唐正宗, 等. 大视场多像机视频测量系统的全局标定[J]. 光学精密工程, 2012, 20(2): 369-378.
[15] 冯萍, 魏振忠. 光笔式大视场三维视觉测量系统[J]. 光学精密工程, 2013, 21(09): 2217-2224.
[16] 杨景豪. 大视场视觉测量系统标定方法研究[D]. 大连: 大连理工大学, 2018.
[17] 郭书基, 史立芳, 曹阿秀, 等. 基于大视场人工复眼定位技术[J]. 光子学报, 2016, 45(5): 93-98.
[18] 刘晓磊, 张兴国, 吴衡, 等. 近景大视场角分区域标定方法研究[J]. 计算机测量与控制, 2017, 25(11): 262-264+268.
[19] Jiang H, Sun X. A Camera Calibration Method Based on Differential GPS System for Large Field Measurement[C]. International Conference on Intelligent Robotics and Applications (ICIRA), Wuhan, China, August 16-18 2017, pp. 508-519.
[20] 余淑真, 欧巧凤, 熊邦书, 等. 基于散焦图像的大视场立体视觉标定方法[J]. 应用科学学报, 2019, 37(6): 795-805.
[21] 郜泽仁. 数字图像相关的精度验证方法及工程化应用研究[D]. 合肥: 中国科学技术大学, 2020.
[22] Paulsen U S, Schmidt T, Erne O. Developments in Large Wind Turbine Modal Analysis Using Point Tracking Videogrammetry[C]. Proceedings of the 28th IMAC, A Conference on Structural Dynamics, Jacksonville, USA, June 2010, pp. 187-198.
[23] Kushner L K, Littell J, Cassell A. Photogrammetry of a Hypersonic Inflatable Aerodynamic Decelerator[C], AIAA Aerodynamic Decelerator Systems (ADS) Conference, Daytona Beach, USA, March 25-28 2013, pp. 1-15.
[24] Lundstrom T, Baqersad J, Niezrecki C. Monitoring the Dynamics of a Helicopter Main Rotor With High-Speed Stereophotogrammetry[J]. Experimental Techniques, 2015, 40(3): 907-919.
[25] Brown D C. Decentering distortion of lenses[J]. Photogrammetric Engineering, 1966, 32(3): 444-462.
[26] 肖振中. 基于工业摄影和机器视觉的三维形貌与变形测量关键技术研究[D]. 西安: 西安交通大学, 2010.
[27] Abdel-Aziz Y I, Karara H M. Direct linear transformation from comparator coordinates into object space in close-range photogrammetry[C]. Proceedings of the ASP/UI Symposium on Close-Range Photogrammetry, Falls Church, VA, January 1971, pp. 1-18.
[28] 张一平, 龚志辉, 王勃. 基于大倾角的角锥体空间后方交会研究[J]. 测绘工程, 2012, 21(2): 17-20.
[29] 张春森, 张觅. 依初值与无初值单位四元数空间后方交会的比较[J]. 测绘科学, 2015, 40(4): 3-6+10.
[30] Luhmann S R T, Kyle S, Harley I. Close Range Photogrammetry: Principles, Techniques and Applications[M]. London: Whittles Publishing, 2006.
[31] 晋毅东. 大视场相机标定方法[D]. 西安: 西安电子科技大学, 2021.
[32] Belongie S, Malik J, and Puzicha J. Shape matching and object recognition using shape contexts[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(4): 509-522.
[33] 郇战, 陈学杰, 万彩艳, 等. 结合形状序列与LTN的步态身份识别[J]. 计算机工程与设计, 2019, 40(9): 2625-2630.
[34] Fischler M A, Bolles R C. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography[J]. Communications of the ACM, 1981, 24(6): 381-395.
[35] Kalman R E. A New Approach to Linear Filtering and Prediction Problems[J]. Transactions of the ASME Journal of Basic Engineering, 1960, 82(1): 35-45.
[36] Bourgeois F, Lassalle J C. An extension of the Munkres algorithm for the assignment problem to rectangular matrices[J]. Communications of the ACM, 1971, 14(12): 802-806.
[37] Bouguet J Y. Camera Calibration Toolbox for Matlab[EB/OL]. http://www.vision.caltech.edu/ bouguetj/calib_doc/. [2022-03-01].
[38] 王庆春, 何晓燕, 梁俊龙. 对景深理论的证明及探讨[J]. 陕西工学院学报, 2001, 17(1): 72-75.
[39] 闫文娟, 韩新月, 程朗, 等. 大型风电叶片的结构分析和测试[J]. 可再生能源, 2014, 32(8): 1140-1143.
[40] 萨昊亮, 李成良, 余启明, 等. 风电叶片疲劳试验振动分析与研究[J]. 玻璃钢/复合材料, 2013, 2: 57-59.
[41] 石可重, 赵晓路, 徐建中. 大型风电机组叶片疲劳试验研究[J]. 太阳能学报, 2011, 32(8): 1264-1268.
[42] 李成良, 张金峰, 张登刚, 等. 大型风电叶片全尺寸结构测试准确性研究[J]. 复合材料科学与工程, 2021, 4: 83-88.
[43] 杨婷, 杜文超, 杨贺, 等. 风电叶片静载荷应变测试试验[J]. 实验室研究与探索, 2011, 30(11): 33-35+39.
[44] 羊森林, 黄永东, 钟贤和, 等. 某大型风机叶片静力加载失效破坏原因分析[J]. 东方汽轮机, 2015, 2: 25-29.