沈跃, 朱嘉慧, 刘慧, 孙力. 基于深度和彩色双信息特征源的Kinect植物图像拼接[J]. 农业工程学报, 2018, 34(5): 176-182. DOI: 10.11975/j.issn.1002-6819.2018.05.023
    引用本文: 沈跃, 朱嘉慧, 刘慧, 孙力. 基于深度和彩色双信息特征源的Kinect植物图像拼接[J]. 农业工程学报, 2018, 34(5): 176-182. DOI: 10.11975/j.issn.1002-6819.2018.05.023
    Shen Yue, Zhu Jiahui, Liu Hui, Sun Li. Plant image mosaic based on depth and color dual information feature source from Kinect[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2018, 34(5): 176-182. DOI: 10.11975/j.issn.1002-6819.2018.05.023
    Citation: Shen Yue, Zhu Jiahui, Liu Hui, Sun Li. Plant image mosaic based on depth and color dual information feature source from Kinect[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2018, 34(5): 176-182. DOI: 10.11975/j.issn.1002-6819.2018.05.023

    基于深度和彩色双信息特征源的Kinect植物图像拼接

    Plant image mosaic based on depth and color dual information feature source from Kinect

    • 摘要: 图像拼接对制作全景图具有重要作用,传统SIFT(scale-invariant feature transform)图像拼接方法受光照不均匀或风吹影响,存在错位和缺失等情况。针对这一问题,该文提出一种基于Kinect彩色和深度双信息特征源的图像拼接方法。首先对获取的植株彩色图像采用SIFT算法进行特征点提取和特征点匹配,利用Kinect收集到的植株深度数据去除误匹配,采用RANSAC算法寻找投影变换矩阵,最后通过最佳缝合线算法进行图像融合。室内和室外试验结果表明,该文基于Kinect彩色和深度双信息特征源的图像拼接方法能够有效克服光照、风吹等环境因素的影响,避免了图像缺失、亮暗差异、重影等拼接错误,该文方法图像拼接时间较短,平均匹配准确率达96.0%,较传统SIFT图像拼接方法平均准确率提高了5.9%。

       

      Abstract: Abstract: In agricultural production, image mosaic plays an important role for a panorama view when images are collected by on-board real-time sensor. Most of the agricultural operation is outside, and traditional method of image mosaic based on SIFT (scale-invariant feature transform) algorithm is affected by uneven illumination or wind and some other factors, so image after mosaic presents dislocation or absence of information and other mistakes. It is difficult to meet the requirements of the reliability of agricultural vehicle applications by using image feature element method for image mosaic. Aiming at this problem, the paper proposes a new method of image mosaic based on the color and depth dual information feature source from Kinect. Firstly, depth information and color images are collected by Kinect sensor from 3 different angels. Kinect sensor is fixed on the slider, and when the motion slider moves at a constant speed of 0.5 m/s, the Kinect moves at the same speed. The Kinect is invoked to obtain color image and depth information and save them every 0.6 s through MATLAB. Three consecutive time points are selected in order, and the color images of 3 angles are taken as the mosaic images. Secondly, SIFT algorithm is used to get feature points from color images. SIFT algorithm can extract the feature points which have invariance for illumination, affine and projection transformation. It is helpful to reduce the number of feature points matching and improve the speed and accuracy of feature point matching. Thirdly, feature points matches are gotten by similarity measure, and the Euclidean distance with high efficiency is used as the similarity measure in the 2 pictures in this paper. A key point in an image is gotten, and the closest 2 key points in another image are found. If the ratio of the nearest distance to the next closest distance is less than a certain threshold, then the pair of matching points are received. But some wrong matches exist with this method. Too many mismatches may result in mosaic errors. Therefore, a solution is needed to remove mismatches to improve the accuracy of the matches. From the nature of Kinect, if Kinect moves horizontally, the depth data of a fixed point is the same. Based on this characteristic, some mismatches would be removed. If the depth data of these 2 feature points are the same, the match is retained, and otherwise they would be removed. As a result, the accuracy of matches is improved. Next, by RANSAC (random sample consensus) algorithm, projection transformation matrix can be found. The RANSAC algorithm uses the least possible points to estimate the model and then as far as possible to expand scope of the influence of the model. The projection transformation matrix is more accurate than traditional image mosaic method on account of the removing of mismatches. At last, through the best suture line algorithm, image fusion is relatively smooth. From indoor and outdoor test, the mosaic method based on color and depth dual information feature source has obvious advantages, and it can effectively overcome the influence of light, wind and other environmental factors and avoid mosaic errors such as the loss of image and the difference of brightness. In the indoor test, the mosaic method of this article takes 9.70 s, the accuracy of matches is 92.9%, while traditional method based on SIFT algorithm takes 13.04 s, the accuracy of matches is 88.1%. In the outdoor test, the mosaic method of this article takes 71.15 s, the accuracy of matches is 99.1%, while traditional method based on SIFT algorithm takes 77.67 s, the accuracy of matches is 92.1%. So the mosaic method in this article takes less time than the traditional method based on SIFT algorithm. The data of mosaic accuracy show that the average matching accuracy of the method in this article is 96.0%, and the average accuracy is 5.9% higher than the traditional image mosaic method based on SIFT. So, this method can be further applied in other occasions of image mosaic. It can realize precise spraying of drug fertilizers and the control of pests and diseases based on information collected by Kinect.

       

    /

    返回文章
    返回