王粮局, 张立博, 段运红, 张铁中. 基于视觉伺服的草莓采摘机器人果实定位方法[J]. 农业工程学报, 2015, 31(22): 25-31. DOI: 10.11975/j.issn.1002-6819.2015.22.004
    引用本文: 王粮局, 张立博, 段运红, 张铁中. 基于视觉伺服的草莓采摘机器人果实定位方法[J]. 农业工程学报, 2015, 31(22): 25-31. DOI: 10.11975/j.issn.1002-6819.2015.22.004
    Wang Liangju, Zhang Libo, Duan Yunhong, Zhang Tiezhong. Fruit localization for strawberry harvesting robot based on visual servoing[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2015, 31(22): 25-31. DOI: 10.11975/j.issn.1002-6819.2015.22.004
    Citation: Wang Liangju, Zhang Libo, Duan Yunhong, Zhang Tiezhong. Fruit localization for strawberry harvesting robot based on visual servoing[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2015, 31(22): 25-31. DOI: 10.11975/j.issn.1002-6819.2015.22.004

    基于视觉伺服的草莓采摘机器人果实定位方法

    Fruit localization for strawberry harvesting robot based on visual servoing

    • 摘要: 为解决基于手眼系统的视觉伺服方法在草莓采摘机器人应用中存在的视觉信息反馈延迟大、频率低以及深度信息无法确定等带来的定位耗时长、精度低的问题,采用摄像机曝光信号触发控制卡进行高速位置锁存,结合位置传感器的反馈信息,来减少定位耗时;采用基于运动恢复结构的方法,提高果实采摘参数的精度。在垄坡和摄像机像平面的夹角为±10°范围内的情况下,针对包含1~3粒成熟草莓的果实域,采用直角坐标式机械臂草莓采摘机器人样机进行了定位试验。试验结果表明:定位时间在0.633~0.886 s之间;草莓深度信息的相对误差在?4.34%~0.95%范围内。

       

      Abstract: Abstract: As the complexity of the fruits' growth environment, in which the geometric parameters of the fruits and the relative position between fruits and robot need to be structured. The harvesting robots are equipped with custom-made, end-effectors and machine vision system to harvest precisely. Currently, stereo vision and eye-in-hand visual servoing are usually used to locate fruits. The former needs the image matching algorithm which is complex, time-consuming and requires visual sensors with the same parameters and high mounting accuracy. This is one of the difficulties in the field of machine vision. The latter, with simple construction, needs real-time visual feedback which results in large amount of computation and can't get fruits' depth. Under the circumstance of ridge strawberry culture, since the fruits lie on the ridge slope which approximates a plane, the depth of strawberry is almost unchanged during harvesting. Considering these, in this paper, we took the latter method to get the position parameters for harvesting with the eye-in-hand system. Because there was no obstacle between fruits and the end-effector, a Descarte robot which was simple and easy to be controlled but with weak obstacle-avoidance capability was developed. The robot consisted of a gantry walking mechanism, a machine vision based precision positioning mechanism and an end-effector. Firstly, the kinematic equations that describe the position relation between the world coordinate system and the camera coordinate system were deduced. Secondly, the circumscribed rectangle of fruits region's outline was used to describe target fruits region to reduce computation of vision analysis after the image was binarized. Thirdly, to design the visual servoing PID controller, the image Jacobian matrix which relates the end-effector's velocity in the task space to the change rate of feature parameter in the image space was deduced according to the pinhole camera projection principle and the harvesting process. The position of servo motor encoder was recorded with high speed when the motion controller was triggered by the exposure signal of camera to solve the problem that the position at which the camera capturing image inconsistencies with the image caused by the lag of visual feedback. Real-time position information from servo motor encoder was taken to compensate the low frequency of visual feedback. Fourthly, the equation of harvesting position parameters was deduced, from which the distance was obtained between the top of the target fruits' region and the end-effector and the width of the target fruits' region. With these parameters, the end-effector's lifting height and opening width was known. Fifthly, according to different centroids of target fruits' region in the image while the camera moved to different positions during the visual servoing process, the depth of target fruits' region was obtained on the basis of structure from motion. The test of fruit localization was respectively conducted when the angle between the ridge plan and the image plan was ±10, ±5, 0 degrees while the target fruits region contained 1, 2, 3 strawberries. Because the relation between harvesting position parameters and the depth of target fruits' region can be drawn that the accuracy of harvesting position parameters met the harvesting requirements when the error of the fruits depth was within a certain range, and only the depth error was obtained to evaluate the accuracy of harvesting position parameters in the test. The results showed that the angle, the number of strawberries of the target fruit region and their interaction had significant effects (P=0) on the localization time which was between 0.633 s and 0.886 s. The angle and the two elements' interaction had significant (P=0) effects on the relative error of the fruit depth which was in the range of -4.34% ~ 0.95% when the angle was invariant. The localization time increased with the number of strawberries of the target fruit region as the increase of the target fruit region centroid which increased with the number of fruits decreased the controlling speed. That the standard error of the relative error of the fruit depth increased with the number of strawberries of the target fruit region showed that a wider target fruit region led to a lower robustness of the relative error of the fruits depth.

       

    /

    返回文章
    返回