基于视触觉传感的草莓夹持法向力与接触区域感知方法

    Perceiving strawberry contact information using visual-tactile sensing

    • 摘要: 为解决草莓自动化夹持中的损伤问题,该研究提出一种基于视触觉融合的接触力精确感知方法。设计了集成视触觉传感器的夹爪,并构建了AFE-Unet3+(area and force estimation unet3+)神经网络。该网络基于Unet3+网络,融合高效局部注意力机制(efficient local attention,ELA),构建多尺度结构相似性损失函数(multi-scale structural similarity index,MS-SSIM),实现了对接触区域和力值的同步估计。AFE-Unet3+与Unet3+及CANFnet相比,AFE-Unet3+模型在接触区域预测的交并比(intersection over union, IoU)上分别提升了0.01和0.06;在法向力预测的平均绝对误差(mean absolute error, MAE)上分别降低了0.057和0.63,且力值回归的平均校正决定系数(adjusted coefficient of determination, R2)高达0.976。对3个品种草莓的损伤阈值试验测得,其安全夹持力范围为0.3~0.6 N,试验结果表明所设计的视触觉夹爪可为草莓无损抓取提供有效的力信息支持。

       

      Abstract: Strawberries are highly susceptible to damage during harvesting, due to their exposure and soft pseudocarp structure. Mechanical damage and quality degradation can cause during strawberry grasping. Flexible grippers and tactile feedback are often required for the picking of strawberries. This study aims to optimize the contact-based grasping using visuotactile sensing technology, with a specific focus on the combination of visuotactile sensing and strawberry grasping. A series of field tests were proposed to extract the strawberry contact information using visuotactile sensing. Firstly, a visuotactile sensing finger was developed based on the Gelsight principle. Its core component, a silicone elastomer, comprised four layers: a transparent layer, a marker array, a reflective film, and an object contact layer. This sensing finger served to acquire the tactile perception information.Two fingers were mounted on a servo-driven rack-and-pinion end-effector mechanism, enabling parallel and opposing movement to form a functional visuotactile sensing gripper. A multi-scale structural similarity index (MS-SSIM) loss function was constructed using the Unet3+ network and incorporating the efficient local attention (ELA) mechanism. The Area and Force Estimation Unet3+ (AFE-Unet3+) neural network model was developed to effectively estimate normal force distribution images and force tracking curves. The results show that the AFE-Unet3+ model was 0.06 improvement in the intersection over union (IoU) and a reduction of 0.63 in mean absolute error (MAE), compared with the CANFnet model. Additionally, the ELA-B (base) attention mechanism demonstrated the superior performance, compared with the squeeze-and-excitation block (SE), convolutional block attention module (CBAM), coordinate attention (CA), and ELA-T (tiny), with the IoU improvements of 0.004, 0.001, 0, and 0.005, respectively, and MAE reductions of 0.065, 0.049, 0.043, and 0.027, respectively. An experimental platform, grasping and data collection were performed at different contact positions on strawberries. Comparative analysis was made between the model-predicted force and the measured values from a pressure testing machine. Four trials show that the AFE-Unet3+ was achieved an Adjusted R-Square (R2) of 0.976 between the predicted force curves and the ground truth, indicating the improvements of 0.235, 0.414, 0.243, and 0.278, compared with the SE, CBAM, CA, and ELA-T attention mechanisms, respectively. Force threshold experiments were conducted on three strawberry varieties—Dandong, Sichuan Daliangshan, and Guangzhou Huadu strawberries. The samples also included Dandong strawberries at three ripening stages. The greater gripping forces were found in the Dandong strawberries that characterized by their large, plump shape and firm skin. While Guangzhou Huadu strawberries with their smaller size and tiny achenes, and Sichuan Daliangshan strawberries, with the intermediate characteristics, were more prone to damage. The experiments confirmed that the contact information perception model exhibited the significant advantages in the accuracy and efficiency of force distribution estimation and force tracking. Additionally, the force threshold tests indicated that the stable and damage-free grasping were achieved, when the applied force was ranged between 0.3 and 0.6 N. In conclusion, the visuotactile sensing-based end-effector gripper can fully meet the structural and functional requirements for the information extraction of strawberry contact. The finding can also provide the theoretical support to design and control the strawberry grippers during harvesting.

       

    /

    返回文章
    返回