基于改进HRNet的单幅图像苹果果树深度估计方法

    Depth estimation of apple tree in single image using improved HRNet

    • 摘要: 针对苹果自动采收获取深度信息的实际需求,以及目前单幅图像深度估计算法存在的空间分辨率低和边缘模糊问题,提出一种基于改进高分辨率网络(High-Resoultion Net, HRNet)的单幅图像苹果果树深度估计模型。首先基于HRNet构建多分支并行的编码器网络,提取多尺度特征,并通过引入密集连接机制强化特征传递过程中的连续性;为了减少冗余特征造成的噪声干扰,使用卷积注意力模块在通道及像素层级对融合特征进行重标定,强化特征图结构信息。在解码器网络中,使用条纹细化模块自适应地优化特征图的边界细节信息,突出边缘特征,改善边缘模糊问题,最后经上采样生成深度图。在NYU Depth V2公共数据集和果树深度数据集上进行试验。试验结果表明,引入密集连接机制,添加卷积注意力模块、条纹细化模块均能提升模型性能。提出的改进HRNet网络在果树深度数据集上的平均相对误差、均方根误差、对数平均误差、深度边缘准确误差和边缘完整性误差分别为0.123、0.547、0.051、3.90和10.59,在1.25、1.252、1.253阈值下的准确率分别达到了0.850、0.975、0.993;在主观视觉上,改进HRNet网络生成的深度图有清晰的边缘以及较多的纹理细节。该方法在客观指标和主观效果上均有良好的表现。

       

      Abstract: An accurate and rapid estimation of apple tree depth can be widely applied to the precise fruit positioning and robot autonomous harvesting in recent years. In this study, an improved High-Resolution Network (HRNet) was proposed to estimate the monocular depth of apple tree in the real scene. The actual requirements of the depth were obtained from a single RGB image for the apple mechanized picking. Firstly, a multi-branch parallel encoder network was constructed to extract the multi-scale features using the HRNet. A dense connection mechanism was introduced to enhance the continuity in the feature transfer process. Secondly, the Convolutional Block Attention Module (CBAM) was used to recalibrate the fused feature maps at the channel and pixel levels, in order to reduce the noise interference that caused by redundant features. Furthermore, the different weight distributions of the feature maps were effectively learned to enhance the structure information. In the decoder network, the Stripe Refinement Module (SRM) was used to gather the boundary pixels in the horizontal and vertical orthogonal directions. The boundary details of the feature map were adaptively optimized to highlight the edge features. As such, the blurry edge was reduced in the predicted images. Finally, the up-sampling was utilized to generate the prediction depth images of the same size as the RGB images. An image acquisition platform was constructed to collect the RGB and depth images of apple orchards at different times. The data was then enhanced using horizontal mirroring, color jitter, and random rotation. After data enhancement, the 3374 orchard RGB and depth images were obtained for the depth datasets. A series of experiments were also conducted on the NYU Depth V2 dataset and the orchard depth dataset. Ablation experiments were firstly performed on the HRNet networks with different degrees of improvement. The predictive performance of different improved networks was improved significantly, compared with the traditional HRNet network. It indicated that the dense connection mechanism, CBAM, and SRM were added to improve the model performance. Secondly, the mean relative error (MRE), root mean square error (RMS), logarithmic mean error, depth edge accuracy error, and edge integrity error of the improved HRNet network on the orchard depth dataset were 0.123, 0.547, 0.051, 3.90 and 10.59, respectively, compared with the current mainstream networks. The accuracy reached 0.850, 0.975 and 0.993 at different thresholds, respectively. More accurate spatial resolution was achieved in the depth map that generated by the improved HRNet network, in terms of subjective vision. The improved network can be expected to better present the depth information distribution of the image, particularly with the clear edges and more texture details. More importantly, the depth information of some small-sized objects was also displayed, indicating the best overall effect closer to the real depth map. The ablation analysis demonstrated the higher effectiveness of depth estimation using the improved network, compared with the subjective and objective ones. The experiment also verified that the proposed network was outperformed for both visual quality and objective measurement on the NYU Depth V2 and the orchard depth dataset. The finding can provide a new idea to obtain depth information in the apple automatic picking machine.

       

    /

    返回文章
    返回