Liang Xifeng, Zhang Xinyu, Wang Yongwei. Recognition method for the pruning points of tomato lateral branches using improved Mask R-CNN[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2022, 38(23): 112-121. DOI: 10.11975/j.issn.1002-6819.2022.23.012
    Citation: Liang Xifeng, Zhang Xinyu, Wang Yongwei. Recognition method for the pruning points of tomato lateral branches using improved Mask R-CNN[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2022, 38(23): 112-121. DOI: 10.11975/j.issn.1002-6819.2022.23.012

    Recognition method for the pruning points of tomato lateral branches using improved Mask R-CNN

    • Branch and leaf pruning has been one of the most important links in the process of tomato planting for reducing the disease rate and increasing economic benefits. However, the manual pruning of tomato branches and leaves cannot fully meet large-scale production in recent years, due to the time-consuming and labor-intensive task. An accurate and rapid identification of the pruning position can be greatly contributed to the automatic operation of tomato branch and leaf pruning. In this study, a Recognition method was proposed for the pruning point of the tomato lateral branch using an improved Mask R-CNN. Firstly, the backbone network of ResNet50 in the original Mask R-CNN was replaced with the MobileNetv3-Large to reduce the model complexity. Efficient Channel Attention was added to the feature map C3 and C4, in order to focus more on the features of the lateral and main branch rather than other features. Then, the tomato lateral and main branches were predicted using the improved Mask R-CNN. Three steps were selected to avoid some single branches taken as multiple masks. The lateral and main branch masks were first distinguished by the aspect ratio of the bounding boxes. The overlap and pole constraints were then analyzed for the adjacent masks that belonged to the same branch. The masks with similar constraints were finally merged and joined in the images. The pruning point of the lateral branch was only positioned at one of the two ends of the lateral branch. The lateral pruning point identification was proposed with the help of the main branch, in order to determine the coordinate of the lateral pruning point. The range near the main branch was first determined. And then the branch pruning end was determined by estimating which one of the lateral branch left and right endpoints was in the range. The center point close to the endpoint of the pruning end was finally determined as the pruning point of the lateral branch. The original and improved Mask R-CNN were also compared to verify the detection performance of the lateral and main branches. The recall rate and precision of the original Mask R-CNN were 87.9% and 93.3%, respectively, whereas, the recall rate and precision of the improved Mask R-CNN were 91.2% and 88.6%, respectively. The number of backbone network parameters in the improved Mask R-CNN was only 21.1% of that in the original one. The average segmentation time of the improved Mask R-CNN decreased by 0.038 s than before. The results showed that the backbone network of MobileNetv3-Large reduced the model parameters with the high speed in the improved Mask R-CNN. More branches were recognized, particularly when adding the Efficient Channel Attention mechanism into the feature map C3 and C4. Lateral and main branches that were divided into multiple masks were selected randomly to verify the performance of merging masks. The merging success rate of lateral branch masks was lower than that of the main branch masks, due to the more outstanding curved shape of the lateral branch. The average success rate of merging masks was 86.2%, indicating the excellent performance of merging masks. The presence of multiple pruning points was effectively reduced, where the single branch was normally taken as the multiple masks. Some images were selected randomly in the test set to verify the recognition accuracy for the pruning point of the lateral branch. The result showed that the recognition success rate on sunny days was higher than that on cloudy. The average recognition success rate was 82.9%, which fully met the requirements of lateral branch pruning. This finding can provide the technical support for the tomato branch and leaf pruning automatically.
    • loading

    Catalog

      /

      DownLoad:  Full-Size Img  PowerPoint
      Return
      Return