闫建伟, 赵源, 张乐伟, 苏小东, 刘红芸, 张富贵, 樊卫国, 何林. 改进Faster-RCNN自然环境下识别刺梨果实[J]. 农业工程学报, 2019, 35(18): 143-150. DOI: 10.11975/j.issn.1002-6819.2019.18.018
    引用本文: 闫建伟, 赵源, 张乐伟, 苏小东, 刘红芸, 张富贵, 樊卫国, 何林. 改进Faster-RCNN自然环境下识别刺梨果实[J]. 农业工程学报, 2019, 35(18): 143-150. DOI: 10.11975/j.issn.1002-6819.2019.18.018
    Yan Jianwei, Zhao Yuan, Zhang Lewei, Su Xiaodong, Liu Hongyun, Zhang Fugui, Fan Weiguo, He Lin. Recognition of Rosa roxbunghii in natural environment based on improved Faster RCNN[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(18): 143-150. DOI: 10.11975/j.issn.1002-6819.2019.18.018
    Citation: Yan Jianwei, Zhao Yuan, Zhang Lewei, Su Xiaodong, Liu Hongyun, Zhang Fugui, Fan Weiguo, He Lin. Recognition of Rosa roxbunghii in natural environment based on improved Faster RCNN[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(18): 143-150. DOI: 10.11975/j.issn.1002-6819.2019.18.018

    改进Faster-RCNN自然环境下识别刺梨果实

    Recognition of Rosa roxbunghii in natural environment based on improved Faster RCNN

    • 摘要: 为了实现自然环境下刺梨果实的快速准确识别,根据刺梨果实的特点,该文提出了一种基于改进的Faster RCNN刺梨果实识别方法。该文卷积神经网络采用双线性插值方法,选用Faster RCNN的交替优化训练方式(alternating optimization),将卷积神经网络中的感兴趣区域池化(ROI pooling)改进为感兴趣区域校准(ROI align)的区域特征聚集方式,使得检测结果中的目标矩形框更加精确。通过比较Faster RCNN框架下的VGG16、VGG_CNN_M1024以及ZF 3种网络模型训练的精度-召回率,最终选择VGG16网络模型,该网络模型对11类刺梨果实的识别精度分别为94.00%、90.85%、83.74%、98.55%、96.42%、98.43%、89.18%、90.61%、100.00%、88.47%和90.91%,平均识别精度为92.01%。通过对300幅自然环境下随机拍摄的未参与识别模型训练的刺梨果实图像进行检测,并选择以召回率、准确率以及F1值作为识别模型性能评价的3个指标。检测结果表明:改进算法训练出来的识别模型对刺梨果实的11种形态的召回率最低为81.40%,最高达96.93%;准确率最低为85.63%,最高达95.53%;F1值最低为87.50%,最高达94.99%。检测的平均速度能够达到0.2 s/幅。该文算法对自然条件下刺梨果实的识别具有较高的正确率和实时性。

       

      Abstract: Rosa roxburghii is widely distributed in warm temperate zone and subtropical zone, mainly in Guizhou, Yunnan, Sichuan and other places in China. Panxian and Longli are the most abundant the most varieties and the highest yield Rosa roxburghii resources in Guizhou. The harvesting of Rosa roxburghii fruit is the most time-consuming and labor-consuming work in Rosa roxburghii production, and its labor input accounts for 50%-70% of the production process. Hand-picking of Rosa roxburghii fruit is of high cost, high labor intensity and low picking efficiency. In recent years, convolutional neural network has been widely used in target recognition and detection. However, there is no relevant literature on the application of neural network in Rosa roxburghii fruit recognition. In this paper, in order to realize rapid and accurate identification of Rosa roxburghii fruits in natural environment, according to the characteristics of Rosa roxburghii fruits, the structure and parameters of VGG16, VGG_CNN_M1024 and ZF network models under the framework of Faster RCNN were optimized by comparing them. The convolutional neural network adopted bilinear interpolation method and selected alternating optimization training method of Faster RCNN. ROI Pooling in convolutional neural network is improved to ROI Align regional feature aggregation. Finally, VGG16 network model is selected to make the target rectangular box in the detection result more accurate. 6 540 (80%) of 8 175 samples were selected randomly as training validation set (trainval), the remaining 20% as test set, 80% as training set, the remaining 20% as validation set, and the remaining 300 samples that were not trained were used to test the final model. The recognition accuracy of the network model for 11 Rosa roxburghii fruits was 94.00%, 90.85%, 83.74%, 98.55%, 96.42%, 98.43%, 89.18%, 90.61%, 100.00%, 88.47% and 90.91%, respectively. The average recognition accuracy was 92.01%. The results showed that the recognition model trained by the improved algorithm had the lowest recall rate of 81.40%, the highest recall rate of 96.93%, the lowest accuracy rate of 85.63%, the highest 95.53%, and the lowest F1 value of 87.50%, the highest 94.99%. Faster RCNN (VGG16 network) has high recognition accuracy for Rosa roxburghii fruit, reaching 95.16%. The recognition speed of single fruit is faster, and the average recognition time of each Rosa roxburghii fruit is about 0.2 seconds. The average time has some advantages, which is 0.07 s faster than the methods of Fu Longsheng. In this paper, a Faster RCNN Rosa roxburghii fruit recognition network model based on improved VGG16 is proposed, which is suitable for Rosa roxburghii fruit recognition model training. The algorithm proposed in this paper has good recognition effect for Rosa roxburghii fruit under weak and strong illumination conditions, and is suitable for effective recognition and detection of Rosa roxburghii fruit in complex rural environment. This paper is the first study on the depth extraction of Rosa roxburghii fruit image features by using convolution neural network. This research has high recognition rate and good real-time performance under natural conditions, and can meet the requirements of automatic identification and positioning picking of Rosa roxburghii fruit. It lays a certain foundation for intelligent identification and picking of Rosa roxburghii fruit, and opens a new journey for the research of automatic picking technology of Rosa roxburghii fruit.

       

    /

    返回文章
    返回