谢为俊, 魏硕, 郑招辉, 杨光照, 丁鑫, 杨德勇. 深度多分支模型融合网络的胡萝卜缺陷识别与分割[J]. 农业工程学报, 2021, 37(2): 177-186. DOI: 10.11975/j.issn.1002-6819.2021.2.021
    引用本文: 谢为俊, 魏硕, 郑招辉, 杨光照, 丁鑫, 杨德勇. 深度多分支模型融合网络的胡萝卜缺陷识别与分割[J]. 农业工程学报, 2021, 37(2): 177-186. DOI: 10.11975/j.issn.1002-6819.2021.2.021
    Xie Weijun, Wei Shuo, Zheng Zhaohui, Yang Guangzhao, Ding Xin, Yang Deyong. Classification and segmentation of defect carrots using deep multi-branch models fusion network[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(2): 177-186. DOI: 10.11975/j.issn.1002-6819.2021.2.021
    Citation: Xie Weijun, Wei Shuo, Zheng Zhaohui, Yang Guangzhao, Ding Xin, Yang Deyong. Classification and segmentation of defect carrots using deep multi-branch models fusion network[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(2): 177-186. DOI: 10.11975/j.issn.1002-6819.2021.2.021

    深度多分支模型融合网络的胡萝卜缺陷识别与分割

    Classification and segmentation of defect carrots using deep multi-branch models fusion network

    • 摘要: 缺陷检测是胡萝卜上市销售前的重要环节,开裂缺陷区域的分割提取是开裂胡萝卜修整的必要条件。基于图像处理的传统的胡萝卜表面缺陷识别算法复杂,通用性、鲁棒性较差。该研究提出一种集胡萝卜缺陷种类识别(C-Net)和开裂缺陷分割(S-Net)为一体的深度多分支模型融合网络(CS-Net)。C-Net将预训练的ResNet-50作为胡萝卜图像特征提取器,分别输出ResNet-50不同卷积层特征,将其作为支持向量机(Support Vector Machine, SVM)的输入训练不同的分类模型,并利用不同策略将其融合以获取最终的分类模型;S-Net将预训练的ResNet-50作为分割网络的编码器,根据不同的分割网络构造思想设计解码器,构造胡萝卜开裂区域分割提取网络。结果表明,C-Net中,ResNet-50第49层输出模型在测试集上的准确率为94.71%,利用Stacking融合方法得到的模型性能最好,在测试集上的准确率为98.40%;S-Net中,根据U-net构造思想构造的网络分割效果最好,分割像素准确率(Pixel Accuracy,PA)、类别平均像素准确率(Mean Pixel Accuracy, MPA)和平均交互比(Mean Intersection over Union, MIoU)分别为98.31%、96.05%和92.81%。该研究构建的胡萝卜缺陷识别分割网络对胡萝卜外观品质的量化评价和表面缺陷的修整具有重要意义。

       

      Abstract: Abstract: Detection of carrot defects plays an important role in the sale of carrots. The segmentation and extraction of carrot crack regions have become necessary to automatically evaluate the crack degree of carrots, and further trim the area of the crack. In the traditional detection of carrot external quality, different image processing was designed using the features of different defects, showing high complexity while low robustness. In this study, a deep multi-branch models fusion network (CS-net) was proposed to integrate the recognition of carrot defects and segmentation of crack regions. The network contained two parts: the classification of carrot defects (C-Net), and segmentation extraction of carrot crack regions (S-Net). In C-Net, the ResNet-50 pre-trained on the ImageNet dataset was taken as an image feature extractor of carrot. The output features in the 1st, 10th, 22nd, 40th and 49th layers of ResNet-50 were processed by different pooling methods, including Average Pooling (AVP), Global Average Pooling (GAP), and Spatial Pyramid Pooling (SPP), as well as dimension reduction (principal component analysis, ReliefF). The extracted features were then used as input of Support Vector Machines (SVM) to obtain five classification models. Besides, the five classification models were ensemble with different fusion strategies (hard voting, soft voting and stacking) to obtain the final classification model. In S-Net, the pre-trained ResNet-50 was served as the encoder of segmentation network, and then the network decoder was designed to build the segmentation network of carrot crack regions. The results showed that the output features in the 49th layer of the ResNet-50 with SVM model performed best with the test accuracy of 94.71% among the single model. The fusion model with the stacking ensemble performed best with the accuracy of 98.40%, indicating a better performance in the fusion model than the single model. Different pooling methods had different effects on the performance of the model. In the low-level feature maps, the order of performance for different pooling methods was SPP > AVP > GAP. However, the pooling methods had little impact on the model performance with the high-level semantic features. It was found that dimensionality reduction reduced the number of features and then improved the performance of the model. In the segmentation part, the constructed segmentation network with the U-net construction ideas (Res-U-net) performed best with the Pixel Accuracy (PA), Mean Pixel Accuracy (MPA) and mean intersection over union (MIoU) of 98.31%, 96.05% and 92.81%, respectively. The performance of Res-U-net was not affected by the cracking area and different positions of crack. Comparing with Deeplabv3+, the PA and the MIoU in the Res-U-net were similar to those of Deeplabv3+, while the MPA was better than that of Deeplabv3+, and the model size was only half of that of Deeplabv3+. In addition, the segmentation speed of single image was faster than that of Deeplabv3+. The Res-U-net reached an advanced level in the segmentation task of carrot crack defects. The defect recognition and segmentation network have a positive significance on the quantitative evaluation of carrot external quality and the automatic trim of carrot crack.

       

    /

    返回文章
    返回