Non-contact vibration detection of sub-healthy apple with moldy core using SHAP-interpretable depth features

Moldy core in apples is one of the most common internal diseases that is caused by fungi. The apples with mold are confined to the core by the endocarp wall, also called sub-healthy fruit. It is also the risk of storage deterioration. However, the identification of the sub-healthy apples is still challenging, due to the minor differences between healthy and slightly diseased apples. In this study, a non-contact vibration device was developed to acquire the vibration signals. The apple was impacted by the air jet under the optimal operating parameters, which were determined as follows: air pressure of 600 kPa, excitation distance of 55 mm, and excitation time of 2 ms. The laser Doppler vibrometer was employed for the sense signal. Furthermore, the signals of the vibration response were converted into the time-domain Gramian angular summation fields (GASF) images, frequency-domain GASF images, and S transform time-frequency images. These multi-domain images were combined and then input into the vision transformer (ViT) network, in order to extract the depth features. Subsequently, a cluster analysis was conducted on these features using the uniform manifold approximation and projection (UMAP). The clustering performance of the deep features was quantitatively evaluated using the silhouette coefficient, Calinski-Harabasz score, and Davies-Bouldin index. The depth features with the best clustering performance were fed into the k-nearest neighbor (kNN), support vector machine (SVM), logistic regression (LR), random forest (RF), and decision tree (DT) classifiers. A comparison was made of the performance of the classification. The better classification was combined with the Shapley additive explanations (SHAP) analysis to determine the attributes of the deep features, and then explain the contributions of the various features to classification and generalization. In the classification task of the sub-healthy apples, the high clustering performance was observed in the depth features of the class token from the eighth encoder block layer to the twelfth encoder block of the ViT network. The deep features from the eighth encoder block were selected for the subsequent analysis. As such, the highly discriminative deep features were extracted using fewer computational resources. The ViT-kNN model achieved relatively better classification with the overall accuracy, F1-score, Kappa coefficient, and Matthew’s correlation coefficient values of 83.10 %, 82.18 %, 74.14 % and 74.08 %, respectively, using the layer of the depth features. The 768 deep features were used by the ViT-kNN model. Among them, 60 features were determined as the influential features, including 35 positive features and 25 negative features. Then, two interpretable models, ViT-InfF-kNN (using 60 influential features) and ViT-PosF-kNN (using 35 positive features) were constructed to determine the contribution rate of the deep features. The 60 influential features greatly contributed to the classification and generalization of the improved model. The remaining features (the 768 features, removing 60 influential features) contributed little to the overall classification, and also interfered with the identification of the diseased fruits. Yet they still contained a small amount of useful information to distinguish the healthy fruit. In these 60 influential features, 35 positive features contained more information to identify the healthy fruits. Although the 25 negative features inhibited the classification, there was an important impact on the generalization of the model. The feature selection during model construction should be approached with caution. This study can provide a strong reference for the early internal disease detection of the pome (pears and apples) fruit. Also, there was a great contribution of the features to the classification and generalization of the model. Therefore, the vibration technology was integrated with the visible-near infrared spectroscopy in future work, in order to detect the sub-healthy apples with the moldy core.