Detecting wheat ears per unit area using an improved YOLOX

Wheat production is closely related to the food security in world. The yield forecast of wheat can provide a strong reference for the agricultural production and management, particularly for the decision-making on the rural land policy and grain market. Among them, the number of wheat ears per unit area is one of the most important indicators to estimate the wheat yield, including the crop phenotypic parameters, yield prediction, and field management. However, the traditional image processing and manual counting of wheat ears cannot fully meet the large-scale production in recent years. Particularly, the manual counting is cumbersome, labor-intensive, and highly subjective. It is a high demand to improve the detection accuracy of the traditional image processing. A generalized model is also required for a lot of experience, the robustness to lighting, and sufficient soil conditions in complex scenes. Much effort has been made to combine the deep learning for the detection and counting of the wheat ears per unit area, particularly with the rapid development of crop phenotype research. It is still lacking on the recognition accuracy of dense and occluded wheat ears under complex conditions. Taking the image of wheat ears per unit area as the research object, this study aims to accurately obtain the number of wheat ears per unit area using the improved YOLOX. Firstly, a simple sampling frame was designed to directly realize the counting of wheat ears per unit area. The corner detection network was trained to identify the sampling frame, further to extract the unit area of wheat. The Content-Aware ReAssembly of Features (CARAFE) map was used in the feature fusion layer of the wheat ear detection network. Secondly, the sampling was replaced with the up-sampling in the YOLOX-m model. The iterative attention feature fusion module was also used to increase the extraction of spatial information and semantic information of wheat ears. Thirdly, the wheat canopy images captured by the smartphone were taken as the research object. The images were selected at the wheat grain filling and mature stages under three weather conditions of clear, overcast, and cloudy. A total of 600 images of wheat ears without the sampling frame (image resolution of 4 000 × 3 000 pixels) were collected, where the original images were randomly cropped into the 3 072 images of wheat ears of 800 × 800 pixels. Fourthly, the dataset was augmented after the mirroring and rotation operation, where the image data of the training set was expanded from 3 072 to 9 216 images. There were 218 wheat ears images with the sampling frame (image resolution was 4 000 × 3 000 pixels). Among them, the sampling frame was contained 350-520 target wheat ears. Finally, the performance of the model was evaluated using the precision, recall, Average Precision (AP), F1 score, Frame per Second (FPS), determination coefficient (R2) and Root Mean Square Error (RMSE). The experimental results show that the improved YOLOX-m model was significantly improved the detection performance of dense and occluded wheat ears. Specifically, the AP value was improved by 10.26, 8.2 and 1.14 percentage points, respectively, compared with the SSD, CenterNet, and original YOLOX-m model. Consequently, the wheat ears per unit area were accurately detected and counted in the natural environment. The finding can provide a strong reference for the intelligent counting of wheat ears in the actual production of wheat yield prediction.