Application of deep learning using YOLOv5s to apple flower detection in natural scenes

Flower thinning is one of the most important management measures in apple cultivation. Mechanical thinning has been the most promising way for thinning flowers in recent years. Accurate and rapid detection of flowers can be critical to the highly efficient operation of flower thinning robots. In this study, an apple flower detection was proposed using machine vision and YOLOv5s deep learning. 3 005 apple flower images were collected, including 1 611 apple images on sunny days, 512 on cloudy, 519 on overcast sky days, and 363 on light rainy days. Two lighting conditions were considered, where 1 830 apple images under the front lighting and 1 175 apple images under the backlight. Two occlusion situations were selected, where 1 602 apple images with occlusion, and 1 403 apple images without occlusion. The apple flower images were taken to annotate in the field, and then sent to the fine-tuned YOLOv5s target detection network for the detection of the apple flower. 300 iterations of training were implemented after the test. The better performance was achieved, where the precision of the model was 87.70%, the recall was 0.94, the mean average precision was 97.20%, the model size was 14.09 MB, and the detection speed was 60.17 f/s. Specifically, the recall increased by 7, 15, and 7 percental points, respectively, compared with the YOLOv4, SSD, and Faster-RCNN models, while the mAP increased by 8.15, 9.75, and 9.68 percental points, respectively, the model size decreased by 94.23%, 84.54%, and 86.97%, respectively, as well as the detection speed increased by 126.71%, 32.30%, and 311.28%, respectively. At the same time, the study detected apple flowers in different weather, colors and light conditions. The results showed that the precision values of the model to detect the white, pink, rose and red flowers were 84.70%, 91.70%, 89.40%, and 86.90%, respectively, while the recall were 0.93, 0.94, 0.93, and 0.93, respectively, as well as the mean average precision were 96.40%, 97.70%, 96.50%, and 97.90%, respectively. The precision values of the model for detecting apple blossoms under sunny, cloudy, overcast, and light rain were 86.20%, 87.00%, 87.90%, and 86.80%, respectively, the recall were 0.93, 0.94, 0.94, and 0.94, respectively, as well as the mean average accuracy were 97.50%, 97.30%, 96.80%, and 97.60%, respectively. The precision values of this model for detecting apple flowers under forwarding and backlight conditions were 88.20% and 86.40%, respectively, the recalls were 0.94 and 0.93, respectively, as well as the mean average accuracy was 97.40% and 97.10%, respectively. Consequently, the YOLOv5s can be expected to detect the apple flowers accurately and rapidly. The higher robustness and the smaller size were more conducive to the migration and application of the model. The finding can provide strong technical support to develop the flower thinning equipment.