Apple recognition based on two-level segmentation and region-marked gradient Hough circle transform

Apples are produced in the large quantities each year, particularly for as the largest economic fruit in China. It is highly required for the rapid picking within the harvesting period. Therefore, the automatic apple picking is essential to the apple harvesting in intensive farming. An accurate and rapid identification of fruit can be fundamental for the automatic picking. However, some environmental factors surrounding the fruit can pose a great interference in the fruit identification under the natural, complex, and variable backgrounds, such as the light intensity, occlusion, and overlap of the fruit. In this study, an apple recognition was proposed using two-level segmentation and region-marked Hough transform. Experimental results show that the robust and practical performance was achieved for the apple recognition under different illumination, branch and leaf occlusion, as well as the fruit overlap. Specific steps were as follows. Firstly, the front camera (NikonD90) of the information acquisition robot was used to capture from 600-800 mm away from the fruit tree under the conditions of nature natural light and backlight, respectively. The brightness adaptive correction algorithm was then used to correct the brightness of apple images with the uneven distribution of surface brightness, in order to enhance the image details. The Cr component images of YCbCr color space were extracted as the preprocessing samples to combine with the feature of the color of the apple. Secondly, the improved Otsu algorithm was utilized to obtain the binary image of the apple target for the initial segmentation, in order to accurately extract the contour of the target fruit under different growth states (mainly including single and double fruits with the overlap and occlusion). A morphological open-close reconstruction filter was also introduced to the Otsu algorithm to remove the background noise. The traversal range of the gray level was reduced to shorten the complexity and running time of the algorithm for the high segmentation rate. Thirdly, the watershed algorithm was combined to perform the secondary segmentation of the segmented fruit region using distance transformation. The conglutinated and overlapping apples were separated to effectively extract the apple target contour. Finally, the gradient Hough circle transformation was selected to identify the number of apples. But the algorithm traveled through the whole image for the computational complexity, time time-consuming, and easy to produce the false identification. Therefore, the minimum circumscribed rectangle outside the contour was set as the effective area for the gradient Hough circle transform in the effective area, particularly for the recognition speed and accuracy. The experimental results show that: 1) The improved Otsu algorithm was achieved in the higher segmentation accuracy of fruit targets, especially with the less segmentation time. The improved algorithm was also filled the tiny holes in the apple to suppress the noise in the background branches and leaves, further to more clearly segment the target region more than before. The average segmentation time was 1.458 s, which was 0.643 and 1.060 s shorter than the Otsu and K-means algorithm before the improvement. In the adhesion of partial apple regions in the binary images, a watershed segmentation with the distance transformation was used for the quadratic segmentation to effectively separate the sticky apples for the full apple target boundary. 2) The gradient Hough circle transform was used to recognize the 200 apple images under different lighting conditions, where the recognition accuracy was 90.75% in the nature natural light and 89.79% in the backlight, which were improved by 15.03 and 16.41 percentage points, respectively, compared with the traditional. The average recognition time was 0.665 and 0.693 s, which was 0.664 and 0.643 s shorter than before. Therefore, the proposed algorithm can meet the requirements of apple-picking robots, in terms of recognition speed and accuracy. The findings can provide a strong reference for the fast recognition of spherical fruits, such as apples.