Monocular distance measurement algorithm for pomelo fruit based on target pixels change

Accurate identification of target depth is the critical premise for the manipulator of fruit and vegetable picking in intelligent agriculture. However, the general ranging of fruit has posed a great challenge on that the orchard in the mountain areas, due mainly to the light change, as well as the branch and leaf occlusion. In this study, a novel imaging algorithm was proposed to detect the monocular distance between the pomelo fruits and camera using target pixels change. The pomelo fruit off the tree in the orchard was chosen as the samples for data collection. Multiple regression was also established to verify the number change of pixels in the target areas and imaging distance. Furthermore, the pomelo fruit on the tree was involved to test the applicability in the samples. Additionally, a systematic investigation was made to explore the influences of initial imaging distance and step interval on prediction accuracy. The specific procedure was as follows. A single camera was utilized to capture the imaging data of the sample fruit side within the imaging distance ranging from 25 to 150 cm, where the common range of picking operation was set at the step interval of 2.5 cm. Therefore, there were 51 images side for each fruit for one group of data. 20 pomelo fruits with the representative shape and size were selected for the imaging data collection, including 14 off-tree and 6 on-tree. In the pomelo fruits off the tree, the surface was equally divided into four sides, where one group of data was acquired from each side. As such, a total of 56 data groups were collected from the samples of the tree. Subsequently, the 40 data groups were randomly selected to establish the multiple regressions between the imaging distance and the number of pixels change in the target area on the image, while the rest 16 data groups were used to optimize the algorithm. In the pomelos fruits on the tree, only the side towards the outside of the canopy was shot as one sample side, where 6 data groups were collected for testing. The numbers of pixels were then measured for the target areas in the image using Photoshop software. MATLAB 2018 platform was finally utilized to calculate the regression and curve fitting. The results showed that the error of predicted distance decreased gradually, as the camera approached the fruit target from 150 to 120 cm. In the fruit samples off the tree, the accuracy of distance prediction was at the medium level closer than 130 cm at the high level of about 120 cm. The relative ranging errors of 16 samples off-tree were less than 5% within the imaging distance of 120 cm, when 150 cm was as the initial distance, indicating that an excellent detection performance of imaging distance between the target and camera. In the fruit samples on the tree, the ranging accuracies were set as 125-137.5 cm and 25.0-125.0 cm for the medium and high levels, respectively. Correspondingly, the relative ranging errors of 6 samples on the tree were less than 5% within 125 cm imaging distance, fully meeting the accuracy requirements of target positioning for the picking manipulator. In addition, there was a significant effect of initial imaging distance on the measurement accuracy. Consequently, the measurement method of monocular distance can widely be expected to realize the rapid prediction of the distance between the fruit target and camera in the complex orchard environment, especially in the hilly mountain areas. Meanwhile, the finding can also provide a feasible scheme for the accurate recognition of fruit targets for picking manipulators in modern orchards.