Abstract:
In the mechanical harvesting of fence-type grapes, the occlusion by vines and leaves makes it difficult to locate the peduncles of grapes, which affects the harvesting success rate. In order to solve this problem, this study proposed a novel leaf occlusion handling method for grape harvesting, where the act of removing obscuring vines and leaves was followed by the harvesting of the grape bunches. Based on collaborative mechanical arms, a fence-type grape leaf-removing and harvesting robot was developed to achieve efficient and low-damage automated harvesting of grapes in complex occlusion scenarios. The fence-type grape leaf-removing and harvesting robot included visual perception module, mechanical arm and end effector module, mobile chassis module, and data processing and storage module. Firstly, taking relative length, consistency of orientation and continuity as the key indicators, a quantitative discriminant model of peduncle visibility was constructed by calculating the visibility coefficient of grape peduncles. This model determined a visibility coefficient in the 0~1 interval, enabling objective assessment of the visibility degree of peduncles, including high peduncle visibility, medium peduncle visibility, and low peduncle visibility. For grape peduncles with medium and low visibility, this paper identified the optimal intervention point for removing occlusions by vines and leaves. A spatial quadrilateral was constructed using the endpoints of the grape peduncle and the occluding branches. The Limited-memory Broyden-Fletcher-Goldfarb-Shanno method was employed to solve for the Fermat-Torricelli point as the optimal intervention point. Then, the optimal intervention point was used to remove occlusions by vines and leaves by the mechanical arm. After the occlusion by vines and leaves was removed, the harvesting arm completed the harvesting operation. The timing deviation of the two arms not exceeded 0.3s. In order to simplify the inverse solution process of the mechanical arm, a nonlinear mapping model from the end Cartesian space to the joint space was constructed. This model could quickly solve the joint angle of the robotic arm corresponding to the end pose. The predicted average root mean square error of the first five joint angles was 7.848° in the harvesting arm and 7.128° in the intervention arm, and the coefficient of determination was not less than 0.98. The inference time does not exceed 0.002 06 s, which meets the needs of real-time control. This study achieved independent harvesting by the grape harvesting arm for grapes with highly peduncle visibility, as well as leaf-removing harvesting operations for grapes with medium and low peduncle visibility. Quantitative discrimination of peduncle visibility was performed on 100 groups of samples. The results showed a visibility discrimination accuracy of 91.0% and a Kappa coefficient of 0.9. Among these, the discrimination accuracy for high peduncle visibility was 94.1%. Field test results indicated that for grapes with highly peduncle visibility, the harvesting damage rate was below 10.0%, the harvesting success rate was 70.0%, and the average single-arm harvesting time was 3.2s per cluster. For grapes with medium and low peduncle visibility, the leaf occlusion handling method was adopted. Among these, for grapes with medium peduncle visibility, the harvesting damage rate was below 23.3%, the harvesting success rate was 53.3%, and the average leaf-removing harvesting time was 8.7s. For grapes with low peduncle visibility, the harvesting damage rate was below 36.6%, the harvesting success rate was 40.0%, and the average leaf-removing harvesting time was 14.8s per cluster. The proposed leaf occlusion handling method could effectively distinguish peduncles with different occlusion degrees and automatically switch operating modes, featuring a relatively high harvesting success rate. It meets the requirements for efficient, low-damage automated harvesting of grapes under complex occlusion conditions in fence-type vineyards, and provides a reference for the design and development of mechanized and automated harvesting machinery for fence-type grapes.