Abstract: Mechanical vibration harvesting is one of the most effective means in the mechanized harvesting of fruit. Two types are mainly divided in the vibration harvesting machinery, including the shaking and comb brush type. In shaking machinery, the vibration excitation equipment is used to excite the trunk or branch, thereby forcing the fruit tree in response to the vibration, and finally the fruit moves in a certain form to produce the inertial force. As such, the fruit falls off, particularly when the inertial force of fruit is greater than the binding force of the fruit stalk. Nevertheless, the vibration transmission of branches varies in the different types of fruit trees, or the different shapes of crown structure in the same kind of fruit trees. In essence, the internal structure and inherent characteristics of fruit trees determine the dynamic characteristics. Correspondingly, the dynamic response of fruit trees depends mainly on the tree structure and inherent features. The natural frequency of fruit trees is determined by the structure and natural characteristics. The natural frequency of fruit trees is one of the most important parameters to design the vibration harvester of fruit trees. The natural frequency can commonly be obtained in the modal test. The traditional modal test is mostly artificial excitation, difficult to cause effective attenuation response for the fruit trees with complex structure, and the accuracy of frequency identification is limited by the accuracy of frequency spectrum test. In this study, a combination was proposed to integrate the data-driven stochastic subspace identification (SSI) and graph theory clustering stability diagram, in order to effectively identify the natural frequency of fruit trees. The data-driven SSI showed excellent noise immunity suitable for dense modal identification. Only the output response signal of fruit trees was used to identify the natural frequency of fruit trees. The actual response signal of the fruit tree structure was directly collected for parameter identification. The link of the input excitation signal was reduced significantly, particularly on the technical requirements and workload. In the process of noise reduction, an order determination of the system was processed, including the data-driven SSI, stabilization diagram generation, graph theory clustering, and the response signal of fruit trees under random or environmental excitation. As such, the natural frequency of fruit trees was effectively identified to minimize the human subjective factors. A field test was performed on a small indoor ginkgo tree and a large outdoor ginkgo tree. The natural frequency was also compared with the impact hammer frequency spectrum. The results showed that there was an excellent correspondence between the natural frequencies identified by data-driven SSI and the impact hammer frequency spectrum, where the relative error was small, the average error was 2.14%, and the maximum error was 4.17%. Furthermore, the average relative error between the recognition of outdoor large fruit trees under environmental excitation and the corresponding frequency spectrum was 2.88%, and the maximum relative error was 6.02%. In general, the relative errors were less than 5% in the most corresponding natural frequencies. Consequently, the data-driven SSI and graph theory clustering were feasible for the natural frequency identification of fruit trees using the output response signals. The stable graph with the distance threshold was utilized to reduce the influence of human factors, while improving the efficiency of natural frequency identification. The finding can provide a promising application in mechanical vibration harvesting, particularly where it is difficult to apply artificial force to fruit trees, or the effect of artificial force is not ideal in an outdoor environment.