Abstract:
3D(three-dimensional) models can be expected for plant phenotyping, breeding, and yield prediction in recent years. It is often required to rapidly and accurately reconstruct the 3D plant models using computer vision and image processing. However, conventional 3D reconstruction can still suffer from the high cost, cumbersome procedure, and less robustness in practical applications, due to the complex variety of the plant species and structure. In this study, a full 3D reconstruction was proposed for the small seedlings using 3D Gaussian splatting (3DGS). Firstly, an image acquisition device was constructed for the small tree seedlings, including a consumer-grade smartphone and an electric turntable. The multi-view images of the seedlings were then captured in a cost-saving and efficient manner. Secondly, the segment anything model (SAM) was applied for image segmentation. The point and box were also prompted to guide the precise segmentation of the tree seedling images. Better performance was achieved to effectively solve the time-consuming labeling in traditional segmentation. The background interference was also removed from the multi-view sequence. Thirdly, the sparse point clouds were generated by the structure from motion (SFM) algorithm, and then initialized as a set of 3D Gaussian distributions for the scene optimization. Finally, 3D Gaussians were utilized as the scene representation. A differentiable rendering was employed for the high-fidelity reconstruction of the tree seedling scene. A dataset was also constructed for the 3D reconstruction of the tree seedlings using an image acquisition device, in order to evaluate the performance of the 3DGS model. The results showed that the time for the 3D reconstruction was reduced by 40.79% using the SAM segmented data, compared with the original data; Compared with the neural radiance fields (NeRF) model, the average peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) of the 3DGS model increased by 70.49% and 48.64%, respectively, whereas, the mean learned perceptual image patch similarity (LPIPS) was reduced by 93.55%, and the average reconstruction time was shortened by 98.04%. The uniform distribution and structural integrity were achieved in the point cloud of the tree seedling 3D model generated by 3DGS, compared with the conventional Colmap 3D reconstruction. The images of the tree seedlings shared high visual fidelity after reconstruction. The point cloud is effectively characterized by the geometrical structure of the seedlings, indicating better performance in handling plants with complex growth structures. The finding can provide a better solution to the low-cost, high-quality 3D reconstruction of the tree seedlings, compared with the conventional 3D reconstruction. The dataset can be expanded to add more tree seedling species in the future. Different growth stages and environmental conditions can be covered to improve the generalization and robustness. Thus, the 3D reconstruction challenges can be solved across the various plant species and environments.