基于3D Gaussian Splatting的小型苗木三维重建

    Three-dimensional reconstruction of small tree seedlings based on 3DGS

    • 摘要: 针对现有苗木三维重建方法存在的图像采集成本高、效率低等问题,该研究提出了一种基于3D高斯溅射(3D gaussian splatting,3DGS)的小型苗木三维重建方法。首先,搭建了一套适用于小型苗木的图像采集装置,通过消费级智能手机获取苗木的多视图图像序列;其次,采用SAM模型(segment anything model)对苗木图像进行精确分割,去除多视图序列中的背景;将使用运动恢复结构(structure from motion,SfM)算法生成的稀疏点云初始化为三维高斯分布,进行场景优化;最后,利用三维高斯作为场景表示,通过3DGS实现苗木重建。该方法在自建数据集上的评估结果表明,相比使用原始数据,使用SAM分割后的数据进行重建的时间缩短了40.79%;相比神经辐射场(neural radiance fields,NeRF)模型,3DGS模型的峰值信噪比(peak signal-to-noise ratio,PSNR)和结构相似性指数(structural similarity index measure,SSIM)均值分别提高了70.49%和48.64%,学习感知图像块相似度(learned perceptual image patch similarity,LPIPS)均值降低了93.55%,平均重建时间缩短了98.04%。利用该文方法生成的苗木重建图像视觉保真度高,重建的点云能够有效表征苗木的几何结构,可以为小型苗木的低成本和高质量三维重建提供技术参考。

       

      Abstract: 3D(three-dimensional) models can be expected for plant phenotyping, breeding, and yield prediction in recent years. It is often required to rapidly and accurately reconstruct the 3D plant models using computer vision and image processing. However, conventional 3D reconstruction can still suffer from the high cost, cumbersome procedure, and less robustness in practical applications, due to the complex variety of the plant species and structure. In this study, a full 3D reconstruction was proposed for the small seedlings using 3D Gaussian splatting (3DGS). Firstly, an image acquisition device was constructed for the small tree seedlings, including a consumer-grade smartphone and an electric turntable. The multi-view images of the seedlings were then captured in a cost-saving and efficient manner. Secondly, the segment anything model (SAM) was applied for image segmentation. The point and box were also prompted to guide the precise segmentation of the tree seedling images. Better performance was achieved to effectively solve the time-consuming labeling in traditional segmentation. The background interference was also removed from the multi-view sequence. Thirdly, the sparse point clouds were generated by the structure from motion (SFM) algorithm, and then initialized as a set of 3D Gaussian distributions for the scene optimization. Finally, 3D Gaussians were utilized as the scene representation. A differentiable rendering was employed for the high-fidelity reconstruction of the tree seedling scene. A dataset was also constructed for the 3D reconstruction of the tree seedlings using an image acquisition device, in order to evaluate the performance of the 3DGS model. The results showed that the time for the 3D reconstruction was reduced by 40.79% using the SAM segmented data, compared with the original data; Compared with the neural radiance fields (NeRF) model, the average peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) of the 3DGS model increased by 70.49% and 48.64%, respectively, whereas, the mean learned perceptual image patch similarity (LPIPS) was reduced by 93.55%, and the average reconstruction time was shortened by 98.04%. The uniform distribution and structural integrity were achieved in the point cloud of the tree seedling 3D model generated by 3DGS, compared with the conventional Colmap 3D reconstruction. The images of the tree seedlings shared high visual fidelity after reconstruction. The point cloud is effectively characterized by the geometrical structure of the seedlings, indicating better performance in handling plants with complex growth structures. The finding can provide a better solution to the low-cost, high-quality 3D reconstruction of the tree seedlings, compared with the conventional 3D reconstruction. The dataset can be expanded to add more tree seedling species in the future. Different growth stages and environmental conditions can be covered to improve the generalization and robustness. Thus, the 3D reconstruction challenges can be solved across the various plant species and environments.

       

    /

    返回文章
    返回