Restoring Reality: 3D Gaussian to Mesh 2.0
beforesandafters.com
Exactly a year ago, KIRI Engine released 3D Gaussian Splatting across all supported platforms, two months later its built-in editing tools, and another two months later 3DGS to Mesh. This brand new 3D scanning method has received lots of utilization and love as optimizations and tools dedicated to it are continously introduced, making this seemingly too-good-to-be-true 3D visualization technique more accessible and practical.Today, a year later, KIRI Engine is excited to announce two revolutionary updates in its 3.12 release: 3D Gaussian Splatting to Mesh 2.0 and KIRI Engine 3DGS Blender Addon V2.0. Along with other quality of life changes and improvements to the overall versatility of the app, this is an update you dont want to miss.3DGS (3D Gaussian Splatting) to Mesh 2.0In January 2024, KIRI released 3DGS to Mesh 1.0. This was made possible in collaboration with CJ Ye and his team, and it was the first attempt to make 3DGS more applicable in widely-used applications as the native PLY point cloud file was hardly accepted and visualized correctly in any rendering engine. This method worked by running a separate algorithm on top of the existing splat, reconstructing surface meshes from the depth map analyzed from the generated gaussian splat result. While this method worked to an extent, the light and depth data was often inaccurate and resulted largely inconsistent results, with featureful objects being converted into mesh files with much higher fidelity than their featureless counterparts.Figure 1: 3DGS to Mesh 1.0 Generated SuitcaseBuilding on top of the solid foundation provided by the lastest gaussian splatting mechanics, the newly updated gaussian splat-to-mesh conversion introduces sophisticated normal prediction and reflection removal techniques. Developed once again by the KIRI Engine team and CJ Ye, the new algorithm draws inspiration from professional 3D scanners, specifically the super-dense point cloud generation methodology. With this change, the algorithm is now able to handle scenarios that used to be impossible for gaussian splat-to-mesh reconstruction, which were scanning reflective and transparent surfaces as they would disrupt the reconstruction process by providing inaccurate depth data.This update has been uploaded on GitHub, along with model viewers that display the differences between 1.0 and 2.0. The model viewers are rigged to be in synch in terms of the camera position, making the models differences stand out clearly.Figure 2: 3DGS-to-Mesh 2.0 versus 1.0 on GitHubFigure 3: 3DGS-to-Mesh 2.0 (detected as reflection) versus 1.0 on GitHubMoreover, 3DGS-to-Mesh 2.0 yields significantly higher quality results and processes the files at a much more consistent level.Figure 4 & 5: 3DGS to Mesh 1.0 vs. 2.0In actual practice, the 3DGS-to-Mesh data will be processed and stored under the Scans list, as they will be high-quality 3D models with tons of editability within the app just like KIRIs Photo Scans and Featureless Object Scans.Needless to say, 3DGS-to-Mesh 2.0 is a state-of-the-art technology and marks a significant leap in raterizational-radiance fields derived geometry research. Additional great news: this technology will be fully open-sourced towards the end of this year or the beginning of 2025.3DGS MaskingIn version 3.12 of KIRI Engine, many changes to gaussian splatting are added directly to aid artists workflows, where features and usability improvements are introduced to the update.3DGS now has masking features, which isolates the object from the rest of the background in the model generation process. For those unfamiliar with this technique, auto-masking has been a tried-and-true method for KIRI Engines Photo Scan, where areas besides the object in focus are automatically cropped out.Figure 6: 3DGS to Mesh 2.0 Generated Suitcase (Masked)3DGS Masking will be available for both conventional 3D gaussian splats as well as the new 3DGS-to-Mesh production, where in both cases the user can obtain a clean model without noise coming from the background. This will be a toggle feature on the upload menu for both methods, where you can get the full scene along with its background when turned off, and the isolated object by itself with the feature turned on.Figure 7: 3DGS Masking Off vs. On3DGS Blender Addon V2.0A couple months ago, KIRI released its very own 3DGS Blender Addon and fully open sourced it. Since then, They have been closely monitoring the addon, gathering community feedback, and in turn, improving the existing features as well as implementing new ones.New FeaturesNew Modifiers Edit mode:Several editing modifiers are created and added to the 3DGS object for importing. These modifiers speed up workflow and performance significantly. The modifiers include:Camera CullingCrop BoxesDecimationRemove ArtifactsFigure 8: Crop Tool Demonstration in KIRIs 3DGS Blender Addon V2.0New Point Edit mode:This mode is for editing point clouds before importing the 3DGS object, which can be used to create an isolated object from a full scene.New Modify Animate mode:This mode allows the user to add several animation presets in a few clicks:Noise displacementsTo points transformationTo curves transformationPixelate transformationOptimizations / Bugfixes:Imported objects are now editableHQ Splat/Render now supports multiple objects, objects can be renamed, and the original file location does not need to be keptBlender will not freeze when using the addon in orthograpic viewLike the initial release of KIRI Engine 3DGS Blender Addon V1.0, the new release will be fully free to download and open-sourced for developers, and it continues to be the most usable addon for 3DGS rendering/editing in Blender.Learn more about 3DGS Render on GitHub here.Get 3DGS Render Addon on Blender Market here.Quality of Life UpdatesFeatureless Object ScanMesh quality improvementUIChanged the Include Mesh from the 3DGS upload page to 3DGS to MeshAfter selecting 3DGS to Mesh, there will be two files being generated:A gaussian splat under the 3DGS listA mesh file based on the splat under the Scans listA Few Words from the DevsThis is one of our proudest updates to date, especially in the realm of 3D gaussian splatting. As we continuously push for innovations ever since the initial release of this technology, we released many different features and tools for the reality-stealing capturing method to be more accessible, and we could not be happier with how far weve comeThere are loads of possibilities heading into the future with 3D gaussian splats, and the release of 3DGS to Mesh 2.0 and the Blender Addon V2.0 is the first step of many. We are infinitely excited and optimistic to see what wonders lie ahead of the technologys path, and we are so proud of everything our dev team has achieved. Zion, KIRI Engines PR ManagerThe release of KIRI Engine 3.12 accompanies the programs biggest sales event of the year, offering its premium subscription at the lowest price in the entire year. This is a wonderful opportunity to capitalize on the unlimited access to the state-of-the-art features they newly released. The subscription will be available on all supported platforms.Happy scanning!Check out KIRI Engines official release video for version 3.12 here.Download the KIRI Engine Blender Addon on GitHub.Visit KIRI Engines Official WebsiteDownload KIRI Engine from the Google Play StoreDownload KIRI Engine from the App StoreDont miss KIRI Engines Black Friday sale 55% off! Brought to you by KIRI Engine:This article is part of the befores & afters VFX Insight series. If youd like to promote your VFX/animation/CG tech or service, you can find out more about the VFX Insight series here.The post Restoring Reality: 3D Gaussian to Mesh 2.0 appeared first on befores & afters.
0 Comments ·0 Shares ·137 Views