The Importance of Online Reconstruction from Dynamically-Posed Images
Traditionally, dense 3D reconstruction from RGB images has assumed static camera pose estimates. However, with the increasing focus on real-time methods for mobile devices, this assumption no longer holds true for online execution. In real-time SLAM (Simultaneous Localization And Mapping), camera poses are dynamic and can be updated based on events such as bundle adjustment and loop closure. While the problem of dynamic poses has been addressed in the RGB-D setting, it has largely been untreated in the RGB-only setting.
The New Task of Online Reconstruction
We have formalized the problem of online reconstruction from dynamically-posed images and introduced a dataset called LivePose. This dataset contains dynamic poses obtained from a SLAM system running on ScanNet. Our goal is to support further research in this area, as we believe responding to pose updates is crucial for high-quality reconstruction.
An Adaptation Framework and a Novel De-Integration Module
To tackle the challenge of dynamically-posed images, we have selected three recent reconstruction systems and applied a framework based on de-integration to adapt each one to this setting. Additionally, we have proposed a groundbreaking non-linear de-integration module. This module has the ability to learn and remove stale scene content, resulting in more accurate reconstructions.
Our research demonstrates that effectively responding to pose updates is essential for achieving high-quality reconstruction. Our de-integration framework, along with the novel non-linear de-integration module, provides a successful solution to the problem of online reconstruction from dynamically-posed images.