Explore Structure from Motion with the SfM Module

Structure from motion (SfM) is the process of recovering both the positions of cameras looking at a scene, and the sparse geometry of the scene. The motion between the cameras imposes geometric constraints that can help us recover the structure of objects, hence why the process is called SfM. Since OpenCV v3.0+, a contributed ("contrib") module called sfm was added, which assists in performing end-to-end SfM processing from multiple images. In this chapter, we will learn how to use the SfM module to reconstruct a scene to a sparse point cloud, including camera poses. Later, we will also densify the point cloud, adding many more points to it to make it dense by using an open Multi-View Stereo (MVS) package called OpenMVS. SfM is used for high-quality three-dimensional scanning, visual odometry for autonomous navigation, aerial photo mapping, and many more applications, making it one of the most fundamental pursuits within computer vision. Computer vision engineers are expected to be familiar with the core concepts of SfM, and the topic is regularly taught in computer vision courses.

The following topics will be covered in this chapter:

• Core concepts of SfM: Multi-View Geometry (MVG), three-dimensional reconstruction, and Multi-View Stereo (MVS)
• Implementing a SfM pipeline using the OpenCV SfM modules
• Visualizing the reconstruction results
• Exporting the reconstruction to OpenMVG and densifying the sparse cloud into a full reconstruction