Implementing SfM in OpenCV

OpenCV has an abundance of tools to implement a full-fledged SfM pipeline from first principles. However, such a task is very demanding and beyond the scope of this chapter. The former edition of this book presented just a small taste of what building such a system will entail, but luckily now we have at our disposal a tried and tested technique integrated right into OpenCV's API. Although the sfm module allows us to get away with simply providing a non-parametric function with a list of images to crunch and receive a fully reconstructed scene with a sparse point cloud and camera poses, we will not take that route. Instead, we will see in this section some useful methods that will allow us to have much more control over the reconstruction and exemplify some of the topics we discussed in the last section, as well as be more robust to noise.

This section will begin with the very basics of SfM: matching images using key points and feature descriptors. We will then advance to finding tracks, and multiple views of similar features through the image set, using a match graph. We proceed with 3D reconstruction, 3D visualization, and finally MVS with OpenMVS.