Finding the convexity defects of a convex hull

As is evident from the preceding screenshot, not all points on the convex hull belong to the segmented hand region. In fact, all the fingers and the wrist cause severe convexity defects—that is, points of the contour that are far away from the hull.

We can find these defects by looking at both the largest contour (max_contour) and the corresponding convex hull (hull), as follows:

defects = cv2.convexityDefects(max_contour, hull) 

The output of this function (defects) is a NumPy array containing all defects. Each defect is an array of four integers that are start_index (index of the point in the contour where the defect begins), end_index (index of the point in the contour where the defect ends), farthest_pt_index (the index of the farthest point from the convex hull within the defect), and fixpt_depth (the distance between the farthest point and the convex hull).

We will make use of this information in just a moment when we try to estimate the number of extended fingers.

For now, though, our job is done. The extracted contour (max_contour) and convexity defects (defects) can be returned to recognize, where they will be used as inputs to detect_num_fingers, as follows:

return max_contour, defects 

So, now that we have found the defects, let's move on and learn how to perform hand gesture recognition using the convexity defects, which will bring us toward the completion of the app.