Generating a color painting and a cartoon

A strong Bilateral filter smoothens flat regions while keeping edges sharp; and therefore, is great as an automatic cartoonifier or painting filter, except that it is extremely slow (that is, measured in seconds or even minutes, rather than milliseconds!). Therefore, we will use some tricks to obtain a nice cartoonifier, while still running in acceptable speed. The most important trick we can use is that we can perform Bilateral filtering at a lower resolution and it will still have a similar effect as a full resolution, but run much faster. Lets reduce the total number of pixels by four (for example, half width and half height):

    Size size = srcColor.size(); 
    Size smallSize; 
    smallSize.width = size.width/2; 
    smallSize.height = size.height/2; 
    Mat smallImg = Mat(smallSize, CV_8UC3); 
    resize(srcColor, smallImg, smallSize, 0,0, INTER_LINEAR);

Rather than applying a large Bilateral filter, we will apply many small Bilateral filters, to produce a strong cartoon effect in less time. We will truncate the filter (see the following figure) so that instead of performing a whole filter (for example, a filter size of 21x21, when the bell curve is 21 pixels wide), it just uses the minimum filter size needed for a convincing result (for example, with a filter size of just 9x9 even if the bell curve is 21 pixels wide). This truncated filter will apply the major part of the filter (gray area) without wasting time on the minor part of the filter (white area under the curve), so it will run several times faster:

Therefore, we have four parameters that control the Bilateral filter: color strength, positional strength, size, and repetition count. We need a temp Mat since the bilateralFilter()function can't overwrite its input (referred to as in-place processing), but we can apply one filter storing a temp Mat and another filter storing back the input:

    Mat tmp = Mat(smallSize, CV_8UC3); 
    int repetitions = 7;  // Repetitions for strong cartoon effect. 
    for (int i=0; i<repetitions; i++) { 
      int ksize = 9;     // Filter size. Has large effect on speed.  
      double sigmaColor = 9;    // Filter color strength. 
      double sigmaSpace = 7;    // Spatial strength. Affects speed. 
      bilateralFilter(smallImg, tmp, ksize, sigmaColor, sigmaSpace);    
      bilateralFilter(tmp, smallImg, ksize, sigmaColor, sigmaSpace); 
    }

Remember that this was applied to the shrunken image, so we need to expand the image back to the original size. Then we can overlay the edge mask that we found earlier. To overlay the edge mask sketch onto the Bilateral filter painting (left side of the following figure), we can start with a black background and copy the painting pixels that aren't edges in the sketch mask:

    Mat bigImg; 
    resize(smallImg, bigImg, size, 0,0, INTER_LINEAR); 
    dst.setTo(0); 
    bigImg.copyTo(dst, mask);

The result is a cartoon version of the original photo, as shown on the right side of the following figure, where the sketch mask is overlaid on the painting: