L1 and L2 normalization

L1 and L2 normalization are common regularization techniques that control how much the weights can grow or shrink in the network during training. It has the effect of not giving too much importance to a specific feature, similar to dropout. In L1 regularization, the loss function increases in direct proportion to the size of the weights, whereas in L2 normalization, it increases in proportion to the square of the weights.