Classification metrics

In Keras, we can find the following classification metrics:

• Binary accuracy: This measures the accuracy of the result of a binary classification problem. In keras, it's possible to use the functions binary_accuracy and acc.
• ROC AUC: It measures the AUC of a binary classification problem. In keras, it's possible to use the functions categorical_accuracy and acc.
• Categorical accuracy: It measures the accuracy of the result of a multiclass classification problem. In keras, it's possible to use the categorical_accuracy and acc functions.
• Sparse categorical accuracy: It has the same functionality of categorical accuracy but for a sparse problem, sparse_categorical_accuracy.
• Top k categorical accuracy: It returns the accuracy of the top k elements. In keras, it's possible to use the top_k_categorical_accuracy functions (this requires you to specify a k parameter).
• Sparse top k categorical accuracy: It returns the accuracy of the top k elements. In keras, it's possible to use the sparse_top_k_categorical_accuracy function (requires you to specify a k parameter).

The first thing we need to determine is whether the dataset in is either balanced or unbalanced, regarding the classes we need to predict. If the dataset is unbalanced (for example, 99% if the instances belong to only one class), metrics such as precision and accuracy can be misleading. In that case, if our system always predicts the most common class, both precision and recall will look very good, but the system will be totally useless. That's why it's important to choose metrics that are useful for the system we are modeling. In this case, for example, ROC AUC will be better as it's looking at the misclassifications our algorithm does and how bad the error is.