# Exploring Symmetry of Binary Classification Performance Metrics

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Definitions

- ${a}_{i}^{k}$ is the number of positive elements in ${D}_{i}^{k}$ correctly classified as positive;
- ${b}_{i}^{k}$ is the number of negative elements in ${D}_{i}^{k}$ correctly classified as negative;
- ${f}_{i}^{k}$ is the number of positive elements in ${D}_{i}^{k}$ incorrectly classified as negative; and
- ${g}_{i}^{k}$ is the number of negative elements in ${D}_{i}^{k}$ incorrectly classified as positive.

#### 2.2. Representation of Metrics

#### 2.3. Transformations

#### 2.3.1. One-Dimensional Transformations

#### 2.3.2. Multidimensional Transformations

#### 2.3.3. Combined Transformations.

#### 2.4. Performance Metrics

#### 2.5. Exploring Symmetries

#### 2.6. Statistical Symmetries

## 3. Results

#### 3.1. Identifying Symmetries

#### 3.2. Identifying Cross-Symmetries

#### 3.3. Skewness of the Statistical Descriptions of the Metrics

## 4. Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Speiser, A. Symmetry in science and art. Daedalus
**1960**, 89, 191–198. [Google Scholar] - Wigner, E. The Unreasonable Effectiveness of Mathematics. In Natural Sciences–Communications in Pure and Applied Mathematics; Interscience Publishers Inc.: New York, NY, USA, 1960; Volume 13, p. 1. [Google Scholar]
- Islami, A. A match not made in heaven: On the applicability of mathematics in physics. Synthese
**2017**, 194, 4839–4861. [Google Scholar] [CrossRef] - Siegrist, J. Symmetry in social exchange and health. Eur. Rev.
**2005**, 13, 145–155. [Google Scholar] [CrossRef] [Green Version] - Varadarajan, V.S. Symmetry in mathematics. Comput. Math. Appl.
**1992**, 24, 37–44. [Google Scholar] [CrossRef] [Green Version] - Garrido, A. Symmetry and Asymmetry Level Measures. Symmetry
**2010**, 2, 707–721. [Google Scholar] [CrossRef] [Green Version] - Xiao, Y.H.; Wu, W.T.; Wang, H.; Xiong, M.; Wang, W. Symmetry-based structure entropy of complex networks. Phys. A Stat. Mech. Appl.
**2008**, 387, 2611–2619. [Google Scholar] [CrossRef] [Green Version] - Magee, J.J.; Betke, M.; Gips, J.; Scott, M.R.; Waber, B.N. A human–computer interface using symmetry between eyes to detect gaze direction. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum.
**2008**, 38, 1248–1261. [Google Scholar] [CrossRef] - Liu, Y.; Hel-Or, H.; Kaplan, C.S.; Van Gool, L. Computational symmetry in computer vision and computer graphics. Found. Trends Comput. Gr. Vis.
**2010**, 5, 1–195. [Google Scholar] [CrossRef] - Tai, W.L.; Chang, Y.F. Separable Reversible Data Hiding in Encrypted Signals with Public Key Cryptography. Symmetry
**2018**, 10, 23. [Google Scholar] [CrossRef] - Graham, J.H.; Whitesell, M.J.; II, M.F.; Hel-Or, H.; Nevo, E.; Raz, S. Fluctuating asymmetry of plant leaves: Batch processing with LAMINA and continuous symmetry measures. Symmetry
**2015**, 7, 255–268. [Google Scholar] [CrossRef] - Bishop, C.M. Pattern Recognition and Machine Learning (Information Science and Statistics); Springer: New York, NY, USA, 2006. [Google Scholar]
- Top 10 Technology Trends for 2018: IEEE Computer Society Predicts the Future of Tech. Available online: https://www.computer.org/web/pressroom/top-technology-trends-2018 (accessed on 18 October 2018).
- Brachmann, A.; Redies, C. Using convolutional neural network filters to measure left-right mirror symmetry in images. Symmetry
**2016**, 8, 144. [Google Scholar] [CrossRef] - Zhang, P.; Shen, H.; Zhai, H. Machine learning topological invariants with neural networks. Phys. Rev. Lett.
**2018**, 120, 066401. [Google Scholar] [CrossRef] [PubMed] - Luque, A.; Gómez-Bellido, J.; Carrasco, A.; Barbancho, J. Optimal Representation of Anuran Call Spectrum in Environmental Monitoring Systems Using Wireless Sensor Networks. Sensors
**2018**, 18, 1803. [Google Scholar] [CrossRef] - Romero, J.; Luque, A.; Carrasco, A. Anuran sound classification using MPEG-7 frame descriptors. In Proceedings of the XVII Conferencia de la Asociación Española para la Inteligencia Artificial (CAEPIA), Granada, Spain, 23–26 October 2016. [Google Scholar]
- Luque, A.; Romero-Lemos, J.; Carrasco, A.; Barbancho, J. Non-sequential automatic classification of anuran sounds for the estimation of climate-change indicators. Exp. Syst. Appl.
**2018**, 95, 248–260. [Google Scholar] [CrossRef] - Glowacz, A. Fault diagnosis of single-phase induction motor based on acoustic signals. Mech. Syst. Signal Process.
**2019**, 117, 65–80. [Google Scholar] [CrossRef] - Glowacz, A. Acoustic-Based Fault Diagnosis of Commutator Motor. Electronics
**2018**, 7, 299. [Google Scholar] [CrossRef] - Caruana, R.; Niculescu-Mizil, A. Data mining in metric space: An empirical analysis of supervised learning performance criteria. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, USA, 22–25 August 2004. [Google Scholar]
- Ferri, C.; Hernández-Orallo, J.; Modroiu, R. An experimental comparison of performance measures for classification. Pattern Recognit. Lett.
**2009**, 30, 27–38. [Google Scholar] [CrossRef] [Green Version] - Hossin, M.; Sulaiman, M.N. A review on evaluation metrics for data classification evaluations. Int. J. Data Min. Knowl. Manag. Process
**2015**, 5, 1. [Google Scholar] - Ting, K.M. Confusion matrix. In Encyclopedia of Machine Learning and Data Mining; Springer: Boston, MA, USA, 2017; p. 260. [Google Scholar]
- Aly, M. Survey on multiclass classification methods. Neural Netw.
**2005**, 19, 1–9. [Google Scholar] - Tsai, M.F.; Yu, S.S. Distance metric based oversampling method for bioinformatics and performance evaluation. J. Med. Syst.
**2016**, 40, 159. [Google Scholar] [CrossRef] - García, V.; Mollineda, R.A.; Sánchez, J.S. Index of balanced accuracy: A performance measure for skewed class distributions. In Iberian Conference on Pattern Recognition and Image Analysis; Springer: Berlin/Heidelberg, Germany, 2009; pp. 441–448. [Google Scholar]
- López, V.; Fernández, A.; García, S.; Palade, V.; Herrera, F. An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics. Inf. Sci.
**2013**, 250, 113–141. [Google Scholar] [CrossRef] - Daskalaki, S.; Kopanas, I.; Avouris, N. Evaluation of classifiers for an uneven class distribution problem. Appl. Artif. Intell.
**2006**, 20, 381–417. [Google Scholar] [CrossRef] - Amin, A.; Anwar, S.; Adnan, A.; Nawaz, M.; Howard, N.; Qadir, J.; Hussain, A. Comparing oversampling techniques to handle the class imbalance problem: A customer churn prediction case study. IEEE Access
**2016**, 4, 7940–7957. [Google Scholar] [CrossRef] - Jeni, L.A.; Cohn, J.F.; De La Torre, F. Facing imbalanced data--recommendations for the use of performance metrics. In Proceedings of the 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, Geneva, Switzerland, 2–5 September 2013. [Google Scholar]
- Powers, D.M. Evaluation: From Precision, Recall and F-measure to ROC, Informedness, Markedness and Correlation; Technical Report SIE-07-001; School of Informatics and Engineering, Flinders University: Adelaide, Australia, 2011. [Google Scholar]
- Sokolova, M.; Lapalme, G. A systematic analysis of performance measures for classification tasks. Inf. Process. Manag.
**2009**, 45, 427–437. [Google Scholar] [CrossRef] - Matthews, B.W. Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochim. Biophys. Acta Protein Struct.
**1975**, 405, 442–451. [Google Scholar] [CrossRef] - Jurman, G.; Riccadonna, S.; Furlanello, C. A comparison of MCC and CEN error measures in multi-class prediction. PLoS ONE
**2012**, 7, e41882. [Google Scholar] [CrossRef] [PubMed] - Gorodkin, J. Comparing two K-category assignments by a K-category correlation coefficient. Comput. Biol. Chem.
**2004**, 28, 367–374. [Google Scholar] [CrossRef] [PubMed] - Flach, P.A. The geometry of ROC space: Understanding machine learning metrics through ROC isometrics. In Proceedings of the 20th International Conference on Machine Learning (ICML-03), Washington, DC, USA, 21–24 August 2003. [Google Scholar]
- Sokolova, M.; Japkowicz, N.; Szpakowicz, S. Beyond accuracy, F-score and ROC: A family of discriminant measures for performance evaluation. In Australasian Joint Conference on Artificial Intelligence; Springer: Berlin/Heidelberg, Germany, 2006; pp. 1015–1021. [Google Scholar]
- Bishop, C.M. Pattern Recognition and Machine Learning; Springer: Berlin/Heidelberg, Germany, 2006. [Google Scholar]
- Chakraborty, S.; Das, S. k—Means clustering with a new divergence-based distance metric: Convergence and performance analysis. Pattern Recognit. Lett.
**2017**, 100, 67–73. [Google Scholar] [CrossRef] - Wang, Y.; Lin, X.; Wu, L.; Zhang, W.; Zhang, Q.; Huang, X. Robust subspace clustering for multi-view data by exploiting correlation consensus. IEEE Trans. Image Process.
**2015**, 24, 3939–3949. [Google Scholar] [CrossRef] - Wang, Y.; Zhang, W.; Wu, L.; Lin, X.; Zhao, X. Unsupervised metric fusion over multiview data by graph random walk-based cross-view diffusion. IEEE Trans. Neural Netw. Learn. Syst.
**2017**, 28, 57–70. [Google Scholar] [CrossRef] - Wu, L.; Wang, Y.; Shao, L. Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrieval. IEEE Trans. Image Process.
**2019**, 28, 1602–1612. [Google Scholar] [CrossRef] [PubMed] - Jolliffe, I. Principal component analysis. In International Encyclopedia of Statistical Science; Springer: Berlin/Heidelberg, Germany, 2011; pp. 1094–1096. [Google Scholar]
- Earle, D.; Hurley, C.B. Advances in dendrogram seriation for application to visualization. J. Comput. Gr. Stat.
**2015**, 24, 1–25. [Google Scholar] [CrossRef]

**Figure 1.**Node of the Wireless Sensor Network where the symmetry of classification performance metrics has been primarily applied.

**Figure 2.**Imbalance coefficient (solid blue line) and imbalance ratio (dashed green line) vs. the proportion of positive elements in the dataset.

**Figure 3.**3D representation of a 4-dimension metric value ${\mu}_{j}^{k}\left({\alpha}^{k},{\beta}^{k},{\delta}^{k}\right)$. The value of the metric ${\mu}_{j}^{k}$ is colour-coded for every point in the $\left({\alpha}^{k},{\beta}^{k},{\delta}^{k}\right)$ 3D space.

**Figure 4.**Representation of a metric value ${\mu}_{j}^{k}\left({\alpha}^{k},{\beta}^{k}\right)$ for $\delta =0.75$. (

**a**) Slice of the 3D graphic by a plane corresponding to $\delta =0.75$; (

**b**) 2D representation of the slice.

**Figure 5.**Heat map of a metric value ${\mu}_{j}^{k}\left({\alpha}^{k},{\beta}^{k}\right)$ for $\delta =0.75$.

**Figure 6.**Panel of heat maps representing the metric ${\mu}_{j}^{k}\left({\alpha}^{k},{\beta}^{k},{\delta}^{k}\right)$.

**Figure 7.**Transformation type $\alpha $ of a metric. (

**a**) Baseline metric. (

**b**) Reflection symmetry with respect to the hyperplane $\alpha =0.5$.

**Figure 8.**Transformation type $\beta $ of a metric. (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the hyperplane $\beta =0.5$.

**Figure 9.**Transformation type $\delta $ of a metric. (

**a**) Baseline metric. (

**b**) Reflection symmetry with respect to the hyperplane $\delta =0$.

**Figure 10.**Transformation type $\mu $ of a metric. (

**a**) Baseline metric. (

**b**) Reflection symmetry with respect to the hyperplane $\mu =0$.

**Figure 11.**Transformation type $\sigma $ of a metric. (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the hyperplane $\alpha =\beta $.

**Figure 12.**Transformation by inverse labelling of classes (${T}^{L}$). (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the main diagonal (${T}^{\sigma}$); (

**c**) Reflection symmetry with respect to the plane $\delta =0$ (${T}^{\delta})$; (

**d**) Reflection symmetry with respect to the plane $\mu =0$ (colour inversion, ${T}^{\mu}$).

**Figure 13.**Transformation by inverse scoring (${T}^{S}$). (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the plane $\alpha =0$ (${T}^{\alpha}$); (

**c**) Reflection symmetry with respect to the plane $\beta =0$ (${T}^{\beta})$; (

**d**) Reflection symmetry with respect to the plane $\mu =0$ (colour inversion, ${T}^{\mu}$).

**Figure 14.**Transformation by full inversion scoring (${T}^{F}$). (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the plane $\alpha =0$ (${T}^{\alpha}$); (

**c**) Reflection symmetry with respect to the plane $\beta =0$ (${T}^{\beta})$. (

**c**) Reflection symmetry with respect to the main diagonal (${T}^{\sigma}$); (

**d**) Reflection symmetry with respect to the plane $\delta =0$ (${T}^{\delta})$. (

**e**) Reflection symmetry with respect to the plane $\mu =0$ (colour inversion, ${T}^{\mu}$).

**Figure 16.**Symmetry of accuracy with respect to inverse labelling (${T}^{L}$). (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the main diagonal (${T}^{\sigma}$); (

**c**) Reflection symmetry with respect to the plane $\delta =0$ (${T}^{\delta})$.

**Figure 17.**Symmetry of accuracy with respect to the inverse scoring (${T}^{S}$). (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the plane $\alpha =0$ (${T}^{\alpha}$); (

**c**) Reflection symmetry with respect to the plane $\beta =0$ (${T}^{\beta})$; (

**d**) Reflection symmetry with respect to the plane $\mu =0$ (colour inversion, ${T}^{\mu}$).

**Figure 18.**Symmetry of accuracy with respect to the full inversion (${T}^{F}$). (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the plane $\alpha =0$ (${T}^{\alpha}$); (

**c**) Reflection symmetry with respect to the plane $\beta =0$ (${T}^{\beta})$; (

**d**) Reflection symmetry with respect to the main diagonal (${T}^{\sigma}$); (

**e**) Reflection symmetry with respect to the plane $\delta =0$ (${T}^{\delta})$; (

**f**) Reflection symmetry with respect to the plane $\mu =0$ (colour inversion, ${T}^{\mu}$).

**Figure 19.**Symmetry of precision with respect to the full inversion (${T}^{F}$). (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the plane $\alpha =0$ (${T}^{\alpha}$); (

**c**) Reflection symmetry with respect to the plane $\beta =0$ (${T}^{\beta})$; (

**d**) Reflection symmetry with respect to the main diagonal (${T}^{\sigma}$); (

**e**) Reflection symmetry with respect to the plane $\delta =0$ (${T}^{\delta})$; (

**f**) Reflection symmetry with respect to the plane $\mu =0$ (colour inversion, ${T}^{\mu}$).

**Figure 20.**Symmetry of geometric mean with respect to ${T}^{\sigma}$. (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the main diagonal (${T}^{\sigma}$).

**Figure 21.**Symmetry of bookmaker informedness with respect to ${T}^{\sigma}$. (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the main diagonal (${T}^{\sigma}$).

**Figure 22.**Symmetry of bookmaker informedness with respect to the inverse scoring (${T}^{S}$). (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the plane $\alpha =0$ (${T}^{\alpha}$); (

**c**) Reflection symmetry with respect to the plane $\beta =0$ (${T}^{\beta})$; (

**d**) Reflection symmetry with respect to the plane $\mu =0$ (colour inversion, ${T}^{\mu}$).

**Figure 23.**Symmetry of bookmaker informedness with respect to the full inversion (${T}^{F}$). (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the plane $\alpha =0$ (${T}^{\alpha}$); (

**c**) Reflection symmetry with respect to the plane $\beta =0$ (${T}^{\beta})$; (

**c**) Reflection symmetry with respect to the main diagonal (${T}^{\sigma}$); (

**d**) Reflection symmetry with respect to the plane $\delta =0$ (${T}^{\delta})$; (

**e**) Reflection symmetry with respect to the plane $\mu =0$ (colour inversion, ${T}^{\mu}$).

**Figure 24.**Symmetry of sensitivity with respect to the combined transformation (${T}^{\alpha \mu}$). (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the plane $\alpha =0$ (${T}^{\alpha}$); (

**c**) Reflection symmetry with respect to the plane $\mu =0$ (colour inversion, ${T}^{\mu}$).

**Figure 25.**Symmetry of specificity with respect to the combined transformation (${T}^{\beta \mu}$) (

**a**) Baseline metric; (

**b**) Reflection symmetry with respect to the plane $\beta =0$ (${T}^{\beta}$); (

**c**) Reflection symmetry with respect to the plane $\mu =0$ (colour inversion, ${T}^{\mu}$).

**Figure 28.**Cross-symmetry of the $PRCn-NPVn$ pair with respect to the inverse labelling (${T}^{L}$). (

**a**) Baseline $PRCn$ metric; (

**b**) Baseline $NPVn$ metric; (

**c**) Reflection symmetry of $NPVn$ with respect to the main diagonal (${T}^{\sigma}$); (

**d**) Reflection symmetry of $NPVn$ with respect to the plane $\delta =0$ (${T}^{\delta})$.

**Figure 29.**Cross-symmetry of the $PRCn-NPVn$ pair with respect to the inverse scoring (${T}^{S}$). (

**a**) Baseline $PRCn$ metric; (

**b**) Baseline $NPVn$ metric. (

**c**) Reflection symmetry of $NPVn$ with respect to the plane $\alpha =0$ (${T}^{\alpha}$); (

**d**) Reflection symmetry of $NPVn$ with respect to the plane $\beta =0$ (${T}^{\beta})$; (

**e**) Reflection symmetry of $NPVn$ with respect to the plane $\mu =0$ (colour inversion, ${T}^{\mu}$).

**Figure 30.**Cross-symmetry of the $SNSn-SPCn$ pair with respect to the inverse labelling (${T}^{L}$). (

**a**) Baseline $SNSn$ metric; (

**b**) Baseline $SPCn$ metric; (

**c**) Reflection symmetry of $SPCn$ with respect to the main diagonal (${T}^{\sigma}$); (

**d**) Reflection symmetry of $SPCn$ with respect to the plane $\delta =0$ (${T}^{\delta})$.

**Figure 31.**Cross-symmetry of the $SNSn-SPCn$ pair with respect to the full inversion (${T}^{F}$). (

**a**) Baseline $SNSn$ metric. (

**b**) Baseline $SPCn$ metric. (

**c**) Reflection symmetry of $SPCn$ with respect to the main diagonal (${T}^{\sigma}$). (

**d**) Reflection symmetry of $SPCn$ with respect to the plane $\delta =0$ (${T}^{\delta})$. (

**e**) Reflection symmetry with respect to the plane $\alpha =0$ (${T}^{\alpha}$). (

**f**) Reflection symmetry with respect to the plane $\beta =0$ (${T}^{\beta})$. (

**g**) Reflection symmetry with respect to the plane $\mu =0$ (colour inversion, ${T}^{\mu}$).

**Figure 33.**Local probability density function of every metric as a function of $\delta $. The value of pdf is colour coded.

Transformation | ${\mathit{\alpha}}^{\mathit{k}}$ | ${\mathit{\beta}}^{\mathit{k}}$ | ${\mathit{\delta}}^{\mathit{k}}$ | ${\mathit{\mu}}^{\mathit{k}}$ |
---|---|---|---|---|

$\alpha $ | $1-{\alpha}^{B}$ | ${\beta}^{B}$ | ${\delta}^{B}$ | ${\mu}^{B}$ |

$\beta $ | ${\alpha}^{B}$ | $1-{\beta}^{B}$ | ${\delta}^{B}$ | ${\mu}^{B}$ |

$\delta $ | ${\alpha}^{B}$ | ${\beta}^{B}$ | $-{\delta}^{B}$ | ${\mu}^{B}$ |

$\mu $ | ${\alpha}^{B}$ | ${\beta}^{B}$ | ${\delta}^{B}$ | $-{\mu}^{B}$ |

$\sigma $ | ${\beta}^{B}$ | ${\alpha}^{B}$ | ${\delta}^{B}$ | ${\mu}^{B}$ |

Transformation Code | $\mathit{\mu}$ | $\mathit{\sigma}$ | $\mathit{\delta}$ | $\mathit{\beta}$ | $\mathit{\alpha}$ |
---|---|---|---|---|---|

28 | 1 | 1 | 1 | 0 | 0 |

Symbol | Metric | Scoring |
---|---|---|

$SNS$ | Sensitivity | $\frac{a}{a+f}$ |

$SPC$ | Specificity | $\frac{b}{b+g}$ |

$PRC$ | Precision | $\frac{a}{a+g}$ |

$NPV$ | Negative Predictive Value | $\frac{b}{b+f}$ |

$ACC$ | Accuracy | $\frac{a+b}{a+f+b+g}$ |

${F}_{1}$ | ${F}_{1}\text{}\mathrm{score}$ | $2\frac{PRC\xb7SNS}{PRC+SNS}$ |

$GM$ | Geometric Mean | $\sqrt{SNS\xb7SPC}$ |

$MCC$ | Matthews Correlation Coefficient | $\frac{a\xb7b-g\xb7f}{\sqrt{\left(a+g\right)\left(a+f\right)\left(b+g\right)\left(b+f\right)}}$ |

$BM$ | Bookmaker Informedness | $SNS+SPC-1$ |

$MK$ | Markedness | $PPV+NPV-1$ |

Code | $\mathit{\mu}$ | $\mathit{\sigma}$ | $\mathit{\delta}$ | $\mathit{\beta}$ | $\mathit{\alpha}$ | Specific Order | Any Order |
---|---|---|---|---|---|---|---|

12 | 0 | 1 | 1 | 0 | 0 | $\delta \sigma $ | |

15 | 0 | 1 | 1 | 1 | 1 | $\alpha \sigma \beta \left(=\sigma \right)$ $\beta \sigma \alpha \left(=\sigma \right)$ | $\delta $ |

19 | 1 | 0 | 0 | 1 | 1 | $\alpha \beta \mu $ | |

31 | 1 | 1 | 1 | 1 | 1 | $\alpha \beta \sigma $ $\beta \alpha \sigma $ $\sigma \alpha \beta $ $\sigma \beta \alpha $ | $\delta \mu $ |

Code | $\mathit{\mu}$ | $\mathit{\sigma}$ | $\mathit{\delta}$ | $\mathit{\beta}$ | $\mathit{\alpha}$ | Specific Order | Any Order |
---|---|---|---|---|---|---|---|

31 | 1 | 1 | 1 | 1 | 1 | $\alpha \beta \sigma $ $\beta \alpha \sigma $ $\sigma \alpha \beta $ $\sigma \beta \alpha $ | $\delta \mu $ |

Code | $\mathit{\mu}$ | $\mathit{\sigma}$ | $\mathit{\delta}$ | $\mathit{\beta}$ | $\mathit{\alpha}$ | Specific Order | Any Order |
---|---|---|---|---|---|---|---|

4 | 0 | 0 | 1 | 0 | 0 | $\delta $ | |

8 | 0 | 1 | 0 | 0 | 0 | $\sigma $ | |

11 | 0 | 1 | 0 | 1 | 1 | $\alpha \sigma \beta \left(=\sigma \right)$ $\beta \sigma \alpha \left(=\sigma \right)$ | |

12 | 0 | 1 | 1 | 0 | 0 | $\delta \sigma $ | |

15 | 0 | 1 | 1 | 1 | 1 | $\alpha \sigma \beta \left(=\sigma \right)$ $\beta \sigma \alpha \left(=\sigma \right)$ | $\delta $ |

Code | $\mathit{\mu}$ | $\mathit{\sigma}$ | $\mathit{\delta}$ | $\mathit{\beta}$ | $\mathit{\alpha}$ | Specific Order | Any Order |
---|---|---|---|---|---|---|---|

4 | 0 | 0 | 1 | 0 | 0 | $\delta $ | |

8 | 0 | 1 | 0 | 0 | 0 | $\sigma $ | |

11 | 0 | 1 | 0 | 1 | 1 | $\alpha \sigma \beta \left(=\sigma \right)$ $\beta \sigma \alpha \left(=\sigma \right)$ | |

12 | 0 | 1 | 1 | 0 | 0 | $\delta \sigma $ | |

15 | 0 | 1 | 1 | 1 | 1 | $\alpha \sigma \beta \left(=\sigma \right)$ $\beta \sigma \alpha \left(=\sigma \right)$ | $\delta $ |

19 | 1 | 0 | 0 | 1 | 1 | $\alpha \beta \mu $ | |

23 | 1 | 0 | 1 | 1 | 1 | $\alpha \beta \delta \mu $ | |

27 | 1 | 1 | 0 | 1 | 1 | $\alpha \beta \sigma $ $\beta \alpha \sigma $ $\sigma \alpha \beta $ $\sigma \beta \alpha $ | $\mu $ |

31 | 1 | 1 | 1 | 1 | 1 | $\alpha \beta \sigma $ $\beta \alpha \sigma $ $\sigma \alpha \beta $ $\sigma \beta \alpha $ | $\delta \mu $ |

Code | $\mathit{\mu}$ | $\mathit{\sigma}$ | $\mathit{\delta}$ | $\mathit{\beta}$ | $\mathit{\alpha}$ | Specific Order | Any Order |
---|---|---|---|---|---|---|---|

2 | 0 | 0 | 0 | 1 | 0 | $\beta $ | |

4 | 0 | 0 | 1 | 0 | 0 | $\delta $ | |

6 | 0 | 0 | 1 | 1 | 0 | $\beta \delta $ | |

17 | 1 | 0 | 0 | 0 | 1 | $\alpha \mu $ | |

19 | 1 | 0 | 0 | 1 | 1 | $\alpha \beta \mu $ | |

21 | 1 | 0 | 1 | 0 | 1 | $\alpha \delta \mu $ | |

23 | 1 | 0 | 1 | 1 | 1 | $\alpha \beta \delta \mu $ |

Code | $\mathit{\mu}$ | $\mathit{\sigma}$ | $\mathit{\delta}$ | $\mathit{\beta}$ | $\mathit{\alpha}$ | Specific Order | Any Order |
---|---|---|---|---|---|---|---|

1 | 0 | 0 | 0 | 0 | 1 | $\alpha $ | |

4 | 0 | 0 | 1 | 0 | 0 | $\delta $ | |

5 | 0 | 0 | 1 | 0 | 1 | $\alpha \delta $ | |

18 | 1 | 0 | 0 | 1 | 0 | $\beta \mu $ | |

19 | 1 | 0 | 0 | 1 | 1 | $\alpha \beta \mu $ | |

22 | 1 | 0 | 1 | 1 | 0 | $\beta \delta \mu $ | |

23 | 1 | 0 | 1 | 1 | 1 | $\alpha \beta \delta \mu $ |

Metric | Independent of | Symmetry (under Inversion of) | ||||
---|---|---|---|---|---|---|

$\mathit{\alpha}$ | $\mathit{\beta}$ | $\mathit{\delta}$ | Labelling | Scoring | Full | |

$SNSn$ | ✓ | ✓ | ✓ | |||

$SPCn$ | ✓ | ✓ | ✓ | |||

$PRCn$ | ✓ | |||||

$NPVn$ | ✓ | |||||

$ACCn$ | ✓ | ✓ | ✓ | |||

${F}_{1}n$ | ||||||

$GMn$ | ✓ | ✓ | ||||

$MCC$ | ✓ | ✓ | ✓ | |||

$BM$ | ✓ | ✓ | ✓ | ✓ | ||

$MK$ | ✓ | ✓ | ✓ |

Code | $\mathit{\mu}$ | $\mathit{\sigma}$ | $\mathit{\delta}$ | $\mathit{\beta}$ | $\mathit{\alpha}$ | Specific Order | Any Order |
---|---|---|---|---|---|---|---|

12 | 0 | 1 | 1 | 0 | 0 | $\delta \sigma $ | |

15 | 0 | 1 | 1 | 1 | 1 | $\alpha \sigma \beta \left(=\sigma \right)$ $\beta \sigma \alpha \left(=\sigma \right)$ | $\delta $ |

19 | 1 | 0 | 0 | 1 | 1 | $\alpha \beta \mu $ |

Code | $\mathit{\mu}$ | $\mathit{\sigma}$ | $\mathit{\delta}$ | $\mathit{\beta}$ | $\mathit{\alpha}$ | Specific Order | Any Order |
---|---|---|---|---|---|---|---|

8 | 0 | 1 | 0 | 0 | 0 | $\sigma $ | |

9 | 0 | 1 | 0 | 0 | 1 | $\alpha \sigma $ | |

10 | 0 | 1 | 0 | 1 | 0 | $\sigma \beta $ | |

11 | 0 | 1 | 0 | 1 | 1 | $\alpha \sigma \beta \left(=\sigma \right)$ $\sigma \beta \alpha \left(=\sigma \right)$ | |

12 | 0 | 1 | 1 | 0 | 0 | $\sigma \delta $ | |

13 | 0 | 1 | 1 | 0 | 1 | $\alpha \sigma $ | $\delta $ |

14 | 0 | 1 | 1 | 1 | 0 | $\sigma \beta $ | $\delta $ |

15 | 0 | 1 | 1 | 1 | 1 | $\alpha \sigma \beta \left(=\sigma \right)$ $\sigma \beta \alpha \left(=\sigma \right)$ | $\delta $ |

25 | 1 | 1 | 0 | 0 | 1 | $\sigma \alpha $ | $\mu $ |

26 | 1 | 1 | 0 | 1 | 0 | $\beta \sigma $ | $\mu $ |

27 | 1 | 1 | 0 | 1 | 1 | $\alpha \beta \sigma $ $\beta \alpha \sigma $ $\sigma \alpha \beta $ $\sigma \beta \alpha $ | $\mu $ |

29 | 1 | 1 | 1 | 0 | 1 | $\sigma \alpha $ | $\delta \mu $ |

30 | 1 | 1 | 1 | 1 | 0 | $\beta \sigma $ | $\delta \mu $ |

31 | 1 | 1 | 1 | 1 | 1 | $\alpha \beta \sigma $ $\beta \alpha \sigma $ $\sigma \alpha \beta $ $\sigma \beta \alpha $ | $\delta \mu $ |

Metric | Cross-Symmetry (under Inversion of) | ||
---|---|---|---|

Labelling | Scoring | Full | |

$SNSn$ | $SPCn$ | $\left(SNSn\right)$ | $SPCn$ |

$SPCn$ | $SNSn$ | $\left(SPCn\right)$ | $SNSn$ |

$PRCn$ | $NPVn$ | $NPVn$ | $\left(PRCn\right)$ |

$NPVn$ | $PRCn$ | $PRCn$ | $\left(NPVn\right)$ |

Metric | Statistical Symmetry | ||
---|---|---|---|

Local | Global (Skewness) | ||

$SNSn$ | ✓ | ✓ | |

$SPCn$ | ✓ | ✓ | |

$PRCn$ | ✓ | ||

$NPVn$ | ✓ | ||

$ACCn$ | ✓ | ✓ | |

${F}_{1}n$ | (0.14) | ||

$GMn$ | (0.18) | ||

$MCC$ | ✓ | ✓ | |

$BM$ | ✓ | ✓ | |

$MK$ | ✓ | ✓ |

**Table 15.**Examples of symmetric behaviour of metrics under several transformations (for balanced classes). Numbers in bold represent cases of asymmetric behaviour.

Metric | Baseline $\mathit{\alpha}:0.8$ $;\text{}\mathit{\beta}:0.7$ | Labelling Inversion $\mathit{\alpha}:0.7;\mathit{\beta}:0.8$ | Scoring Inversion $\mathit{\alpha}:0.2$ $;\text{}\mathit{\beta}:0.3$ | Full Inversion $\mathit{\alpha}:0.3$ $;\text{}\mathit{\beta}:0.2$ |
---|---|---|---|---|

$ACCn$ | $0.500$ | $0.500$ | $-0.500$ | $-0.500$ |

$MCC$ | $0.503$ | $0.503$ | $-0.503$ | $-0.503$ |

$BM$ | $0.500$ | $0.500$ | $-0.500$ | $-0.500$ |

$MK$ | $0.505$ | $0.505$ | $-0.505$ | $-0.505$ |

$GMn$ | $0.497$ | $0.497$ | $-\mathbf{0.510}$ | $-\mathbf{0.510}$ |

$SNSn$ | $0.600$ | $\mathbf{0.400}$ | $-0.600$ | $-\mathbf{0.400}$ |

$SPCn$ | $0.400$ | $\mathbf{0.600}$ | $-0.400$ | $-\mathbf{0.600}$ |

$PRCn$ | $0.455$ | $\mathbf{0.556}$ | $-\mathbf{0.556}$ | $-0.455$ |

$NPVn$ | $0.556$ | $\mathbf{0.455}$ | $-\mathbf{0.455}$ | $-0.566$ |

${F}_{1}n$ | $0.524$ | $\mathbf{0.474}$ | $-\mathbf{0.579}$ | $-\mathbf{0.429}$ |

Cluster | Metric | Independent of | Symmetry (under Inversion of) | Statistical Symmetry | ||||||
---|---|---|---|---|---|---|---|---|---|---|

$\mathit{\alpha}$ | $\mathit{\beta}$ | $\mathit{\delta}$ | Labelling | Scoring | Full | Local | Global (Skewness) | |||

I | a | $ACCn$ | ✓ | ✓ | ✓ | ✓ | ✓ | |||

$MCC$ | ✓ | ✓ | ✓ | ✓ | ✓ | |||||

$MK$ | ✓ | ✓ | ✓ | ✓ | ✓ | |||||

b | $BM$ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |||

II | $SNSn$ | ✓ | ✓ | $SPCn$ | ✓ | $SPCn$ | ✓ | ✓ | ||

$SPCn$ | ✓ | ✓ | $SNSn$ | ✓ | $SNSn$ | ✓ | ✓ | |||

III | $PRCn$ | $NPVn$ | $NPVn$ | ✓ | ✓ | |||||

$NPVn$ | $PRCn$ | $PRCn$ | ✓ | ✓ | ||||||

IV | $GMn$ | ✓ | ✓ | (0.18) | ||||||

V | ${F}_{1}n$ | (0.14) |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Luque, A.; Carrasco, A.; Martín, A.; Lama, J.R.
Exploring Symmetry of Binary Classification Performance Metrics. *Symmetry* **2019**, *11*, 47.
https://doi.org/10.3390/sym11010047

**AMA Style**

Luque A, Carrasco A, Martín A, Lama JR.
Exploring Symmetry of Binary Classification Performance Metrics. *Symmetry*. 2019; 11(1):47.
https://doi.org/10.3390/sym11010047

**Chicago/Turabian Style**

Luque, Amalia, Alejandro Carrasco, Alejandro Martín, and Juan Ramón Lama.
2019. "Exploring Symmetry of Binary Classification Performance Metrics" *Symmetry* 11, no. 1: 47.
https://doi.org/10.3390/sym11010047