Data Augmentation for Reliability and Fairness in Counselling Quality Classification

The mental health domain poses serious challenges to the validity of existing Natural Language Processing (NLP) approaches. Scarce and unbalanced data limits models’ reliability and fairness, therefore hampering real-world application. In this work, we address these challenges by using our recently released Anno-MI dataset, containing professionally annotated transcriptions in motivational interviewing (MI). To do so, we inspect the effects of data augmentation on classical machine (CML) and deep learning (DL) approaches for counselling quality classification. First, we adopt augmentation to balance the target label in order to improve the classifiers’ reliability. Next, we conduct the bias and fairness analysis by choosing the therapy topic as the sensitive variable. Finally, we implement a fairness-aware augmentation technique, showing how topic-wise bias can be mitigated by augmenting the target label with respect to the sensitive variable.Our work is the first step towards increa sing reliability and reducing the bias of classification models, as well as dealing with data scarcity and imbalance in mental health.