In recent years, regulatory technology (RegTech) for monitoring,reporting and compliance has become more and more important [1].Specifically in a financial context, an increased amount of regulation haspaved the way for technologies supporting regulatory authorities conductingsupervisory tasks [2,3]. Apart from high predictive performance,RegTech applications have to fulfill two important additionalrequirements which have been under-researched so far: applicationsbased on machine learning techniques have to offer explanations why aspecific classification decision was made to foster transparency andencourage trust towards system output [4]. Additionally, related systemsshould be fair, i.e. discrimination of members of specific groups orpopulations should be avoided [5].