Debias

Introduction

Actable offers you the possibility to debias features from your datasets to offer you unbiased predictions. By extrapolating the debiased features with the biased groups, our algorithm calculates residual values of this prediction and use them for your classification to offer you unbiased results.

Parameters

  • Sensitive Groups: Groups that are biased and are creating bias in other features of the dataset.
  • Proxy Features: Features that need to be debiased from the biased groups.

Evaluation

Actable AI debias feature works for both classification and regression. We demonstrate our debias feature using the following biased dataset: titanic survival passenger number and glassdoor base pay record.

Classification

The RMS Titanic sank on 15th April of 1912, which is one of the most famous maritime disasters. There is post-disaster research pointing out the fact that passengers aboard survived in different chances depending on which class they are travelling with. People travelling with first class would have higher chances to be survivors.

We manually select a subset of data as our prediction dataset that guarantees that our prediction dataset has an equal distribution of survivors for different travelling classes. However, the prediction result does not show an equal chance for different passenger classes to survive.

By enabling our debias feature, we can observe a significant improvement in the distribution. The following image shows the survival distribution across first, second and third-class passengers for the original dataset, prediction without debias, prediction with debias enabled, from left to right. The x-axis shows the possibility of survival, and the values y-axis stands for survived (0) and not survived (1).

_images/distribution_titanic_survival.png

Regression

As we may know, the gender pay gap is a real-world discrimination problem that shows a difference between the remuneration among different genders. We will use the glassdoor dataset that indicates the distribution for the base pay across males and females to illustrate this problem.

Due to the nature of the data distribution in this dataset, the prediction result is also biased, even after manually selecting the prediction dataset to have the same base pay distribution for males and females. The prediction result for base pay is showing a clear bias that females are underpaid compared to males, and the distribution is inclined to align with the original dataset.

By enabling our debias feature, we can clearly observe a significant improvement in the distribution. The following image shows the base pay distribution across males and females for the original dataset, prediction without debias, prediction with debias enabled, from left to right. The x-axis represents the base pay value, and y-axis stands for the density.

_images/distribution_glassdoor_basepay.png

Performance Impact

Enabling the debias might slightly impact your model’s performance. Take the titanic example, by comparing to the model trained without debias, the prediction accuracy for the model trained, with debias enabled drops around 7%, from 76.033% to 69.421%. The AUC score drops around 0.01, from 0.775 drops to 0.759. About the base pay prediction, the prediction made by our debiased model has an RMSE of 24595.627 compared to the model without debias enabled 10819.215