[WiP] Individual Fairness Notion

[pfistfl]

The goal of this PR is to implement individual fairness notions in mlr3fairness.

Scope:

• Implement a flexible measure IndividualFairness that allows measuring individual fairness.
  Functionality should have the following arguments:

  • dX: Distance metric in X-space (difference between individuals).
  • dY: Distance metric in Y-space (difference between observations).
  • L: Lipschitz constant. Relates distances in X and Y.
  • cols: Which columns in X should be used to compute dX?
  • comparison: What comparator to use, see below

  The current implementation compares to all other data points, not sure if this is reasonable.

• Implement visualizations, that e.g. show most unfairly treated individuals etc.

• Perhaps one could also use iml to understand sources of unfairness.

Open Questions:

• For an individual x_i, who do we compare to? All others / comparator point / clusters?
  How would we select comparator points?
• Does reporting aggregated scores make sense to assess fairness on a model level (instead of an individual level?)
  How should we aggregate this?
• Technical: Measures are now valid for a single data set. How can we use this e.g. in benchmarks?

Required Research

• How is this solved in other fairness-aware software packages?
• Individual fairness relies on a strong assumption regarding no measurement errors/biases.
  Is there anything we can do to help with assessing that?

Other:

• Disregard individual fairness metrics for survival for now. Solve for classif and regr for now.

Relevant Literature:

• Dwork, 2012
• Binns, 2019
• Mukherjee, 2020
• Zemel, 2013