It is an interesting paper from Microsoft Research on the bias. I’d be interested to see how it is applicable in healthcare.
In the paper, we also consider two interventions: decoupling the assessment rule by group (using a different assessment rule for each group instead of a joint assessment rule) and subsidizing the cost of investment for a disadvantaged group. We find that under these dynamics, the welfare effects of decoupling crucially depend on realizability as well as the initial qualification rates. If the classification problem is non-realizable, it is in fact possible for decoupling to hurt a group with low initial qualification rate by reinforcing the status quo, whereas a joint assessment rule would have increased that group’s qualification rates in the long run.
via When bias begets bias: A source of negative feedback loops in AI systems – Microsoft Research
RadOncNotes 