Bias in AI: When bias begets bias; A source of negative feedback loops in AI systems

It is an interesting paper from Microsoft Research on the bias. I’d be interested to see how it is applicable in healthcare.

In the paper, we also consider two interventions: decoupling the assessment rule by group (using a different assessment rule for each group instead of a joint assessment rule) and subsidizing the cost of investment for a disadvantaged group. We find that under these dynamics, the welfare effects of decoupling crucially depend on realizability as well as the initial qualification rates. If the classification problem is non-realizable, it is in fact possible for decoupling to hurt a group with low initial qualification rate by reinforcing the status quo, whereas a joint assessment rule would have increased that group’s qualification rates in the long run.

via When bias begets bias: A source of negative feedback loops in AI systems – Microsoft Research