Business analysis glossary
AI Bias
What it is
AI bias is when an AI system produces systematically unfair results for some groups of people, often because its training data reflects past inequalities or is not representative, for example a loan model that approves fewer applicants from certain areas for reasons unrelated to repayment.
Why it matters
Biased systems harm people, damage trust and can break laws and regulator expectations. Analysts help by asking which groups could be affected, requiring fairness checks before and after release and making sure decisions can be explained and challenged by the people affected.
Example
Testing a hiring screening model, the team finds it ranks candidates from tier three cities lower even with the same skills, because past hires came mostly from metros. The analyst adds a requirement to remove location from the model's inputs and to compare shortlist rates across city tiers every month.