In AI systems, bias refers to systematic errors or skews in model outputs that arise from imbalances or prejudices present in training data, from the design of the objective function, or from the way outputs are filtered and sampled. Statistical bias (a model consistently over- or under-predicting a value) and social bias (a model producing outputs that reflect racial, gender, cultural, or socioeconomic stereotypes) are both real problems, though they require different mitigation strategies. Social bias is particularly consequential in deployed products because it can cause measurable harm to users from affected groups.

Think of it like a bathroom scale that was calibrated using only adult men. It will still weigh children and women, but its readings will be systematically off for those groups — not because the scale is broken, but because the calibration process did not represent the full population it would later serve.

A simple example of demographic bias in text generation: if a model trained on historical hiring data is prompted to write a performance review, it may use more leadership language for male-coded names and more supportive language for female-coded names, mirroring patterns in the training corpus regardless of actual performance. Evaluation tools such as WinoBias or StereoSet present paired sentences that differ only in gendered or racial terms, then measure whether the model assigns different probabilities to otherwise identical completions.

Bias connects directly to the Loss Function and Pretraining (where skewed corpora are ingested at scale), RLHF (which can reduce social bias but can also introduce new biases depending on annotator demographics), and Hallucination (where confident-but-wrong outputs often reflect stereotypical associations rather than factual recall). Overfitting to a non-representative dataset is one of the primary mechanisms through which bias enters a model.

You will encounter bias analysis in model evaluation reports, fairness audits, and any deployment context where outputs influence consequential decisions such as hiring, lending, or medical triage.