Why AI Biases Matter in the Legal Industry

View on LinkedIn

Have you really thought about this?

Imagine you take a photo of a bicycle and ask an AI how much it's worth. Now imagine that you appear in that same photo. Depending on your skin color, the AI might give completely different prices: $1,000 if you are light-skinned, $200 if you are dark-skinned. The same bicycle, different prices, based solely on who appears in the photo.

This is not a hypothetical example. It is a real experiment conducted at Stanford by Alejandro Salinas de León that demonstrates how AI systems, supposedly objective, can reproduce and amplify existing social biases. The AI was not programmed to discriminate — it simply learned from data that reflects real-world economic and social inequalities.

excerpt from the interview at Lawgic's Legal AI Week

Click if you want to see the full content

Why should you care as an attorney?

Because your corporate clients do not want to be left behind and are implementing AI left and right. It is not a trend — it is competitiveness, and this is only going to grow; we are barely at the beginning.

A clear example is the Amazon case. In 2014, Amazon developed an AI system to revolutionize its hiring processes. The idea seemed perfect: automatically analyze résumés to identify the best candidates, eliminating human bias and saving time and resources in the process. Amazon fed the system with 10 years of received résumés, expecting the AI to learn to identify patterns of successful candidates.

However, the result was alarming. The system began to systematically discriminate against female candidates in surprisingly sophisticated ways. It penalized résumés that included the word "women" in any context — for example, "captain of the women's chess team" or "vice president of the Society of Women Engineers."

Amazon's engineers attempted to "fix" the system by removing terms explicitly related to gender. But the AI, demonstrating an unsettling capacity to find hidden patterns, began using other indicators as proxies to identify gender. For example, verbs commonly used in women's résumés ("participated," "facilitated," "organized") became negative factors. Even the writing style and format of the résumé could lead to unintentional discrimination.

The most revealing aspect was that the system was not "defective" — it was functioning exactly as expected, identifying patterns in historical data.

The problem was that this data reflected decades of biased hiring practices in the tech industry. Amazon eventually abandoned the project in 2017, but the case perfectly illustrates how historical biases can infiltrate and be amplified in seemingly objective AI systems.

This type of situation exposes companies to significant legal liability. As legal counsel, you need to understand these risks in order to guide your clients in the responsible implementation of AI, or to defend their interests when these systems produce discriminatory outcomes.

Key concepts: What do we mean when we talk about biases?

Key concepts you need to know to navigate the world of AI and its legal implications

Algorithmic bias: The systematic tendency of an AI model to favor certain outcomes, generally reflecting biases present in the training data.

• Algorithmic bias: The systematic tendency of an AI model to favor certain outcomes, generally reflecting biases present in the training data.

Representation bias: Distortion that occurs when certain groups or characteristics are under- or overrepresented in the training data. Algorithmic discrimination: When an AI system produces outcomes that systematically treat different demographic groups differently. Discriminatory proxies: Apparently neutral variables that the model uses to infer protected characteristics (such as race or gender) and potentially discriminate based on them. Neural lobotomy: An experimental technique to mitigate bias by selectively deactivating specific "neurons" in the model that exhibit biased behavior. Axiological representation: The system of values and weights that the model assigns to different characteristics during its decision-making process. Disparate treatment vs. Disparate impact: The distinction between intentional discrimination in the design of the system versus unintentional discriminatory impacts in its outcomes. Algorithmic audit: A systematic process for evaluating bias and discrimination in AI systems, including fairness testing and impact analysis. Black-box effect: The inherent opacity in how AI models, especially the most complex ones, arrive at their conclusions, making it difficult to detect and correct biases.

• Representation bias: Distortion that occurs when certain groups or characteristics are under- or overrepresented in the training data.
• Algorithmic discrimination: When an AI system produces outcomes that systematically treat different demographic groups differently.
• Discriminatory proxies: Apparently neutral variables that the model uses to infer protected characteristics (such as race or gender) and potentially discriminate based on them.
• Neural lobotomy: An experimental technique to mitigate bias by selectively deactivating specific "neurons" in the model that exhibit biased behavior.
• Axiological representation: The system of values and weights that the model assigns to different characteristics during its decision-making process.
• Disparate treatment vs. Disparate impact: The distinction between intentional discrimination in the design of the system versus unintentional discriminatory impacts in its outcomes.
• Algorithmic audit: A systematic process for evaluating bias and discrimination in AI systems, including fairness testing and impact analysis.
• Black-box effect: The inherent opacity in how AI models, especially the most complex ones, arrive at their conclusions, making it difficult to detect and correct biases.

Remember that each AI model has unique training and, therefore, potentially different biases. For example, GPT-4 and GPT-3.5 may give different responses to the same question.

Algorithmic audits help identify these specific biases, and while technical tools exist — such as adjusting the "temperature" (which controls how creative or conservative the model is) and prompt engineering techniques that can partially mitigate these issues — no solution is definitive.

The key is to understand that we work with different models that require different bias mitigation strategies.