Executive summary

The integration of large language models (LLMs) into investment management workflows has made biased artificial intelligence (AI) a critical governance concern. Financial authorities, including the US Treasury, have identified AI bias as a significant risk that can distort investment recommendations, amplify errors, and lead to adverse regulatory or reputational consequences. This report introduces a framework for understanding LLM biases by drawing analogies to human cognitive and emotional biases, categorising them as either implicit or explicit. Implicit LLM biases stem from pre-training data and architecture, while explicit biases manifest in observable “decision” processes such as tool usage and data selection. Awareness of these biases is essential before professionals can make informed decisions regarding mitigation.

Introduction

Investment professionals maintain a fiduciary responsibility to act in the best interests of their clients, a duty that requires the active management of biases. Early iterations of AI, such as ChatGPT-3.5, demonstrated clear biases, including discrimination against certain races and genders in résumé rating due to under-representation in training data. While newer models feature improved guardrails, developers remain in a cycle of identifying and refining biases as new models are released. In the investment context, LLMs may exhibit implicit preferences for specific sectors or stocks associated with high growth in their training data, potentially leading to the underweighting of contradictory information and systematic errors in judgment.

But what is bias? Clearing up the confusion

Bias is generally defined as a systematic deviation from “truth” or ideal judgment. In humans, cognitive biases are unconscious errors in thinking, such as confirmation bias (seeking confirming information) or anchoring and adjustment bias (relying too heavily on initial data). Emotional biases, such as loss aversion and overconfidence, are more difficult to detect as they stem from intuition rather than facts. In statistics and machine learning (ML), bias is the difference between the expected value of an estimator and the true parameter value. ML models often require a degree of bias to make the “inductive leap” necessary to generalise to unseen data, but this must be balanced against variance to avoid overfitting.

Disentangling human and LLM biases in augmented workflows

In augmented (human + AI) workflows, biases often overlap. Explicit biases can be injected into models by humans through system-level prompts, such as instructions to remain succinct or not to challenge a user’s view. Conversely, human implicit bias manifests when an analyst unconsciously provides only supportive evidence to an LLM’s context window. Distinguishing between these sources is vital for effective detection and mitigation. For instance, an analyst’s confirmation bias regarding a specific stock will lead to an explicitly biased LLM response based on the skewed data provided.

Detecting implicit LLM bias

Detecting implicit bias requires comparative and aggregative strategies, as it is rarely identifiable from a single response. Techniques include constructing controlled prompts with varied phrasing and recording changes in output sentiment or profitability ratios. Another method involves calculating a “self-coherency” score, where an LLM is asked to evaluate its own generated content for bias. A score of 1 indicates the model views the output as acceptable, while 0 indicates it flagged its own behavior as biased. While “bias benchmarks” are emerging to quantify these risks, few currently focus on real-world financial use cases.

Mitigating implicit LLM bias

Mitigation can be achieved by pre-processing content in the LLM’s context window, such as masking company names and dates with placeholders (e.g., “Company A”) to prevent the model from relying on pre-trained knowledge or look-ahead bias. A second approach involves the “wisdom of crowds” phenomenon, using multiple models from different providers and taking a consensus or average result to cancel out individual model biases. Finally, bias-detection LLMs can act as guardrails, assigning a “bias score” to outputs and triggering human review when thresholds are exceeded.

Detecting and mitigating explicit LLM bias

Explicit bias is detected by auditing the observable decisions an LLM makes, including its choice of tools, data sources, and reasoning traces. Mitigation requires governance mechanisms that constrain these decisions, such as hard-coding analytical steps, specifying data source selection, and embedding regular checkpoints for human review. Practitioners must be trained to identify human cognitive patterns, as LLM reasoning often mirrors these tendencies.

Case study: Revealing, quantifying, and mitigating implicit LLM bias and implicit human bias

A controlled case study examined nine OpenAI models to detect framing bias—the tendency to respond differently to the same information depending on whether it is presented positively or negatively. Testing 900 prompts across 10 categories, the study found that drawdown performance and client retention scenarios exhibited the strongest framing effects. Three mitigation methods were evaluated. Method 1 (adding a definition of framing bias) was ineffective. Method 2 (manually augmenting prompts to include both perspectives) provided the greatest reduction in bias. Method 3 (using a separate bias-detection LLM) significantly reduced the framing effect when the agent utilised high levels of reasoning, correctly identifying the bias in 97.2% of cases, though it did not outperform the human-in-the-loop approach.

Recommendations and conclusions

While bias cannot be entirely eliminated, investment professionals should aim to make it detectable and manageable. The report recommends a combination of practitioner education, task-specific experimentation, and the use of internal evaluation benchmarks. For high-risk or frequent-use cases, firms should develop systematic prompt engineering and audit trails. By adopting human-in-the-loop protocols and robust AI governance, organisations can move from unknowingly introducing implicit bias toward a more transparent and effective workflow.