Introduction

In organisational settings, the disclosure of downstream use can fundamentally alter the nature of an intermediate task. For instance, a human assistant asked to summarise interview transcripts might overemphasise the strengths of candidates who eventually receive offers, resulting in summaries that are less faithful to the original data. This behavioural shift reflects rational adaptation to perceived downstream purpose rather than lack of effort. This research investigates whether Large Language Models (LLMs) exhibit similar purpose-conditioned cognition. It explores whether restricting AI to assistant roles ensures neutral and unbiased intermediate outputs when the final prediction task is disclosed to the model.

The study utilizes a research design that holds input text, models, and scoring tasks fixed while varying only whether the downstream use of the output is disclosed. Models such as ChatGPT and Gemini are prompted to generate numerical measures of sentiment and competition intensity from earnings call transcripts. In the goal-blind condition, the model produces a score without reference to its eventual use. In the goal-aware condition, the model is informed that its output will be used to predict future stock returns or earnings. This design isolates the effect of objective disclosure on the model’s intermediate outputs.

Experimental design and data

The objective is to examine if LLMs systematically adjust outputs when informed of the downstream task. The study focuses on two economically important forecasting applications: monthly stock returns and quarterly earnings per share (EPS). Firm earnings call transcripts serve as the sole input. Rather than predicting outcomes directly, the models generate intermediate scores for use in predictive regressions. For the return prediction task, the model generates a continuous sentiment score from -1 to 1. For the earnings prediction task, the model generates a competition score from -1 to 1, capturing the intensity of competitive pressure.

The sample consists of S&P 500 firms over the period from January 2022 to December 2024. Earnings call transcripts are obtained from Capital IQ, stock returns from CRSP, and accounting information from Compustat. The baseline experiment uses the GPT-4o-mini model, which has a fixed knowledge cutoff of October 1, 2023. Robustness tests are conducted using Gemini 2.5, which has a knowledge cutoff of January 2025. This temporal separation allows the research to distinguish between performance driven by prompt design and performance arising from direct access to future information during training.

Findings

Prior to the knowledge cutoff, sentiment scores generated under both prompt designs exhibit economically meaningful return predictability. Goal-aware sentiment earns an average monthly return spread of 1.552%, while the corresponding spread based on goal-blind sentiment is 1.069%. This difference of 0.483 percentage point per month is statistically significant at the 5% level. This indicates that disclosing the downstream task increases the measured economic content of the scores before the cutoff. However, after the knowledge cutoff, this relative advantage disappears. The monthly High–Low spread equals 2.269% for the goal-aware portfolio and 2.239% for the goal-blind portfolio, with no significant difference between the two.

Regression analysis confirms these findings. In the pre-cutoff period, the incremental predictive content of goal-aware sentiment (the ‘Diff’ measure) is positive and significant. After the cutoff, the coefficient collapses to near-zero. Out-of-sample forecasting performance follows a similar pattern: goal-aware prompts outperform goal-blind prompts before the cutoff but deteriorate significantly afterward, falling below the goal-blind benchmark. In the earnings prediction task, goal-aware competition scores show a negative and significant relationship with future earnings pre-cutoff, but this effect vanishes after the knowledge cutoff.

Mechanism and subtlety of goal awareness

The research provides a theoretical framework explaining that LLMs shift from ‘measuring sensors’ under goal blindness to ‘optimising solvers’ when a purpose is disclosed. This behaviour is rooted in how models are trained via reinforcement learning from human feedback (RLHF), where they are rewarded for being helpful and aligned with user intent. Contextual statements about intended use enter the effective objective through the reward function, tilting outputs toward return-predictive patterns learned during pretraining. Crucially, bias can arise even without explicit instruction; a conversational exchange that merely hints at the downstream task is enough for the model to infer the objective and adjust its scores accordingly.

Further analysis

Replicating the experiment with GPT-5 provides direct evidence of temporal leakage through reasoning traces. When goal-aware, the model explicitly conditioned its April 2023 sentiment score for First Republic Bank on its subsequent seizure and failure on May 1, 2023, arriving at a negative sentiment of -0.55. Under goal-blind conditions, the model confined its reasoning to the transcript content, producing a modestly positive score of 0.3. This confirms that goal-aware prompting induces the model to incorporate post-period outcomes to reverse-engineer scores consistent with the downstream objective. Attempts at prompt-based regularisation, such as minimax optimisation, were found to attenuate the bias by roughly 45% but could not eliminate it.

Conclusion

Large language models are not neutral processors of information. When downstream use is disclosed, the model stops functioning as a measuring device and begins functioning as an optimiser that trades input fidelity for alignment with evaluation criteria. This results in stronger in-sample performance but weaker out-of-sample generalisation. The research recommends that AI-assisted measurement requires strict separation between measurement and evaluation. Accountability for resulting distortions rests with system design and how objectives are conveyed and constrained, rather than with the algorithm alone.