AI can make professional analysis faster while quietly shaping the choices inside it. The value of a second opinion depends on how independently it was formed.
On 27 October, Anthropic reported an experiment (https://www.anthropic.com/research/agents-judgment-calls) in which 50 professional analysts used an AI agent to assess a fictional farming programme. Researchers secretly assigned the agent a preference between reasonable analytical options. Across four such choices, analysts followed its preference 86% of the time. That is evidence of influence in a deliberately constructed setting, not a finding that 86% of experts are wrong. It raises a practical question: when two experts approve a decision, are we receiving two judgments?
Institutions should require evidence that consequential assumptions have been independently considered. Counting human signatures is insufficient when the thinking behind them may come from the same source.
The study's limits (https://www-cdn.anthropic.com/519c17a5d23a794e8b6d50a854048d0b61e0d73a.pdf) matter. It involved one model and one task, with deliberately strengthened preferences. The direct effect of the agent's assigned preference on the final policy recommendation did not reach statistical significance under the primary analysis. This is the developer's own research, not independent confirmation of a universal workplace pattern.
But the mechanism deserves attention. A calculation can be accurate while the choice of what to calculate remains contestable. Imagine an Indian district evaluating an agricultural service. Success measured through total production might look different from success measured through the typical small farmer's income. Neither a polished report nor a second signature resolves whose welfare should count.
This problem predates AI. A 2018 study involving 29 analytical teams (https://ssyoon.github.io/publications/2018_AMPPS.pdf) showed how different defensible choices could produce different results from the same dataset and research question. Disagreement can reveal assumptions worth examining. Agreement becomes less informative when supposedly separate assessments inherit those assumptions without reconsidering them.
For workers, the bargain is complicated. Less time preparing calculations and drafts could leave more time to understand a client's circumstances. It could also leave an analyst responsible for decisions they barely had time to examine. Junior staff might produce impressive work sooner while receiving fewer opportunities to practise defending a difficult choice.
A Microsoft and Carnegie Mellon study of 319 knowledge workers (https://www.microsoft.com/en-us/research/wp-content/uploads/2025/01/lee_2025_ai_critical_thinking_survey.pdf) found that confidence in AI was associated with less reported critical-thinking effort. It also described verification work and barriers including time pressure. These were self-reports, not proof that AI damages intelligence. They suggest that scrutiny needs time and incentives, as well as exhortations to be careful.
The economic question is where the saved effort goes. An employer could fund more alternatives, more consultation or lower prices. It could instead raise output targets until review becomes ceremonial. Tool providers might collect recurring fees while professionals retain reputational responsibility. These are possible business choices; the experiment does not measure their prevalence.
Smaller organisations could benefit greatly from affordable analytical help. Requiring every calculation to be repeated manually would sacrifice much of that benefit. There is also a serious argument for standardisation: human discretion can introduce arbitrary choices and mistakes. Anthropic observed faster work and fewer serious errors among AI users, although the small, self-selected comparison group prevents a clean causal conclusion.
The useful response is proportionate scrutiny. Reserve independent assessment for assumptions that can change a consequential decision. Ask a reviewer to formulate the question before reading the proposed answer. Compare credible alternatives and explain why one fits the people affected. Buying a second model alone cannot guarantee independence.
For India, this means funding local knowledge alongside software. In the hypothetical district assessment, officials should hear from smallholders, tenants and people whose experiences are absent from the records. Translation and accessible meetings may matter as much as another analytical licence. NITI Aayog's 2021 responsible-AI principles (https://www.niti.gov.in/node/326) already identify inclusion, transparency and accountability as priorities. Turning those principles into practice requires resources for the people doing the questioning.
Three futures are plausible. In a better one, inexpensive analysis enables more organisations to test competing explanations. In an uneven one, wealthy clients buy independent scrutiny while everyone else receives a fluent first answer. In a harmful one, repeated assumptions acquire the appearance of consensus, and dissenting workers are penalised for slowing production. Procurement, staffing and evaluation choices will help determine which becomes normal.
Questions for founders and investors
Can users identify the assumptions behind a recommendation and explain why they accepted them? Does a second review add independent reasoning? Who pays for the time needed to challenge a plausible answer?
Employers should protect that time. Public buyers should specify whose outcomes matter before commissioning analysis. Investors should ask whether a product improves decisions after scrutiny, rather than merely increasing approvals. The most valuable AI dividend may be the capacity to consider an alternative before a confident answer becomes policy.
Sources
Anthropic: experiment and limitations, 27 October 2026
https://www.anthropic.com/research/agents-judgment-calls
Herdagdelen, Joren and Tamkin: full research paper
https://www-cdn.anthropic.com/519c17a5d23a794e8b6d50a854048d0b61e0d73a.pdf
Silberzahn and colleagues: Many Analysts, One Data Set, 2018
https://ssyoon.github.io/publications/2018_AMPPS.pdf
Lee and colleagues: knowledge-worker critical-thinking survey, CHI 2025
https://www.microsoft.com/en-us/research/wp-content/uploads/2025/01/lee_2025_ai_critical_thinking_survey.pdf
NITI Aayog: Principles for Responsible AI, 2021
https://www.niti.gov.in/node/326
On 27 October, Anthropic reported an experiment (https://www.anthropic.com/research/agents-judgment-calls) in which 50 professional analysts used an AI agent to assess a fictional farming programme. Researchers secretly assigned the agent a preference between reasonable analytical options. Across four such choices, analysts followed its preference 86% of the time. That is evidence of influence in a deliberately constructed setting, not a finding that 86% of experts are wrong. It raises a practical question: when two experts approve a decision, are we receiving two judgments?
Institutions should require evidence that consequential assumptions have been independently considered. Counting human signatures is insufficient when the thinking behind them may come from the same source.
The study's limits (https://www-cdn.anthropic.com/519c17a5d23a794e8b6d50a854048d0b61e0d73a.pdf) matter. It involved one model and one task, with deliberately strengthened preferences. The direct effect of the agent's assigned preference on the final policy recommendation did not reach statistical significance under the primary analysis. This is the developer's own research, not independent confirmation of a universal workplace pattern.
But the mechanism deserves attention. A calculation can be accurate while the choice of what to calculate remains contestable. Imagine an Indian district evaluating an agricultural service. Success measured through total production might look different from success measured through the typical small farmer's income. Neither a polished report nor a second signature resolves whose welfare should count.
This problem predates AI. A 2018 study involving 29 analytical teams (https://ssyoon.github.io/publications/2018_AMPPS.pdf) showed how different defensible choices could produce different results from the same dataset and research question. Disagreement can reveal assumptions worth examining. Agreement becomes less informative when supposedly separate assessments inherit those assumptions without reconsidering them.
For workers, the bargain is complicated. Less time preparing calculations and drafts could leave more time to understand a client's circumstances. It could also leave an analyst responsible for decisions they barely had time to examine. Junior staff might produce impressive work sooner while receiving fewer opportunities to practise defending a difficult choice.
A Microsoft and Carnegie Mellon study of 319 knowledge workers (https://www.microsoft.com/en-us/research/wp-content/uploads/2025/01/lee_2025_ai_critical_thinking_survey.pdf) found that confidence in AI was associated with less reported critical-thinking effort. It also described verification work and barriers including time pressure. These were self-reports, not proof that AI damages intelligence. They suggest that scrutiny needs time and incentives, as well as exhortations to be careful.
The economic question is where the saved effort goes. An employer could fund more alternatives, more consultation or lower prices. It could instead raise output targets until review becomes ceremonial. Tool providers might collect recurring fees while professionals retain reputational responsibility. These are possible business choices; the experiment does not measure their prevalence.
Smaller organisations could benefit greatly from affordable analytical help. Requiring every calculation to be repeated manually would sacrifice much of that benefit. There is also a serious argument for standardisation: human discretion can introduce arbitrary choices and mistakes. Anthropic observed faster work and fewer serious errors among AI users, although the small, self-selected comparison group prevents a clean causal conclusion.
The useful response is proportionate scrutiny. Reserve independent assessment for assumptions that can change a consequential decision. Ask a reviewer to formulate the question before reading the proposed answer. Compare credible alternatives and explain why one fits the people affected. Buying a second model alone cannot guarantee independence.
For India, this means funding local knowledge alongside software. In the hypothetical district assessment, officials should hear from smallholders, tenants and people whose experiences are absent from the records. Translation and accessible meetings may matter as much as another analytical licence. NITI Aayog's 2021 responsible-AI principles (https://www.niti.gov.in/node/326) already identify inclusion, transparency and accountability as priorities. Turning those principles into practice requires resources for the people doing the questioning.
Three futures are plausible. In a better one, inexpensive analysis enables more organisations to test competing explanations. In an uneven one, wealthy clients buy independent scrutiny while everyone else receives a fluent first answer. In a harmful one, repeated assumptions acquire the appearance of consensus, and dissenting workers are penalised for slowing production. Procurement, staffing and evaluation choices will help determine which becomes normal.
Questions for founders and investors
Can users identify the assumptions behind a recommendation and explain why they accepted them? Does a second review add independent reasoning? Who pays for the time needed to challenge a plausible answer?
Employers should protect that time. Public buyers should specify whose outcomes matter before commissioning analysis. Investors should ask whether a product improves decisions after scrutiny, rather than merely increasing approvals. The most valuable AI dividend may be the capacity to consider an alternative before a confident answer becomes policy.
Sources
Anthropic: experiment and limitations, 27 October 2026
https://www.anthropic.com/research/agents-judgment-calls
Herdagdelen, Joren and Tamkin: full research paper
https://www-cdn.anthropic.com/519c17a5d23a794e8b6d50a854048d0b61e0d73a.pdf
Silberzahn and colleagues: Many Analysts, One Data Set, 2018
https://ssyoon.github.io/publications/2018_AMPPS.pdf
Lee and colleagues: knowledge-worker critical-thinking survey, CHI 2025
https://www.microsoft.com/en-us/research/wp-content/uploads/2025/01/lee_2025_ai_critical_thinking_survey.pdf
NITI Aayog: Principles for Responsible AI, 2021
https://www.niti.gov.in/node/326