# The Honest Counterargument Author: Pramod Prasanth Published: 2026-05-05 Canonical: https://pramod.ch/essays/the-honest-counterargument Description: Essay 3 of 5, Decision-Native 2040 series. Most decisions should be automated by 2040. The honest case for that, and what it concedes without meaning to. --- *Essay 3 of 5, Decision-Native 2040 series. Draft 1, 2026-04-23.* The previous essay showed automation hurting the best practitioners first. This essay is the honest case for automating most decisions anyway, and what the case quietly concedes. In 1954, Paul Meehl showed that simple actuarial formulas beat experienced clinicians at predicting who would attempt suicide, who would respond to treatment, who would succeed in graduate school.[^1] A 2000 meta-analysis by Grove and colleagues covered 136 studies across medicine, mental health, academic prediction, and forensics.[^2] The finding held. On average, mechanical prediction was about ten percent more accurate than clinical judgement. In a third of comparisons, the algorithm substantially beat the experts. In about a tenth, the experts substantially beat the algorithm. Most comparisons tied. *Figure: Across Grove and colleagues' 136-study meta-analysis, mechanical prediction substantially beat clinical judgement in a third to a half of comparisons, clinicians substantially beat mechanical prediction in about one in ten, and the rest tied.* That last detail is the one to keep. > **What is a decent baseline?** > Not the best possible answer. The baseline is the consistent, defensible recommendation the system produces for a decision of this kind, drawn from the record of comparable cases before it. The average of the judgements the room would have produced, hardened into a default. > > That shifts the burden. Overriding the recommendation requires a reason. Accepting it does not. Algorithms tie or win routinely enough that by 2040, every human decision will have to justify itself against a decent baseline. Dominance is not required for that. For most decisions, the cost of producing a reason that beats the default is higher than the value of the override. Most humans will not produce the reason. Most will be right not to. The threshold for this effect is lower than it sounds. The baseline does not need to be better than human judgement. It only needs to be good enough that pausing to form an independent view feels unnecessary. > The baseline does not need to be better than human judgement. It only needs to be good enough that pausing to form an independent view feels unnecessary. This essay is about what happens next. ## The case for automating almost everything The current evidence doesn't just cover clinical prediction. In credit, Upstart's model produced 27% more approvals at 16% lower APRs than the FICO counterfactual, at equivalent default risk.[^3] In judicial bail, Kleinberg and colleagues showed algorithmic prediction could cut crime by up to 24.7% at unchanged jail populations, or cut jail populations by up to 41.9% at unchanged crime.[^4] In hiring, Cowgill's work on algorithmic resume screening produced 14% more interviews granted to candidates who later performed well.[^5] In retinopathy, Gulshan and colleagues reached sensitivity and specificity in the 90%-plus range against a panel-graded reference.[^6] In dermatology, Esteva's convolutional network matched 21 board-certified dermatologists on melanoma classification.[^7] None of these results are perfect. None of them replicate cleanly across every deployment context. Dressel and Farid showed that COMPAS, a 137-variable commercial algorithm for recidivism prediction, was no more accurate than untrained Mechanical Turk workers given seven features.[^8] Obermeyer's 2019 paper in *Science* documented an algorithm guiding care for roughly 100 million US patients that used cost as a proxy for need. Correcting the target variable raised the share of Black patients receiving additional help from 17.7% to 46.5%.[^9] The model worked. The target was wrong. Those failures are real. They argue for paying attention to what target the algorithm is optimising, not for refusing automation altogether. By 2040, after fifteen more years of auditing and deployment discipline, most of the broken-objective and brittle-complexity failures will have been engineered out of production. The ones that remain will be exceptional, not baseline. And then there is novelty. I used to think novelty was the last preserved zone. RFdiffusion, from the Baker Lab at the University of Washington, is producing de novo proteins with no natural analog. The all-atom variants now bind small molecules like digoxigenin.[^10] These are arguably new categories of molecule, generated by a model. They are not a trick of training distribution. They are an existence proof that constrained recombination, inside a material feedback loop with human selection pressure, can produce things that did not exist before. If even the novelty frontier is contestable, the preserved zone for human judgement in 2040 is very narrow. ## What the argument misses So the case is strong. Granted. Before I look at what it leaves out, I have to say what happened to me last week. I asked an AI agent to do a reconnaissance across a design document: read every page, inventory the shared components, pull the code snippets. The agent reported back that it had done the recon. It had used an updated MCP server. It listed what it had found. I accepted the report. I accepted the code it generated on top of that recon. The output was off. When I looked into why, I discovered the agent had read perhaps two pages. The rest was hallucination, dressed as the kind of structured summary a thorough recon would produce. The report's legibility was what I had acted on. Its truthfulness was absent. I had a pause available to me that I did not take. Three years ago, if working with a remote outsourced team, I would have asked the team to quote three specific things from page seven. Last week I did not. The agent's statement of capability was coherent enough that pausing to verify felt unnecessary. The threshold for my own atrophy was lower than *the system is better than me*. It was *the system's statement of what it did is coherent enough that I accepted it*. Now look at what the case for automation leaves out. What transparency does to the humans still in the loop is the real shift in 2040. Capability is the minor story. When every decision is structured, logged, and justified, three things happen. First, corrupt decisions get harder to hide. A recommendation the system flags, a deviation the system records, a reason the decision-maker has to name: these make certain kinds of malfeasance traceable in ways they were not before. Back-channel allocations, favours dressed as merit, politically-incorrect calls made without ownership. Most of that shrinks. Good. Second, unconventional decisions get harder to make. The ones where your reason is illegible even to yourself. The hunch. The taste call. The loyalty call. The system of legibility has no rubric for "I don't know why, but I think we should do this." You cannot log it. You cannot justify it. If your decision diverges from the system's recommendation, you have to produce a reason, and the reason has to be of a kind the system recognises. Third, over time, people stop making decisions they cannot defend. Not because anyone forbids them. Because defending them is exhausting, and the rewards are captured by people who move faster and produce fewer unjustifiable calls. The selection pressure is subtle, and it runs in one direction. What disappears, slowly, is the class of decisions that build new categories. Those are always indefensible in advance. The first person who decided to use a fishing hook to pull thread through fabric was not starting from a consensus view that this was a reasonable thing to do. The first accountant to write credits and debits on the same page was not resolving a known problem with a legible method. New categories are made by people doing things for reasons they cannot yet say out loud. > New categories are made by people doing things for reasons they cannot yet say out loud. Legibility is incompatible with that. ## The narrative becomes cheap "But people can just justify unconventional decisions," someone will say. "Language models will write the 47 bullet points. Defence is free." Yes. That is the problem. In 2040, any decision can be dressed in legible reasoning in seconds. Context, constraints, counterfactuals, risk bands, three comparable precedents, a concluding paragraph. Every decision looks defended. Legibility is preserved. The *truthfulness* of legibility is not. > Legibility is preserved. The truthfulness of legibility is not. The signal-to-noise ratio of defences collapses. A person with a hunch and a person with a post-hoc rationalisation produce indistinguishable documents, and the system that aggregates those documents has no reliable way to tell them apart. Whatever legibility was doing for accountability is now gone. Everyone passes the audit. Nobody has been audited. The burden of judgement shifts from *what is the right decision* to something harder: *among the abundant manufactured defences, which signal do I act on*. That is curation. ## Curation gets gamed There is a warehouse inside the Geneva Freeport, a bonded facility outside national tax jurisdiction, that holds roughly 1.2 million artworks. Somewhere near a thousand of them are Picassos. The total value is estimated around one hundred billion dollars.[^11] The estimate has not been audited. It does not need to be. The point of the warehouse is not display. The warehouse is not the story. The curation around it is. A team of real connoisseurs, selecting real Old Masters and blue-chip contemporary work. In its narrow function, the judgement is correct. These pieces will hold value. That judgement is real. In its broader claim, the curation is hollow. The works are not chosen to be seen. They are chosen to be stored. The curators are performing a judgement-shaped service whose actual function is financial. The Nahmad family's 4,500 works, including about 300 Picassos, sit in a 15,000-square-foot vault in the same freeport. In 2016 the Panama Papers exposed that Modigliani's *Seated Man with a Cane*, allegedly looted from Oscar Stettiner by the Nazis, had been held in that vault under the name of a Panamanian shell company controlled by David Nahmad. It took until 2026 for the painting to be formally restituted.[^12] Museums display twenty to thirty percent of their collections. The rest is stored for conservation.[^13] The 1.2 million works in the Geneva Freeport are stored for the opposite reason. They are meant to be unseen. The curation exists. It serves the balance sheet, not the viewer. > The curation exists. It serves the balance sheet, not the viewer. This is what happens when the burden of judgement shifts to curation without anything preventing the shift from being repurposed. The curators are still curators. They still have taste. Their taste is being spent on something other than what taste is for. The same dynamic transfers to a decision-native organisation, and for the same reasons. When curation is where the load has shifted, whoever controls curation eventually captures it for their own ends, unless the criteria themselves are visible to outside review. The curators at Geneva answered to their clients alone. The criteria by which a work was selected for storage rather than display were held inside the team, never made visible to anyone outside. Curation with inspectable criteria is a different system. The divergence between stated purpose and actual purpose shows up earlier, at the criterion level, rather than compounding silently for years. Neither version eliminates the capture risk. One just lets it run undetected for longer. ## What is actually preserved By 2040, most things that look like judgement will be judgement-shaped. They will produce outputs indistinguishable from judgement from the outside. A small fraction will actually be doing what judgement used to do: decisions made because someone is trying to build something new, protect something the system cannot see, or preserve a kind of variance the organisation will need later without knowing it yet. There is no reliable external test to tell which is which. "Illegibility protected on purpose" sounds like the criterion. It is not enough. The Geneva warehouse meets that bar; so do a thousand other rackets. The preserved zone needs a second condition: *the illegibility has to serve invention, not investment, status, or political cover*. That zone is small. It is fragile. It can be replaced with its performance, and most people will not notice. From the outside, a genuine unconventional decision and a laundered unconventional decision look the same. The person making each writes the same defence document. The system produces the same approval record. The outcomes may even come out similar for a long time. The difference shows over a generation. Legible systems compound toward convergence. Homogenised decision-making, averaged across a cohort, produces outputs well-adapted to today's environment and poorly adapted to whatever is coming. The few decisions that break the average, that keep some variance in the system, are what let an organisation survive a shift it did not model. ## It does not resolve The rest of this series follows people trying to stay inside the small fraction that is actually doing what judgement used to do, knowing they might already be outside it without knowing. I am not going to argue anyone out of automating most decisions. Most decisions should be automated. By 2040 they will be. The cost of not automating them will be obvious, and the defenders of human judgement at scale will mostly be selling nostalgia. But there is a narrow zone where the defence holds: what keeps an organisation alive is whatever its best people still do in the register that cannot be audited. That register is hard to protect. It is easy to lose without noticing. Once it is gone, it is not obvious what replaces it. That is the honest counterargument. It concedes the ground. It does not win. > It concedes the ground. It does not win. What remains in the ground it gives up is not a principle. It is a practice. Someone, somewhere in the organisation, doing a piece of cognitive work the system has stopped requiring, for reasons they cannot easily name, at mild cost to themselves. The next essay, *The Thing She Cannot Name*, is about one of those people. --- ### Sources [^1]: [Paul E. Meehl, *Clinical versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence* (Minneapolis: University of Minnesota Press, 1954)](https://doi.org/10.1037/11281-000). The foundational monograph that prompted decades of follow-up work; synthesised the actuarial-versus-clinical prediction literature available at the time. Reprinted by Jason Aronson (1996) and Echo Point Books (2013). [^2]: [William M. Grove, David H. Zald, Boyd S. Lebow, Beth E. Snitz, and Chad Nelson, "Clinical versus Mechanical Prediction: A Meta-Analysis," *Psychological Assessment* 12(1): 19–30 (2000)](https://pubmed.ncbi.nlm.nih.gov/10752360/). Synthesises 136 comparative studies across medicine, mental health, academic prediction, and forensics. Mechanical prediction substantially beat clinical judgement in 33–47% of comparisons; clinicians substantially beat mechanical prediction in 6–16%. The remainder were ties. [^3]: [Consumer Financial Protection Bureau and Upstart reporting to regulators on the no-action letter programme (2017–2022)](https://www.federalreserve.gov/SECRS/2022/October/20221028/OP-1743/OP-1743_070121_138216_351653676425_1.pdf), documenting approval and pricing differences against a FICO-based counterfactual. [^4]: [Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan, "Human Decisions and Machine Predictions," *Quarterly Journal of Economics* 133(1): 237–293 (2018)](https://academic.oup.com/qje/article-abstract/133/1/237/4095198). [^5]: [Bo Cowgill, "Bias and Productivity in Humans and Algorithms: Theory and Evidence from Resume Screening," Columbia Business School working paper (2018)](https://conference.iza.org/conference_files/MacroEcon_2017/cowgill_b8981.pdf). [^6]: [Varun Gulshan et al., "Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs," *JAMA* 316(22): 2402–2410 (2016)](https://jamanetwork.com/journals/jama/fullarticle/2588763). [^7]: [Andre Esteva et al., "Dermatologist-level classification of skin cancer with deep neural networks," *Nature* 542: 115–118 (2017)](https://www.nature.com/articles/nature21056). [^8]: [Julia Dressel and Hany Farid, "The accuracy, fairness, and limits of predicting recidivism," *Science Advances* 4(1): eaao5580 (2018)](https://www.science.org/doi/10.1126/sciadv.aao5580). [^9]: [Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan, "Dissecting racial bias in an algorithm used to manage the health of populations," *Science* 366(6464): 447–453 (2019)](https://www.science.org/doi/10.1126/science.aax2342). [^10]: [Joseph L. Watson et al., "De novo design of protein structure and function with RFdiffusion," *Nature* 620: 1089–1100 (2023)](https://www.nature.com/articles/s41586-023-06415-8). See also the [Baker Lab summary](https://www.bakerlab.org/2023/07/11/diffusion-model-for-protein-design/). [^11]: ["Über-warehouses for the ultra-rich," *The Economist*, 23 November 2013](https://www.economist.com/briefing/2013/11/23/uber-warehouses-for-the-ultra-rich). Origin of the 1.2 million works / ~US$100 billion estimate for the Geneva Freeport, still widely cited though never independently audited. Background: [Geneva Freeport on Wikipedia](https://en.wikipedia.org/wiki/Geneva_Freeport). [^12]: [ICIJ, "Contested Modigliani seized from freeport as Swiss open investigation," 11 April 2016](https://www.icij.org/investigations/panama-papers/swiss-art-freeport-search/). See also [ICIJ Pandora Papers, "From Banksy to Picasso, offshore world awash in valuable art"](https://www.icij.org/investigations/pandora-papers/from-banksy-to-picasso-offshore-world-awash-in-valuable-art/). [^13]: [ICOM Working Group on Collections in Storage, *Report on collections in storage* (2024)](https://icom.museum/wp-content/uploads/2024/05/Report_ICOM-STORAGE_EN_Final.pdf). Museum-collection display fractions vary by institution; 20–30% is the representative range.