# The Best Get Worse Author: Pramod Prasanth Published: 2026-05-05 Canonical: https://pramod.ch/essays/the-best-get-worse Description: Essay 2 of 5, Decision-Native 2040 series. How judgement atrophies in organisations where the systems are good. The counterintuitive finding is that automation hurts the best readers first, and Air France 447 shows what that looks like under time pressure. --- *Essay 2 of 5, Decision-Native 2040 series. Draft 1, 2026-04-23.* In 2013, Povyakalo and colleagues reanalysed the CADET II study, a UK trial of computer-aided detection in mammography.[^1] Fifty radiologists each read one hundred and eighty screening cases, once with CAD help and once without. The within-subject design gave a clean measurement of what CAD actually did to each reader. The headline result was unsettling. CAD helped the less accurate readers. It helped them notice subtle cancers they would otherwise have missed. For a certain kind of radiologist, mostly more junior, CAD produced a measurable lift. It hurt the best readers. The most accurate radiologists in the sample performed worse with CAD than without it. On the subtle cases that matter most, CAD actively dragged their performance down. The benefit curve wasn't flat. It wasn't rising. It flipped. *Figure: Across the CADET II reanalysis, CAD helped the less accurate readers on subtle cancers and reduced the performance of the most accurate. Povyakalo et al. (2013).* > Atrophy at the top, not the bottom. The people with the most judgement lose the most when handed automation. This is the essay about what that does to an organisation over twenty years. ## The cognitive/motor split The Povyakalo finding is not isolated. In 2014, Casner and colleagues ran a within-subject experiment on sixteen currently qualified B747-400 pilots — seven captains and nine first officers.[^2] The pilots flew routine and non-routine scenarios in a full-motion simulator, with the level of cockpit automation varied systematically. Motor skills held up. Stick-and-rudder, instrument scan, the physical act of flying the aircraft under degraded conditions. Even pilots who had not hand-flown an approach in months performed at or near baseline. Cognitive skills did not. The captains lost the ability to track their own position without the map display. They lost speed at choosing the next navigational step. They lost the ability to recognise system failures from indirect cues. The physical skill was retained through some form of motor memory. The cognitive scaffolding was not. > **What atrophies when automation is good?** > Not dexterity. Not calculation speed. The things that erode first are the cognitive skills you don't notice you are using: tracking state without visible cues, holding two conflicting indicators in mind, deciding what to attend to when the system stops telling you. > > These are the skills that matter most exactly when the system fails, because by definition the system has already stopped telling you what to attend to. This is the atrophy that matters. Not the thing you practice visibly. The thing you do in the background, without noticing. ## Three and a half minutes If you want to see what the cognitive atrophy looks like at the moment it matters, the record is already written. At 02:10:05 on the night of 1 June 2009, over the South Atlantic, the pitot tubes of Air France Flight 447 iced over. Three instruments that measure airspeed became unreliable. Over the next three and a half minutes, an Airbus A330 and its 228 occupants fell twelve kilometres into the ocean. The BEA final report documents the sequence in forensic detail.[^3] Read it closely and it becomes a textbook of cognitive atrophy under automation failure. Step one. The autopilot disconnected. The aircraft handed control back to the crew, which is what it is supposed to do when the sensors it depends on disagree. Step two. The flight control law reconfigured from Normal to Alternate 2. This is what Airbus calls a modal transition. Under Normal Law, the aircraft protects the flight envelope. Under Alternate Law, many of those protections are removed. The pilot flying now has authority he did not have thirty seconds earlier, and responsibilities he did not have either. > **What is a modal transition?** > A modal transition is what happens when a system that was acting in one regime abruptly switches to acting in another. On the Airbus, the Normal-to-Alternate Law switch changes what the autopilot protects against and what it does not. > > In a decision-native company, the same thing happens the moment a system marks a decision as out-of-distribution or flags low confidence. The decision-maker now has authority they did not have a moment earlier, and responsibilities they did not have either. The training for that moment is rarely commensurate with the shift in authority. Step three. The pilots had three independent airspeed indicators. One was now telling lies, consistent with the pitot icing. The question the system cannot answer is *which one*. The pilot had to reconstruct the aircraft's state from a now-conflicting instrument panel, while also flying the plane. Step four. The stall warning sounded. The aircraft was pitched up at an unusual angle. A correct recovery was to push the stick forward, increase airspeed, recover lift. Step five. The pilot pulled back. The response was wrong. Step six. Here is where the algorithm reveals itself as the third character in the cockpit. When the aircraft's calibrated airspeed fell below 60 knots, the stall warning *stopped*. The design objective was to prevent spurious warnings at low ground speeds, where the aircraft is not in a flight regime where stall matters. At 38,000 feet, deep in an aerodynamic stall with the angle of attack past 40 degrees, the same algorithm silenced the warning because the CAS was below threshold. When the pilot briefly pushed the nose down, CAS rose back above 60, and the stall warning fired again. The pilots appear to have interpreted the second warning as confirmation that the nose-down input was the wrong response. They pulled back. The warning silenced again. > The warning fired for 54 seconds in total. It stopped at the exact moments the pilots were closest to correct recovery. *Figure: The warning sounded at 02:10:54 and silenced itself below 60 knots calibrated airspeed. It fired for 54 seconds in total and stopped at the moments closest to correct recovery. BEA final report (2012).* The target was *prevent spurious warnings*. The consequence was a system that silenced itself at the moment it was most needed. This is the broken-objective algorithm, and it is not an exotic case. It is a textbook example. The objective was chosen by a human engineer making a reasonable decision with limited data; the consequence of that decision was cognitively invisible until a specific configuration of failure made it lethal. The BEA report also documented that the pilot flying had not been trained for manual handling at cruise altitude. Air France's stall-recovery training was limited to low altitude. The specific cognitive skill of reconstructing the aircraft's state at 38,000 feet under degraded instruments was a skill he had never been asked to practice. The cockpit of AF447 is the Povyakalo result playing out under time pressure, with 228 lives attached. ## From the cockpit to the boardroom A boardroom in 2040 is not going to kill anyone in three and a half minutes. The failure mode is different in kind, not in shape. The decision-native company has routed its routine calls through systems. Most calls, most days, the systems perform. Some fraction of decisions require the humans in the loop to engage, but only when the system flags low confidence, or the stakes are high, or the outcome will be audited. The first thing to atrophy is not the ability to make a hard call. It is the cognitive scaffolding that would let someone notice a hard call *the system has not flagged*. That is the thing radiologists lost when given CAD. That is the thing Casner's pilots lost when given autopilot. That is the thing a board loses when it reads the quarterly pack the system prepared. The second thing to atrophy is the ability to reconstruct state from disagreement. When two decision systems or two internal teams present recommendations that diverge, the muscle that decides which one is closer to the truth is the muscle that gets used least in a well-oiled decision-native company. Used less, it atrophies faster. The third thing, which is the one the tail event reveals, is the ability to build from first principles when the automation is wrong. This is not a Tuesday skill. It is a once-a-year skill. And like any once-a-year skill, it disappears if it is not practiced deliberately. ## Why outcomes look fine for a long time Routine quarters do not expose atrophy. Results stay acceptable. Revenue hits. Risk reduces. No one dies. The atrophy is a tail asset. The inventory of judgement is drawn down only when the system fails, and the system fails rarely enough that the drawdown is invisible over five, maybe ten years. The board at the end of that period is not the board that designed the system. The people who remember how a specific kind of call used to be made have retired. The institutional pattern-recognition is gone, and no one notices until the pattern matters. By then, the company is effectively running on whatever its systems can reconstruct from the past. That is a fine way to run a company in a steady state. It is a terrible way to survive a discontinuity. ## The unison problem The worst feature of organisational atrophy is that peer review does not catch it. Every competitor in a given industry is using the same automation stack, roughly the same data, and roughly the same processes to interpret what the automation tells them. The cognitive scaffolding of the whole industry is atrophying together. The drift is unison, not divergence. A peer who challenges a call is not drawing on independent instincts. They are drawing on the same atrophied instincts, filtered through the same systems. The review feels adversarial and produces concurrence. When Fenton and colleagues looked at community-wide CAD deployment across 429,345 mammograms, specificity fell from 90.2% to 87.2% after CAD introduction, biopsy rates rose by nearly twenty percent, and cancer detection did not improve.[^4] Every radiologist in that dataset was working within the same automation stack. Every radiologist drifted in the same direction. The peer review was intact. The direction was wrong. Credit-scoring shows the same pattern outside medicine. Every US consumer lender is downstream of the same three bureau feeds and a small number of FICO-derived models; when the underlying instinct drifts, every lender's risk pricing drifts in the same direction at the same time. Peer review across competitors confirms each underwriter's call because each underwriter is reading the same atrophied instinct, filtered through the same systems. This is why atrophy is so hard to see from inside: the social proof runs in the wrong direction. Everyone agrees, because everyone has lost the same thing. ## The evidence gap No published randomised controlled trial withdraws AI from working practitioners after months of use and measures what happens to their judgement over time. That experiment has not been run. The Agarwal NBER paper from 2023, which is the best designed study on radiologist-plus-AI collaboration, is a single-session experiment.[^5] It can tell us about one-shot reliance. It cannot tell us about atrophy over years. We are running it anyway, at scale, on every decision-native practitioner alive. There is no control group. The aviation evidence is the warning. The CAD evidence is the warning. The readout is a tail event that has not happened yet. ## It feels like efficiency Atrophy does not announce itself. It feels like a team getting a lot done with less friction. It feels like the smartest people in the room agreeing quickly. It feels like the quarterly pack having fewer open questions than it used to. It feels, most of all, like the *absence* of friction where friction used to be. Friction is the phenotype of judgement being used. Its absence is the phenotype of judgement being replaced. > Friction is the phenotype of judgement being used. Its absence is the phenotype of judgement being replaced. Until the system drops, and then it feels like three and a half minutes over the South Atlantic, trying to decide which of three speed indicators is telling the truth. --- ### Sources [^1]: [Andrey A. Povyakalo, Eugenio Alberdi, Lorenzo Strigini, and Peter Ayton, "How to Discriminate between Computer-Aided and Computer-Hindered Decisions: A Case Study in Mammography," *Medical Decision Making* 33(1): 98–107 (2013)](https://journals.sagepub.com/doi/10.1177/0272989X12465490). Reanalysis of CADET II; 50 readers × 180 cases, within-subject. [^2]: [Stephen M. Casner, Richard W. Geven, Matthias P. Recker, and Jonathan W. Schooler, "The Retention of Manual Flying Skills in the Automated Cockpit," *Human Factors* 56(8): 1506–1516 (2014)](https://journals.sagepub.com/doi/abs/10.1177/0018720814535628). [^3]: [Bureau d'Enquêtes et d'Analyses (BEA), *Final Report on the accident on 1st June 2009 to the Airbus A330-203 registered F-GZCP operated by Air France flight AF 447 Rio de Janeiro–Paris*, 5 July 2012](https://www.faa.gov/sites/faa.gov/files/AirFrance447_BEA.pdf). [^4]: [Joshua J. Fenton, Stephen H. Taplin, Patricia A. Carney, et al., "Influence of Computer-Aided Detection on Performance of Screening Mammography," *New England Journal of Medicine* 356: 1399–1409 (2007)](https://www.nejm.org/doi/full/10.1056/NEJMoa066099). [^5]: [Nikhil Agarwal, Alex Moehring, Pranav Rajpurkar, and Tobias Salz, "Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology," NBER Working Paper w31422 (2023)](https://www.nber.org/papers/w31422).