Gwern essay argues breakthroughs are a matter of conviction, not competence

An essay titled "On Really Trying", hosted on gwern.net, argues that the real limit on effective problem-solving effort is psychological rather than a matter of raw competence: people work harder and more effectively when they are convinced a correct answer exists and that finding it is urgent, and they stall when they merely suspect a better answer might be out there.

The essay opens with a Zen teacher, Yasutani, telling a student that realizing one's Self-nature requires conviction first: without it, doubt ("maybe I can get it, maybe I can't") blocks awakening regardless of how much practice follows. The author connects this to Eliezer Yudkowsky's fictional "beisutsukai" stories, in which a teacher, Jeffreyssai, asks his student Brennan why physicists spent over 30 years failing to find the successor to quantum mechanics, despite Einstein, Schrodinger and von Neumann all working in the field. Brennan answers that the Manhattan Project succeeded quickly because it had a specific end in sight, a weapon, in wartime, whereas Eld-era physicists had no comparable urgency: their scientific culture treated eventually replacing a wrong theory as acceptable, so "work expands to fill the time allotted." In the same fictional universe, Jeffreyssai assigns five students one month to independently derive a correct theory of quantum gravity, telling them that simply knowing an answer exists and being asked the right question is itself a tremendous help.

The essay's author extends this into a personal claim: if the historical greats of physics had been handed the necessary experimental data and given only a few months to produce a new theory, under threat of death, they could have matched the fictional students' output. The argument is that this is entirely about motivation, not about competence at paradigm shifts, even though there is no normal way to manufacture that urgency inside an actual scientific community, where most proposed paradigm shifts fail to pan out.

The author backs the claim with real-world cases. On Kaggle, a long-stagnant leaderboard often sees several independent teams reproduce the same breakthrough shortly after one team demonstrates it works, without any of the follower teams knowing how it was done, suggesting the missing ingredient was belief that a large gain was available at all, not the technique itself. Manuel Blum's "Advice to a Beginning Graduate Student" recounts Claude Shannon being stuck on a childhood jigsaw puzzle until his brother said only "You know: I could tell you something," a hint with no content beyond confirming a solution existed, which was enough for Shannon to solve it himself.

In a 1992 interview on the development of public-key cryptography, Whitfield Diffie describes Ralph Merkle independently pursuing the problem of secure communication between strangers, while Diffie himself had concluded it was impossible and even persuaded a colleague, Peter Blatman, of that. Merkle's own paper, submitted first as a class assignment his professor could not follow and rejected, eventually appeared in print as "Secure communication over insecure channels" about five years later. Once Diffie and Martin Hellman's own preprint began circulating, passed to Merkle via Blatman, Merkle recognized people who understood his work and made contact, tying the separate threads together.

A parallel case comes from protein structure prediction: after DeepMind demonstrated that its methods could solve the problem, David Baker and postdoctoral fellow Minkyung Baek saw an opening despite not having DeepMind's code, because, in the words of CASP organizer John Moult, Baker treated DeepMind's result as "an existence proof" that such methods work, which he found sufficient reason to proceed. George Dantzig supplies a similar, self-told story: arriving late to a Berkeley class taught by Jerzy Neyman, he copied two problems off the blackboard, assumed they were homework, and solved them; they turned out to be two long-unsolved problems in statistics, and Neyman later showed up at Dantzig's door about six weeks afterward, already having written an introduction for publication of Dantzig's solutions.

The essay closes with Bill Atkinson's work at Apple making on-screen windows overlap, a graphics trick involving what the essay calls "regions." Atkinson pursued it because he believed he had seen the same capability during a visit to Xerox PARC, when in fact PARC had never built it and was later amazed that Atkinson had. Atkinson is quoted describing the effect as the "empowering aspect of naivete": not knowing something couldn't be done was what enabled him to do it. The extracted text cuts off mid-sentence during this final anecdote, and the essay does not name its own author or give its own publication date; those identifiers are absent from the retrieved text.

Key facts

• The essay argues the real constraint on solving hard problems is psychological, conviction that an answer exists and urgency to find it, rather than raw competence.
• In Yudkowsky's fictional 'Class Project', a teacher gives five students one month to independently derive the theory of quantum gravity, arguing that knowing an answer exists is itself a major help.
• Whitfield Diffie recounts, in a 1992 interview, that he had concluded secure communication between strangers was impossible while Ralph Merkle was independently working on the same problem and eventually published 'Secure communication over insecure channels' about five years after starting.
• George Dantzig mistook two unsolved statistics problems on Jerzy Neyman's blackboard for homework, solved them, and about six weeks later Neyman arrived at his door having already written an introduction for their publication.
• Bill Atkinson built overlapping on-screen windows at Apple because he mistakenly believed Xerox PARC had already done it, calling the effect the 'empowering aspect of naivete'.

Why it matters

The essay's throughline is that the gating factor in landmark technical achievements, quantum gravity in fiction, public-key cryptography and protein-structure prediction in fact, was not missing skill but missing conviction that a solution was reachable on a short timeline. It uses David Baker's protein-structure work after DeepMind's result as its clearest real case: Baker treated a competitor's demonstrated success as, in John Moult's words, 'an existence proof', and that belief alone was enough to justify pursuing a method his lab did not have the code for.

Who it affects

The piece speaks to anyone doing research or hard technical problem-solving, individually or in teams, and to how scientific and engineering communities structure deadlines and incentives. It draws its examples from cryptography (Diffie, Hellman, Merkle), statistics (Dantzig, Neyman), structural biology (Baker, Baek, Moult), computing history (Shannon, Atkinson) and competitive data science (Kaggle leaderboards).

How to use it

The essay offers no product, price or tool, only an argument and a set of anecdotes. Its practical suggestion, made through Yudkowsky's fictional teacher, is that a tight deadline and confirmation that a solution exists can substitute for the decades a field might otherwise assume a problem requires, and that a bare hint confirming a solvable answer exists, as with Claude Shannon's jigsaw puzzle, can be enough to unblock independent effort.

How solid is it

The case rests entirely on anecdote and analogy rather than controlled evidence: a philosophical quote, a work of fiction, one interview, one Kaggle observation, and stories about Dantzig and Atkinson recounted well after the fact. The essay itself does not claim experimental support, and the extracted text cuts off before its final anecdote (Atkinson's) concludes, so any argument made in the essay's closing section is not available here.

Risks and caveats

The essay does not name its own author in the retrieved text, nor does it give its own publication date; only the site (gwern.net) and title identify it. The claim that historical scientific greats could have matched fictional students given a hard deadline is explicitly the author's own speculation, not backed by a study, and the author elsewhere acknowledges that most real-world attempts at forcing a paradigm shift on a deadline fail to pan out.

“I got a feeling for the empowering aspect of naivete. Because I didn't know it couldn't be done, I was enabled to do it.”

— Bill Atkinson