Proof & Machine

Maple at Forty: The Algebra System Built to Fit in a Shared Terminal

2026-08-25T00:00:00.000Z

In November 1980, a group at the University of Waterloo had a problem that sounds quaint now and was not quaint then. They wanted to teach and do research with a computer algebra system. The systems available, chiefly MACSYMA and Reduce, needed more memory than the department could give to more than a handful of simultaneous users. MACSYMA in particular ran on a PDP-10 under a Lisp environment that assumed it owned the machine.

Waterloo had a shared VAX and a lot of students. Keith Geddes and Gaston Gonnet decided the way out was to write a system small enough that many people could run it at once.

The result was Maple. The name is a nod to Canada, and the design is a direct consequence of that memory constraint, in ways that are still visible in the product forty-five years later.

Small kernel, large library

The decision that defined Maple was splitting the system in two.

A small kernel, written in C, handles the things that must be fast and must always be resident: the data structures for expressions, arithmetic on integers and rationals of arbitrary size, memory management, the simplifier, and the interpreter for Maple's own programming language.

Everything else lives in a library written in that language, loaded on demand. Integration, differential equations, linear algebra, number theory, statistics, plotting. If you never call the differential equation solver, its code never enters memory.

The kernel started at something like a hundred kilobytes. The library was and is enormous, and it grows with every release, but you only pay for the parts you touch.

This looks obvious in retrospect. It was not the prevailing design. MACSYMA was a Lisp image, largely monolithic, and its footprint was its footprint. Writing the mathematical library in an interpreted language layered over a compiled core traded some speed for a great deal of flexibility, and it meant the library could be written by mathematicians rather than by systems programmers.

It also meant the library was readable. For most of Maple's history you could type the name of a library procedure with the right settings and see its source. Students learned the algorithms by reading the implementation. That property has eroded somewhat as parts of the system moved into the kernel or into compiled modules, and it remains truer of Maple than of most commercial mathematical software.

The company, and the split

Maple was commercialised through Waterloo Maple Software, founded in 1988, later Maplesoft. The University of Waterloo and the founders held the rights, and the system was sold into universities and engineering firms.

The most consequential business event in Maple's history is one that happened to a competitor. In 1988 Stephen Wolfram released Mathematica, with a notebook interface, aggressive marketing, and a pitch aimed as much at scientists generally as at symbolic mathematics specialists. The two systems have been paired in comparisons ever since, usually to Maple's disadvantage in visibility and to nobody's clear advantage in capability.

There is a footnote worth recording. In the early 1990s the Maple kernel was licensed to a company building a competing product, which is how MathCad got its symbolic engine. For a period, part of Maple was inside a product that competed with Maple.

Maplesoft was acquired by Cybernet Systems, a Japanese company, in 2009. Development continues, with annual releases, and the current emphasis is as much on Maple Flow and MapleSim, which target engineering documentation and physical modelling, as on the symbolic core.

What the language is like

Maple's programming language is a genuine language, not a command syntax, and it is idiosyncratic in ways that reward a little patience.

Everything is an expression, and expressions are represented internally as a directed acyclic graph with structure sharing. Identical subexpressions are stored once. This is invisible until you work with something large, at which point it is the reason a computation that should have exhausted memory did not.

Assignment uses :=, which is Pascal's convention and predates the systems most people learn first. A statement ending in a semicolon prints its result; one ending in a colon does not. That distinction catches every new user exactly once, usually when a computation prints a screenful of output they did not want.

Procedures are first class. The language has lexical scoping, remember tables that cache results automatically when you ask for them, and a type system rich enough to dispatch on structural properties of expressions. type(x, 'polynom(rational, y)') asks whether x is a polynomial in y with rational coefficients, and the answer is a real structural check rather than a guess.

The unevaluated quote is the piece that confuses people coming from other languages. Single quotes delay evaluation by one level, which matters because Maple evaluates aggressively. You use it to pass a name rather than a value, most commonly when a procedure needs somewhere to put a result.

Assumptions, and why they matter

The feature Maple users cite most, and the one that most repays understanding, is the assumption system.

Symbolic mathematics is full of statements that are true under conditions and false without them. The square root of x squared is x when x is non-negative and the absolute value of x otherwise. The logarithm of a product is the sum of the logarithms on a branch. An integral converges for some parameter ranges.

A system that does not track conditions has two options, both bad. It can simplify aggressively and be wrong sometimes, or it can refuse to simplify and be useless often.

Maple's answer is assume, which attaches properties to a name, and assuming, which applies them to a single expression. Tell it that x is real and positive and the square root simplifies. Tell it a parameter exceeds one and an integral evaluates. The properties live in a lattice, so declaring that something is a positive integer implies that it is an integer, rational, real and non-zero, and every relevant simplification unlocks at once.

The system announces its awareness of the constraint by decorating the variable with a tilde in output. This annoys people. It is the system telling you that the result you are looking at is conditional, which is exactly the thing a symbolic answer needs to communicate.

The general lesson transfers to any symbolic tool. When a computer algebra system refuses to simplify something you know is simplifiable, the usual cause is a condition you know and it does not.

Where Maple sits now

The honest picture of the current landscape has four kinds of tool, and Maple's position in it has narrowed without becoming marginal.

Mathematica has more visibility, a larger curated data layer, and a language that people either love or find impenetrable. SymPy is free, embedded in the Python ecosystem, and good enough for a large fraction of what students and engineers need. Sage bundles many open-source engines behind a single interface. Maple retains particular strengths in differential equations, where its solver is widely regarded as the best available, and in exact arithmetic performance on certain classes of problem.

Its other position is institutional. Maple is embedded in engineering curricula and in the workflows of firms that adopted it decades ago, along with Maple T.A. and its successors for automated assessment. That kind of adoption is slow to build and slow to leave.

The connection to everything else on this site

Maple belongs in a publication about artificial intelligence and mathematics for a reason that has nothing to do with Maple having AI features, though recent releases do.

It is the clearest available example of the component that supplies certainty. When a language model produces an integral, the integral is a guess shaped like an answer. When Maple produces one, it ran the Risch algorithm or a decision procedure derived from it, and the result is correct or the system tells you it could not find one in closed form.

The failure modes are opposite in the most useful way. Maple fails loudly, returning an unevaluated expression, and it fails on input it cannot parse or conditions it has not been told. A language model fails silently, in fluent prose, and it fails most often on exactly the long exact computations Maple was built for.

Every reliable workflow described elsewhere on this site is some arrangement of those two halves. The forty-five-year-old system is the half that cannot be wrong, and the reason it cannot be wrong is that a group in Waterloo could not afford the memory to run anything bigger.

Getting a Straight Answer Out of Maple

2026-08-24T00:00:00.000Z

People who give up on computer algebra systems usually give up for the same handful of reasons. The system refuses to simplify something obvious. It returns an answer in a form nobody wants. It hands back a result three pages long. It says nothing at all and sits there.

None of these are bugs. Each is the system behaving correctly under a constraint you have not seen. The examples below use Maple's syntax, and the causes apply to Mathematica, SymPy, Maxima and Sage with the names changed.

It will not simplify the square root

You ask for the square root of x squared and get back the square root of x squared.

> simplify(sqrt(x^2));
                              csgn(x) x

The answer is not x, because x might be negative. It is not the absolute value of x either, because x might be complex, and Maple defaults to assuming a symbol could be any complex number. What comes back is x multiplied by a sign function that resolves once the system knows where x lives.

Tell it:

> simplify(sqrt(x^2)) assuming x::real, x >= 0;
                                  x

This is the single most common source of frustration with any symbolic system, and the fix is always the same. State the domain. A symbol with no declared properties is a complex number, and most of the identities you learned are real-valued identities.

The habit worth building is to declare properties at the point where you introduce a variable rather than at the point where a simplification fails. It costs one line and it prevents a category of confusion rather than resolving instances of it.

It gives an answer, in the wrong shape

Symbolic results have many equivalent forms, and the system has no way of knowing which one you want. A partial fraction decomposition and a single rational expression are the same function. A product of factors and an expanded polynomial are the same polynomial.

Maple's approach is a set of transformation commands, each moving toward a specific normal form: expand, factor, normal, combine, collect, convert, rationalize, and simplify as a general-purpose heuristic.

simplify is the one to reach for last, not first. It applies a large collection of rules and tries to reduce some internal measure of complexity, and its notion of simpler does not always match yours. When you know the form you want, name it. collect(expr, x) gathers powers of x. convert(expr, parfrac, x) gives partial fractions. These are deterministic and they do what they say.

A useful diagnostic when two expressions ought to be equal and do not look it: subtract them and call simplify, or better, normal on the difference. Zero settles it. Comparing two complicated forms by eye settles nothing.

The answer is three pages long

Expression swell is intrinsic to exact computation and it is not a sign that anything went wrong.

The mechanism is easy to see with a determinant. A symbolic matrix of modest size produces a determinant with a number of terms that grows factorially, and the intermediate expressions produced along the way can be much larger than the final result even when the final result is small. Gröbner basis computation is doubly exponential in the worst case and reaches that worst case on inputs that look unremarkable.

Three responses, in order of how often they help.

Substitute numbers earlier. If half your parameters are known constants in the end, putting them in before the expensive step rather than after can change the computation from impossible to instant.

Ask for less. RootOf is Maple's representation for a root of a polynomial that has no useful radical form, and leaving an answer in that representation is often better than forcing it into radicals, which for a quartic produces something nobody can read and nothing can use.

Work modulo something. Many structural questions about polynomials can be settled over a finite field far faster than over the rationals, and the answer transfers.

It returns the integral unevaluated

This is Maple telling you something specific and true, and it is worth appreciating rather than resenting.

For elementary functions, the Risch algorithm decides whether an antiderivative exists in elementary terms. When Maple returns an integral unevaluated, the usual meaning is that no elementary antiderivative exists, or that the integrand falls outside the class the implemented algorithm decides completely.

The integral of e to the minus x squared has no elementary antiderivative. That is a theorem, not a limitation of the software. What you want in that case is the error function, and Maple will give you that if you ask for the definite integral or convert the result.

For definite integrals, an unevaluated result more often means the system could not establish convergence, which is a conditions problem again. Supplying assumptions on the parameters frequently unlocks it.

The general principle: a symbolic system that returns your input unchanged has told you it could not do the thing, which is more information than a confident wrong answer. Treat the unevaluated result as a message rather than a failure.

It disagrees with the numerical answer

When a symbolic result and a numerical check disagree, three causes cover almost every case.

Branch cuts. Complex-valued functions like logarithm and fractional powers have discontinuities, and an identity that holds on one branch fails across it. Evaluating at a point on the wrong side of a cut produces a genuine disagreement between two correct expressions.

Assumptions you supplied that are false at the test point. If you told the system a parameter is positive and then evaluate at a negative value, the simplified form is not valid there.

Floating point in the numerical check. Maple's evalf takes a precision argument for a reason. If a computation is ill-conditioned, the default fifteen digits can be inadequate, and raising Digits to fifty and seeing the disagreement shrink identifies the cause immediately.

Test at several random points, not one, and prefer points well away from zero, one, and any singularity. Agreement at twenty random points to twenty digits is not a proof, and it is enough to act on.

It is slow

The first question is whether you are computing symbolically when you need numbers.

Symbolic computation is expensive because it is exact. If the end product is a plot or a table of values, evalf early and use the numerical routines. Maple's evalhf evaluates in hardware floating point and is dramatically faster than software arbitrary precision when the extra precision is not needed.

The second question is whether you are recomputing. Maple procedures can carry a remember table, caching results by argument, and turning it on for a recursive function changes exponential work into linear work. This is memoisation with one option rather than a rewrite.

Where a language model fits

The pattern that works is narrow and it is the same one that applies to every symbolic tool.

Use the model to produce the Maple, not the answer. Describing a problem in English and asking for the commands that express it plays to what a language model is genuinely good at, which is translating between informal statement and formal syntax. Then run the commands. What comes back is computed by an algorithm, not predicted.

Use the model to interpret a result you already trust. A page of output containing RootOf and hypergeometric functions is correct and unreadable, and asking for an explanation of what the expression represents is low-risk, because the expression is already known correct.

Do not use the model as the source of the answer and the tool as a rubber stamp. The failure mode there is that a wrong answer arrives with a plausible verification attached, and the whole value of the algorithm is that it does not care what anyone expected.

The division holds throughout. One half of the workflow guesses well and fails silently. The other half never guesses and says so when it cannot proceed. Keeping track of which half produced a given line is most of the skill.

Why Mathematics Is So Hard to Get Out of a PDF

2026-08-22T00:00:00.000Z

Copy a paragraph out of a research paper and paste it somewhere. The text arrives, more or less. Now copy an equation from the same page. What arrives is something like f (x) = 1 2π e−x2/2, with the fraction flattened, the exponent inline, and the square root gone entirely.

This is not a bug in your reader. It is what the file contains.

What a PDF actually is

PDF is a page description language, descended from PostScript, and its job is to make a page look identical everywhere. It achieves that by describing the page as a sequence of drawing operations.

A text-showing operation says: using this font resource, at this position, with this transformation matrix, draw these glyph codes. That is the whole model. There are no paragraphs, no sentences, no words in any structural sense. There is not even a space character in many PDFs, because a typesetter that wants a gap simply moves the drawing position and starts the next run of glyphs somewhere else.

Reading text out of a PDF therefore means reconstructing structure that was never stored. An extractor collects the glyphs, sorts them by position, guesses where words end from the gaps, guesses where lines end from the vertical positions, and guesses reading order from the geometry. For ordinary prose in a single column, those guesses are reliable enough that the loss is invisible.

Mathematics breaks every one of them.

Why formulas break the reconstruction

Consider a fraction. In the file it is three things: a run of glyphs at one vertical position, a run of glyphs lower down, and a thin filled rectangle between them. There is nothing marking the relationship. The rectangle is a rectangle, indistinguishable from a rule under a heading or a border on a table.

A superscript is a glyph drawn slightly higher and slightly smaller. Whether that means exponentiation, a footnote marker, or a transpose depends on context the file does not record.

An integral sign is a glyph. Its limits are two small glyph runs positioned above and below it. The relationship is spatial and nothing else.

A square root is a radical glyph plus a horizontal rule whose length happens to cover the expression underneath. The scope of the root is expressed entirely by how long somebody drew a line.

So an extractor faced with mathematics has to infer a tree from a two-dimensional arrangement of marks, using conventions that are typographic rather than semantic. Sorting by position, which works for prose, actively destroys the information: it flattens a two-dimensional layout into one dimension, and the layout was the meaning.

The font problem on top

TeX, which typeset most of the mathematics published in the last forty years, made this worse in a specific way.

The Computer Modern math fonts were built with custom encodings. A glyph's code in the font is not its Unicode code point, and for many mathematical symbols there was no Unicode code point when the fonts were designed. A PDF produced by older TeX pipelines may contain glyph code 1 in a font named cmmi10, and what that means requires knowing that font's encoding table.

Well-behaved PDFs carry a ToUnicode map that translates glyph codes back to characters. Many older ones do not, or carry an incomplete one. When the map is missing, extraction produces plausible-looking garbage: Greek letters that come out as Latin ones, minus signs that come out as hyphens or as nothing.

Ligatures compound it. A single glyph may represent two characters, and without the map there is no way to know.

The three kinds of PDF

Practically, documents fall into three groups, and which group you have determines what is possible.

A born-digital PDF from a modern LaTeX pipeline has correct fonts, a usable ToUnicode map, and sometimes tagging. Text extraction works; mathematics still requires layout analysis, but the raw glyph identities are trustworthy.

A born-digital PDF from an older pipeline has correct positioning and unreliable character identity. This is the frustrating case, because the file looks perfect and extracts into nonsense.

A scanned PDF contains images of pages and no text at all. Everything must come from optical character recognition, and now you have the layout problem and the recognition problem together.

Journals have been scanning back catalogues for decades, so a large share of the mathematical literature before roughly 1995 exists only in the third form.

What tagging was supposed to fix

PDF has had a structural layer since 2001. Tagged PDF, standardised as PDF/UA, lets a document carry a logical structure tree alongside the drawing instructions: this run of glyphs is a heading, these are paragraphs, this is a table with these cells.

For mathematics, the intended mechanism is embedding MathML in the structure tree, so the file carries both the marks and the meaning.

Adoption is poor. Producing tagged PDF requires the generating tool to emit the structure, and most mathematical typesetting pipelines historically did not. This is improving: recent LaTeX releases have made tagged output a priority, and accessibility regulation in several jurisdictions now requires it for public sector documents. The effect on the existing corpus is nil, because the existing corpus is already written.

The accessibility consequence is the one that matters most and gets discussed least. A screen reader on an untagged mathematical PDF reads the flattened glyph sequence, which is meaningless. A blind mathematician reading a paper from 1998 has no route to the equations short of someone transcribing them.

What actually works

Three approaches, with different failure modes.

Get the source. For anything on arXiv since 1991, the LaTeX source is usually available for download alongside the PDF, and the source contains the mathematics as markup. This solves the problem completely and it is consistently the first thing people forget to try.

Rule-based layout analysis. Tools like INFTY and various open-source successors treat formula recognition as a structured problem: identify symbols, build a spatial relationship graph, parse it into a tree using typesetting conventions. These work well on clean input and degrade on anything unusual.

Neural image-to-markup. Treat a cropped formula image as input and generate LaTeX as output, the same architecture as image captioning. This is where most current work sits, and models such as Nougat and the various formula recognition systems have pushed accuracy on clean printed mathematics to a level that is genuinely useful.

The last approach has a failure mode worth naming. The output is always syntactically valid LaTeX, because the model learned to produce LaTeX. Whether it is the right LaTeX is a separate question, and a wrong exponent or a dropped subscript renders perfectly and looks correct. Silent corruption of a formula is worse than a visible failure, and it means any pipeline that extracts mathematics at scale needs a verification step that does not come from the same model.

The practical shape of it

If you need mathematics out of a document, work down this list.

Look for the source. arXiv, the author's page, the publisher's HTML version if one exists.

If the PDF is born-digital, check whether text extraction produces sane characters at all before investing in layout analysis. If Greek letters arrive as Latin, the font map is broken and no amount of layout work fixes it.

If you must recognise formulas from images, render what you extracted and compare it against the original visually. This is the only cheap check that catches silent corruption, and comparing two rendered images is much easier than comparing two strings of LaTeX.

For anything consequential, transcribe by hand. This sounds defeatist and it is the correct answer more often than people expect. A page of dense mathematics takes twenty minutes to type and the result is certainly right.

The underlying situation is unlikely to change for the existing literature. Millions of papers were published in a format that records how a page looked and not what it said, and that decision is now permanent for everything already printed.

Machines Reading the Mathematical Literature

2026-08-20T00:00:00.000Z

Every ambitious claim about artificial intelligence and mathematics rests on a quiet assumption: that the mathematical literature is available to the machine. Millions of papers, a couple of centuries of results, sitting there as training data or as a corpus to search.

It is available in the sense that the files exist. It is available in the sense a library is available to someone who cannot read the shelf labels. The corpus is overwhelmingly PDF, PDF records how a page looked rather than what it said, and the gap between those two things is where a surprising amount of the actual work in this field happens.

The scale of what is there

arXiv holds well over two million preprints and takes more than twenty thousand a month. For submissions since 1991 the LaTeX source is usually available, which makes it the single most valuable mathematical corpus in existence, because the mathematics arrives as markup rather than as marks on a page.

Everything else is harder. zbMATH Open and MathSciNet index millions of items with reviews and metadata, but the reviews are prose about papers rather than the papers. Journal back catalogues digitised in the 1990s and 2000s are page images. Books are mostly scans. Theses vary by institution and are frequently the worst-formed documents in the whole ecosystem.

A reasonable estimate is that the fraction of the mathematical literature available in a form where the formulas are machine-readable, without recognition, is well under half, and heavily skewed toward physics-adjacent fields that adopted arXiv early and toward the last thirty years.

What extraction pipelines do

A modern pipeline has four stages, and each loses something.

Layout analysis decides what is on the page: body text, headings, figures, captions, tables, display equations, footnotes, page furniture. This is object detection on a rendered page image, and it is largely solved for standard journal layouts and reliably wrong on two-column pages with figures spanning columns, on marginal notes, and on anything from before typesetting standardised.

Reading order reconstructs the sequence. Column detection carries most of it. Failures here produce text that is individually correct and globally scrambled, which is worse than it sounds because the scrambling is invisible in the output.

Text and formula recognition converts regions to characters and markup. Inline mathematics is the difficult case, because a symbol embedded in a sentence has to be recognised as mathematics before it can be parsed as mathematics, and the visual cue is often just an italic font.

Semantic recovery, where anyone attempts it, turns markup into meaning. LaTeX says how something is printed, not what it denotes. The same rendered expression can be a product, a function application, or a group action, and the source usually does not distinguish them.

The evaluation problem

Reporting accuracy on formula recognition is harder than it looks, and the reported numbers are softer than they appear.

Exact string match against reference LaTeX punishes correct output that differs cosmetically. Writing \frac{1}{2} where the reference has \dfrac{1}{2}, or omitting braces that were not needed, counts as a miss while rendering identically.

Rendered-image comparison fixes that by compiling both and comparing pixels, which is the metric most current work uses. It has its own hole: two expressions that render identically can carry different meaning, and an expression that renders slightly differently may be mathematically identical.

Neither metric captures the failure that matters most in practice. A model trained to emit LaTeX emits syntactically valid LaTeX essentially always. When it is wrong, it is wrong in a way that compiles and renders and looks like mathematics. A dropped subscript, a sign that flipped, an exponent that migrated. Nothing downstream flags it.

That asymmetry is the reason extraction at scale needs verification from something that is not the extractor. Rendering the output and comparing images against the source crop is the cheap version and it catches most of it.

Where this bites

Three consequences follow, and they are more concrete than the general worry.

Training data quality. A model trained on the mathematical literature is trained on whatever came out of somebody's extraction pipeline. Where the pipeline mangled formulas, the model learned mangled formulas. This is one plausible contributor to a pattern people notice: models are markedly better at mathematics phrased in prose and in common notation than at anything requiring unusual symbols, and the corpus is cleaner in exactly the same places.

Literature search. Searching for a theorem by its statement rather than by keywords requires the statements to be machine-readable. This is the capability mathematicians most consistently say they want, and it is blocked mainly by extraction rather than by search. Formula search exists in zbMATH and in specialised systems, and its coverage tracks the availability of clean markup.

Autoformalisation. Translating a paper into Lean requires reading the paper. If the pipeline delivers a corrupted statement, the formalisation is a faithful rendering of the wrong theorem, and the proof assistant will happily verify a proof of it. The type checker guarantees the proof matches the statement. Nothing guarantees the statement matches the paper.

That last point deserves emphasis because it inverts the usual comfort. Formal verification removes doubt about the proof and relocates all the remaining risk into the statement, and the statement came through the part of the pipeline with the weakest guarantees.

What has improved

The last few years produced real gains, and they are worth stating precisely rather than dismissively.

End-to-end document models that take a page image and emit structured markup, Nougat being the widely used example, handle full pages including inline mathematics rather than requiring formula regions to be cropped first. On clean printed pages from the last two decades they are good enough to use as a first pass.

Vision-language models handle unusual layouts considerably better than rule-based systems, because they were not built around assumptions about column geometry. They also hallucinate more inventively, which is the tradeoff.

Handwriting recognition for mathematics has improved to the point where it is practical for lecture notes, which was not true five years ago.

None of this touches the fundamental issue. A recognised formula is a hypothesis about what was printed. Confidence in that hypothesis has gone from low to fairly high, and it has not become certainty, and the pipelines mostly do not report calibrated confidence at all.

What would actually fix it

Two things, one boring and one already happening.

Publish the source. Where an author has LaTeX, making it available alongside the PDF eliminates the entire problem for that paper. arXiv has done this since the beginning. Most journals do not. This costs nothing technically and is a policy question.

Tag the output. PDF has carried a structural layer since 2001, and embedding MathML in it makes a document carry meaning alongside appearance. Recent LaTeX releases have made tagged output a priority and accessibility regulation is pushing publishers toward it. New documents will increasingly be fine.

Neither helps the existing corpus, which is the whole point. A century of mathematics was published in formats that record appearance, and no policy change reaches backwards. Whatever we recover from it, we recover by recognition, with the error rates that implies.

The framing that holds up: the bottleneck under machine mathematics at scale is not reasoning and is not compute. It is that the input is a picture of a page, and turning a picture into a statement is a guess, dressed in markup that renders perfectly whether or not the guess was right.

How Machines Learned to Do Mathematics

2026-08-18T00:00:00.000Z

The first program that proved a mathematical theorem ran in 1956 on a machine with less memory than a modern doorbell. It was called Logic Theorist, written by Allen Newell, Herbert Simon and Cliff Shaw, and it worked its way through propositions from Whitehead and Russell's Principia Mathematica. It proved thirty-eight of the first fifty-two. For one of them it found a shorter argument than the one in the book. Simon wrote to Bertrand Russell about it. Russell replied that he was delighted to learn the propositions could be proved by machine, and added that if the work had been available earlier he and Whitehead might have saved themselves some years.

That exchange contains the whole argument that has run ever since. A machine did something that looked like mathematics. A mathematician conceded that it was useful. Nobody agreed on whether the machine understood anything.

Rules first

The early decades belonged to symbol manipulation. If mathematics is a formal game with pieces and legal moves, then a computer that knows the moves should be able to play. That premise produced a genuinely impressive line of software, and it is still the software most working scientists actually depend on.

MACSYMA, begun at MIT in 1968, could integrate functions symbolically, factor polynomials and simplify expressions that would take a graduate student an afternoon. Its descendants are Maple, Mathematica, Maxima, SymPy and Sage. When you ask one of these systems for the indefinite integral of a rational function, it does not search or guess. It runs the Risch algorithm, a decision procedure published in 1969 that determines whether an elementary antiderivative exists and constructs it when it does. The answer is not a plausible answer. It is the answer, in the same sense that long division gives the answer.

Alongside computer algebra grew automated theorem proving. Resolution provers, tableau provers, model checkers, SAT and SMT solvers. These systems do not integrate functions; they decide whether a logical statement follows from a set of premises. Modern SAT solvers are one of the quiet triumphs of computer science. They routinely settle problems with millions of variables, and industry uses them for chip verification, scheduling and dependency resolution. The package manager that installs your software probably runs one.

The limitation was never correctness. It was reach. A rule-following system can only follow the rules it has. Ask a computer algebra system to prove something that requires an idea, a substitution nobody has seen before, a clever choice of auxiliary construction, and it has nothing to offer. The search space of possible proofs grows faster than any amount of hardware can absorb. Mathematics, it turned out, is mostly about knowing which branches not to explore, and rules do not encode taste.

Statistics second

The alternative approach spent decades looking hopeless. Instead of encoding the rules of mathematics, train a statistical model on examples of mathematics and let it learn whatever regularities exist in the data. For most of the history of the field this produced systems that could not reliably add two three-digit numbers.

Two things changed. The transformer architecture arrived in 2017 and made it practical to train very large sequence models on very large corpora. And it became clear that a model trained to predict the next token across the internet, including textbooks, forums, homework solutions, arXiv papers and Stack Exchange threads, absorbs an enormous amount of mathematical pattern along the way.

By 2022, large language models could solve grade-school word problems at a rate that surprised the people who built them. By 2024, systems built on the same foundations were scoring at the level of national olympiad medallists on competition geometry and combinatorics. In July 2024, DeepMind's AlphaProof and AlphaGeometry 2 together solved four of the six problems at the International Mathematical Olympiad, a silver medal performance. In 2025, several systems reached gold medal scores on the same competition.

The interesting part is how they did it. AlphaGeometry pairs a language model with a symbolic deduction engine. The symbolic engine grinds out every consequence it can derive from the given configuration. When it gets stuck, the language model proposes an auxiliary construction, the extra point or line that unlocks the problem, which is exactly the step that pure search could never afford to guess at. Then the symbolic engine takes over again and checks everything. Neither half works alone. The neural component supplies intuition and the symbolic component supplies certainty.

What the split means

This is the shape almost every serious mathematical AI system now has. A component that guesses well, and a component that verifies. The guessing component is fast, creative, and wrong a meaningful fraction of the time. The verification component is slow, unimaginative, and never wrong about the things it checks.

It matches, uncomfortably closely, how mathematicians describe their own practice. Nobody derives a theorem by exhaustive deduction from axioms. They see a shape, form a hunch, sketch an argument, and then spend most of their time checking whether the hunch survives contact with the details. Most hunches do not. The formal write-up is a record of the ones that did.

The difference is that a mathematician's verification step is a human referee with limited attention, and a machine's verification step can be a proof assistant that checks every inference down to the axioms. Lean, Coq, Isabelle and HOL Light will not accept a hand wave. When Peter Scholze wanted to be certain of a result whose proof he found exhausting to check, he put it to the Lean community as a challenge. The formalisation was completed. He described the outcome as removing a genuine doubt.

What still does not work

It is worth being precise about the current failure modes, because the marketing is not.

Language models are unreliable at long arithmetic, for structural reasons involving how numbers are broken into tokens. They will produce a confident proof of a false statement if the false statement resembles a true one. They cannot tell you how confident they are in any calibrated way. They perform worse on problems phrased in unfamiliar notation even when the mathematics is identical. And a large share of headline benchmark performance is contested, because the benchmark problems and their solutions were plausibly in the training data.

None of these are fatal. All of them mean that an unverified model output is a conjecture, not a result.

Where this is going

The near-term picture is not machines replacing mathematicians. It is a change in what counts as a routine step. Terence Tao has written about using these tools as a slightly unreliable but very fast collaborator, useful for generating candidate approaches, formalising tedious lemmas and searching literature. Formalisation projects that would have taken a decade are now taking a year or two.

The longer question is whether a system that guesses statistically can produce a genuinely new mathematical idea, as opposed to recombining ideas it has seen. Nobody knows. The honest position is that we have built something that is extremely good at the parts of mathematics that look like pattern recognition, which turns out to be a much larger share of the subject than the romantic story admits, and that the remaining part has not yet been touched.

Russell's reply to Simon is still the right note to end on. The machine did something useful. Whether it understood anything is a separate question, and seventy years later it is still open.

Why a Language Model Cannot Count the R's in Strawberry

2026-08-14T00:00:00.000Z

Ask a state-of-the-art language model how many times the letter R appears in the word strawberry, and for several years the answer was two. The correct answer is three. This became a running joke, then a benchmark, then a genre of screenshot. It is worth understanding properly, because the same mechanism explains a much more consequential failure: why models are shaky at arithmetic on numbers they have not effectively memorised.

The model does not see letters

A language model does not receive text. It receives tokens, which are integer identifiers drawn from a fixed vocabulary of typically 50,000 to 200,000 entries. The text is chopped into these pieces by a tokeniser before the model sees anything, usually with an algorithm called byte pair encoding.

Byte pair encoding builds its vocabulary greedily. Start with individual bytes, then repeatedly find the most frequent adjacent pair in the training corpus and merge it into a new single token. Do this a hundred thousand times and you end up with a vocabulary where common words are single tokens, common word fragments are single tokens, and rare sequences fall back to smaller pieces.

Under a typical vocabulary, strawberry becomes something like str aw berry. Three integers. The model never receives a sequence of eight characters. It receives three opaque symbols whose internal spelling is not part of the input at all.

So when you ask how many R's it contains, the model has to answer a question about information that was destroyed before the first layer of the network ran. It can only answer from statistical association: what it has read about the spelling of words, in text that discusses spelling. That is a much weaker signal than looking, and it is why the failure was so stubborn.

Think of it as asking someone to count the letters in a word they have only ever heard spoken in a language whose spelling is irregular. They can often guess correctly. They are guessing.

The same problem, with digits

Now consider a number like 4,817,293. How does a tokeniser split that?

It depends entirely on the tokeniser. Some split digits into groups of three from the left, which is not how place value works. Some produce chunks of one, two or three digits based on frequency in the training corpus, so 481 might be a single token because it appeared often, while 7293 becomes 72 and 93. Some tokenisers used to split inconsistently depending on whether a comma or a space preceded the number.

The consequence is that the model's internal representation of a number is not positional in any clean way. To add two numbers correctly, an algorithm needs to align digits by place value and propagate carries. A model working from irregular chunks has to reconstruct place value from context before it can do anything else.

The best current models handle this considerably better than earlier ones, mostly through two fixes. Several vocabularies now force digits to tokenise one at a time or in fixed groups, which restores a consistent positional structure. And models are explicitly trained on large quantities of synthetic arithmetic with worked intermediate steps.

But the underlying situation has not changed. Arithmetic in a language model is a learned approximation of an algorithm, not the algorithm. It degrades as numbers get longer, exactly where a real algorithm would not.

Why chain of thought helps

If you ask a model for the product of two four-digit numbers and demand only the answer, accuracy is poor. If you ask it to work through the multiplication in the way a schoolchild would, writing partial products and adding them, accuracy improves a great deal.

The reason is not that the model is thinking harder. A transformer performs a fixed amount of computation per token generated. There is a hard ceiling on how much work can happen between reading your question and emitting the first token of the answer. It is a fixed-depth circuit, and long multiplication requires a number of sequential steps that grows with the length of the numbers.

Writing intermediate steps moves that computation into the output sequence, where it is no longer bounded by network depth. Each partial product is a small enough operation to fit in one forward pass. The model then reads its own written work as context for the next step. Chain of thought converts a depth problem into a length problem, and length is cheap.

This is a genuinely useful mental model. When a task needs a number of sequential dependent operations, either the model writes those operations out, or it does not perform them.

The correct fix

Neither better tokenisation nor longer chains of thought is the real answer for arithmetic. The real answer is that the model should not be doing arithmetic at all.

Calculators exist. They are exact, they are older than the transistor, and they cost nothing to run. Any serious deployment gives the model a tool: a sandboxed interpreter, a calculator function, a computer algebra system. The model's job becomes translating the problem into a form the tool can evaluate, reading the result, and interpreting it. That is the part language models are genuinely good at.

This pattern generalises far past arithmetic. The model is a translator between informal human framing and formal machinery. The formal machinery is what you trust. When someone shows you an impressive mathematical output from an AI system, the useful question is always which component produced the guarantee.

What this means for you

Three practical rules follow.

First, treat any unverified number in a model's output as a rough estimate, including numbers that look precise. Precision in the rendering tells you nothing about precision in the derivation. A model will write 0.03847 with the same confidence it writes 0.04.

Second, when you need arithmetic, make the tool call explicit. Ask for code, run the code. If the interface offers a code execution tool, turn it on. The difference in reliability is not marginal.

Third, remember that the failure is silent. A model that cannot count letters does not tell you it cannot count letters. It answers two, in a fluent sentence, with no hedge. This is the general shape of the problem: the confidence of the output is roughly uncorrelated with its correctness on tasks the architecture cannot support.

A footnote on the strawberry

Newer models mostly answer three now, and it would be easy to read that as the problem being solved. It is worth being careful there. Some of the improvement is better tokenisation. Some is that models now often spell the word out character by character before counting, which is the chain of thought fix. And some is simply that the strawberry question appeared in so much training text, complete with its answer, that the response is now close to memorised.

Test it yourself on a word nobody has written a viral post about. Pick something long with a repeated letter buried in the middle. The failure mode is often still there, just less famous.

Inside a Proof Assistant: What Lean Actually Checks

2026-08-10T00:00:00.000Z

There is a persistent confusion about proof assistants. People hear the name and imagine software that helps you find a proof, in the way a spellchecker helps you find a typo. That is not what these systems do. A proof assistant is a machine that reads a proof you already wrote and refuses to accept it if any step is unjustified. The assistance is in the refusing.

Lean, Coq, Isabelle, Agda and HOL Light differ in their logical foundations and their culture, but they share this. You state a theorem. You construct a term. The system checks the term against the statement. If the check passes, the theorem holds, subject to a small set of assumptions we will get to.

Proofs as programs

The idea underneath most modern proof assistants is the Curry-Howard correspondence, and it is one of the genuinely beautiful facts in logic. Propositions correspond to types. Proofs correspond to programs that inhabit those types.

If A and B are propositions, then "A implies B" corresponds to the type of functions from A to B. A proof of the implication is a function that takes any proof of A and returns a proof of B. Conjunction corresponds to a pair type: a proof of "A and B" is a pair containing a proof of A and a proof of B. Universal quantification corresponds to a dependent function type. Existential quantification corresponds to a dependent pair.

Once you accept this dictionary, checking a proof becomes type checking, which is a mechanical, decidable procedure. There is no cleverness in the checker. It walks the term, applies typing rules, and either everything lines up or it does not.

This is why proof assistants are trustworthy in a way that is difficult to achieve otherwise. The interesting, complicated, potentially buggy parts of the system are the parts that help you write the term. The part that decides whether to believe you is small.

The kernel

That small part is called the kernel. In HOL Light it is a few hundred lines. In Lean 4 it is a few thousand. Everything else in the system, the tactic language, the elaborator, the automation, the library, the editor integration, can be arbitrarily elaborate and buggy, because none of it is trusted. Whatever those layers produce is handed to the kernel, and the kernel checks it from scratch.

This design is called the de Bruijn criterion, after Nicolaas de Bruijn, who built the Automath system in the late 1960s. It means the trusted computing base of a formal proof is a piece of code that a determined person can read in an afternoon. Several projects have written independent checkers that verify Lean's output, precisely so that a bug in Lean's own kernel would be caught by a program that shares none of its code.

What you actually trust when you trust a Lean proof: the kernel implementation, the compiler that compiled it, the hardware it ran on, and the axioms the proof used. That is a short and inspectable list. Compare it to what you trust when a referee reads a ninety-page paper.

Tactics: the part that feels like work

Nobody writes proof terms by hand. They are enormous. Instead you write tactics, which are commands that build the term for you.

A Lean proof of a small lemma might read:

theorem add_zero_right (n : Nat) : n + 0 = n := by
  induction n with
  | zero => rfl
  | succ k ih => simp [Nat.succ_add, ih]

induction splits the goal into the base case and the inductive step. rfl closes a goal where both sides are definitionally equal. simp rewrites using a database of simplification lemmas. Each tactic is a program that manipulates the proof state and ultimately emits a term.

The proof state is the thing you spend your time staring at. It shows your current hypotheses and the goal you still owe. Every tactic transforms it. Proving in Lean feels less like writing and more like playing a puzzle game where you are trying to reduce the goal to nothing.

The learning curve here is real and it is not about mathematics. It is about learning which of several thousand library lemmas is the one that closes your goal, and what it is called. Mathlib, Lean's mathematics library, now contains well over a million lines and something like two hundred thousand theorems. Finding the right one is the daily difficulty.

This is also, not coincidentally, the part where machine learning has been most useful. Premise selection, the problem of picking which lemmas are relevant to the current goal, is a search problem over a large corpus with weak signals, which is exactly what neural retrieval is good at. Tools that suggest tactics or complete proofs from the current state are now routine parts of the workflow.

What formalisation costs, and what it buys

The classical estimate is the de Bruijn factor: the ratio between the size of a formal proof and the size of the informal one. For a long time it sat around four. For well-supported areas with mature libraries it is now often close to one or two. For areas where the library has nothing, it can be enormous, because you have to build the foundations before you can state the theorem.

The famous case studies show the range. Georges Gonthier's formalisation of the four colour theorem took several years. His formalisation of the odd order theorem, a 255-page paper by Feit and Thompson, took six years and produced 170,000 lines of Coq. The Flyspeck project, formalising Thomas Hales's proof of the Kepler conjecture, ran from 1998 to 2014.

Set against that: the Liquid Tensor Experiment. In 2020 Peter Scholze posted a challenge asking whether a particular theorem of his could be formalised, saying he wanted to be sure of it and found the proof difficult to fully check by hand. A community effort led by Johan Commelin completed the main result in about eighteen months. In the process they found that one lemma as originally stated was slightly wrong, in a way that did not break the proof but did need fixing.

That is the actual value proposition. Not certainty for its own sake, but a way to check work that has grown too large or too intricate for the referee system to handle honestly.

Where the AI fits

Two roles, and they are worth keeping separate.

The first is proof search. Given a goal, produce a tactic or a sequence of tactics that closes it. This is what systems like AlphaProof do, in combination with reinforcement learning against the proof assistant itself as an environment. The proof assistant is an unusually good training signal because it gives a clean, non-negotiable reward: the proof checks or it does not. There is no reward hacking available when the referee is a type checker.

The second is autoformalisation. Given a theorem written in ordinary mathematical English, produce the formal statement. This is much harder to evaluate, because a wrong formalisation can be perfectly provable. If you formalise "every continuous function on a compact set is bounded" but get the definition of compact subtly wrong, you will prove something true and useless. There is no automated check for whether the formal statement means what the English meant. A human has to read it.

That asymmetry is the thing to remember. Once a statement is formalised, the machine can be trusted completely with the proof. Getting to the formal statement in the first place remains a human judgement, and no amount of proof automation removes it.

Starting

If you want to try it, the standard entry point is the Natural Number Game, a browser-based tutorial that has you rebuild arithmetic from the Peano axioms using Lean tactics. It takes a few hours and it will tell you very quickly whether you find this kind of work satisfying or maddening. Both reactions are common and both are reasonable.

Ten Math Riddles and How Machines Handle Them

2026-08-06T00:00:00.000Z

Riddles are a poor benchmark and an excellent diagnostic. Poor benchmark because the famous ones are all over the training data, complete with worked solutions, so a correct answer proves nothing about reasoning. Excellent diagnostic because you can take a famous riddle, change one detail so the standard answer becomes wrong, and watch what happens.

Here are ten, with the honest picture of how current models handle each. Try them yourself first.

1. The bat and the ball

A bat and a ball cost 1.10 together. The bat costs 1.00 more than the ball. How much is the ball?

The intuitive answer is 0.10, and it is wrong. If the ball were 0.10 the bat would be 1.10 and the total 1.20. Let the ball be x. Then x plus x plus 1 equals 1.10, so x is 0.05.

This is the most-cited item from Shane Frederick's Cognitive Reflection Test, and roughly half of tested undergraduates get it wrong. Modern language models get it right essentially always, which tells you nothing, because it appears thousands of times in training text.

Change the numbers. Bat and ball cost 5.40, bat costs 4.60 more. Ball is 0.40. Models still handle this reliably, which does suggest the structure was learned rather than the answer memorised.

2. The lily pads

A patch of lily pads doubles in size every day. It covers the lake on day 48. On what day did it cover half the lake?

Day 47. Reliable across models and worth keeping because it is the cleanest illustration of how badly human intuition handles exponentials, which is a habit worth carrying into any argument about growth curves.

3. Monty Hall

Three doors, a car behind one, goats behind two. You pick door one. The host, who knows where the car is, opens door three to reveal a goat and offers you the switch. Do you switch?

Yes. Switching wins two times in three. The intuition that trips people is that the host's choice is not random. He will never open the door with the car and never open your door, so his action carries information about doors two and three collectively.

Models answer this correctly and confidently. Now change it. Suppose the host does not know where the car is, opens door three at random, and it happens to contain a goat. Now switching gives no advantage; both remaining doors are at 1/2. This variant, sometimes called Monty Fall, is a good stress test. Models are noticeably less reliable here, and a common failure is reciting the standard 2/3 answer with the standard justification while the premise no longer supports it.

4. Two envelopes

You are given two envelopes. One contains twice as much money as the other. You open one and find 100. Should you switch?

The tempting argument: the other envelope holds either 50 or 200, each equally likely, so its expected value is 125, so switch. But by symmetry the same argument applies before you open anything, and after you switch it applies again, which would have you swapping forever.

The error is assuming a uniform prior over an unbounded range, which is not a probability distribution. Once you specify an actual prior over the amounts, the paradox dissolves and switching is sometimes right and sometimes wrong depending on what you found.

Models usually identify this as the two-envelope problem and produce a serviceable summary. Push them on which specific step of the naive expected-value calculation is invalid and the answers get vaguer.

5. The hundred prisoners and the boxes

A hundred prisoners are numbered. In a room are a hundred boxes, each containing one prisoner's number in random order. Each prisoner may open fifty boxes, looking for their own number. All must succeed or all are executed. No communication once it starts. What strategy gives the best odds?

Random opening gives each prisoner 1/2 and the group 2 to the power of minus 100, which is nothing. The good strategy: each prisoner opens the box with their own number, reads the number inside, opens that box, and follows the chain.

This succeeds for everyone exactly when the random permutation has no cycle longer than fifty. The probability of a long cycle is a harmonic sum, and the survival probability comes out around 31 percent. Not 31 percent per prisoner. Thirty-one percent for all hundred simultaneously.

The result feels impossible and it is correct. Models will produce it, with the cycle argument, because it is a well-known puzzle. Ask for the exact probability as a harmonic expression and the derivation is where errors creep in.

6. Blue eyes

An island has a hundred people with blue eyes and a hundred with brown. Nobody knows their own eye colour, no mirrors, no discussion of eyes. Anyone who deduces their own colour must leave at midnight. A visitor announces publicly that they can see at least one person with blue eyes.

Everyone already knew that. Yet on the hundredth midnight, all hundred blue-eyed people leave.

The induction: with one blue-eyed person, they see nobody else blue and leave immediately. With two, each sees one other, expects them to leave on night one, and when they do not, both deduce their own colour on night two. And so upward.

What the announcement adds is not the fact but the common knowledge of the fact. Everyone knew it; not everyone knew that everyone knew that everyone knew it, to the required depth. This distinction between mutual and common knowledge is the whole content of the puzzle, and it is a real concept with real consequences in distributed systems and game theory.

Models reproduce the induction competently. They are much weaker when asked to state precisely what changes in the knowledge structure at each level.

7. The surprise examination

A teacher announces an exam next week that will be a surprise: on the morning of the exam, students will not know it is that day. A student argues it cannot be Friday, since by Thursday evening only Friday remains and it would not be a surprise. Eliminating Friday, it cannot be Thursday by the same reasoning. And so on back through Monday. The exam cannot happen. On Wednesday it happens, and it is a surprise.

There is no consensus resolution. The literature runs to hundreds of papers, with treatments through epistemic logic, self-reference and the Knower paradox. This is a good item precisely because there is no answer to memorise. A model that presents a confident resolution is telling you more about its calibration than about the paradox.

8. The wine and water

Two glasses, one of wine and one of water, equal volumes. Take a spoonful of wine, stir it into the water. Take a spoonful of that mixture back into the wine. Is there more wine in the water glass or more water in the wine glass?

Exactly equal. Both glasses end at their original volume, so whatever volume of wine is missing from the wine glass has been replaced by exactly that volume of water. No calculation required, and the result is independent of how much you stirred, how many spoonfuls you moved, or whether the mixing was thorough.

The conservation argument is short and total. Models often reach the right answer through an algebraic slog with a specific spoon size instead, which works but misses the point.

9. Three hats

Three people in a line, each wearing a hat that is black or white, each able to see only the hats in front. Starting from the back, each says whether they know their own colour. Given a particular pattern of answers, deduce the arrangement.

These are cleanly solvable by case elimination and models handle them well. The failure mode appears when you change the visibility rules or let the participants answer out of order. Then the model frequently applies the standard solution's reasoning steps to a configuration where they no longer hold.

10. Ants on a pole

Twenty-five ants are placed on a metre-long pole at random positions, each facing left or right. All walk at one centimetre per second. When two meet, both reverse direction instantly. What is the longest time before the pole is clear?

One hundred seconds. Two ants colliding and reversing is indistinguishable from two ants walking through each other, if you stop tracking identities. So treat them as passing through. Each ant then walks in a straight line and the worst case is one ant at one end walking the full length.

This is the loveliest reframing in the list, and it is the sort of insight that pure search never finds. Models produce it, because the puzzle is well known. Change it to ants of different speeds and the pass-through trick fails, since identities now matter, and the answers degrade quickly.

What the pattern suggests

Across all ten, the reliable answers cluster on famous problems with famous solutions. The unreliable answers cluster on variants where the famous solution is a trap.

That does not mean nothing is being learned. The bat-and-ball generalises to new numbers, which is genuine. But it does mean that a correct answer to a classic puzzle is close to worthless as evidence about reasoning, and that if you want to know what a system can actually do, you have to write a problem nobody has written before.

Olympiad Geometry and the Machine That Draws the Extra Line

2026-08-02T00:00:00.000Z

Olympiad geometry is a strange corner of mathematics. The problems are elementary in the sense that they need no machinery past secondary school: circles, triangles, angles, similar figures. They are also brutally hard, because solving them depends almost entirely on seeing the one construction that makes everything collapse. Draw the right auxiliary line and the problem takes four steps. Miss it and you can work for hours.

That property makes olympiad geometry an unusually clean laboratory for asking what mathematical intuition is.

Why brute force fails here

A geometry problem gives you a configuration and asks you to prove a relationship. In principle you could just derive consequences. Angle chasing is mechanical: this angle equals that one because of the inscribed angle theorem, therefore these triangles are similar, therefore these lengths are proportional. Software has done this well since the 1990s. Wu's method and Gröbner basis approaches can decide large classes of geometric statements algebraically, and they are complete in the sense that if the statement is true in the algebraic model, they will find that out.

The problem is that many olympiad problems are not provable from the given configuration alone. You have to add something. Extend a segment to meet a circle. Drop a perpendicular. Mark the midpoint of an arc nobody mentioned. Construct the reflection of a point over a line.

Once you allow additions, the search space explodes. There are infinitely many points you could add, and no way to score a candidate short of trying it and seeing whether the proof then goes through. This is where classical automated geometry stalls: not on deduction, on invention.

The two-component design

AlphaGeometry, published by a DeepMind team in Nature in January 2024, attacks exactly this split. It has two parts that alternate.

The symbolic engine is a deduction system that takes the current configuration and derives every consequence it can, exhaustively, using a fixed rule set covering angles, ratios, similar triangles and a rule for handling algebraic consequences. It runs to saturation: it keeps deriving until nothing new appears. If the goal shows up in the derived set, the problem is solved and the proof is a trace through the derivation.

The language model is trained to propose auxiliary constructions. When the symbolic engine saturates without reaching the goal, the model looks at the configuration and suggests a new point or line. That gets added, and the symbolic engine runs again on the enlarged configuration.

Loop until solved or out of budget.

The result on a benchmark of thirty IMO geometry problems from 2000 to 2022: twenty-five solved. The average human gold medallist solves about twenty-six. The previous best automated system solved ten.

The training data problem, and the trick

There is no corpus of a hundred million worked olympiad geometry proofs. There are a few thousand problems in total, which is far short of what training a useful model requires.

The team generated data synthetically, and the method is the clever part. Rather than trying to produce problems and then solve them, they ran the process backwards. Sample a random geometric configuration. Run the symbolic engine to saturation, producing a large set of true statements about it. Pick one of those statements as a goal, and trace back which premises the derivation actually used.

Now here is the step that matters. Some points used in the derivation do not appear in the statement of the goal. Those points are, by definition, auxiliary constructions. Strip them from the problem statement and you have a problem whose solution requires inventing exactly those points, together with a proof that does so.

They generated around a hundred million such synthetic proofs, of which roughly nine million involved at least one auxiliary construction. That is the training set: not human proofs, but machine-generated problems that were reverse-engineered to require the skill being taught.

What the division of labour tells us

The architecture is a claim about mathematics, whether or not it was meant that way. It says that deduction is mechanical and can be trusted to a rule engine, and that the creative act is narrow and specific: knowing what to add.

That matches how strong competitors describe their own experience. They do not report agonising over whether an angle chase is valid. They report staring at a diagram until something suggests itself. The valid-step-checking is background; the seeing is the work.

It also means the system's output is trustworthy in a way that a language model's essay is not. The proof is produced by the symbolic engine, which only applies sound rules. The language model's contribution is a suggestion that gets checked. If it suggests a useless construction, nothing breaks; the deduction engine simply fails again and another suggestion comes.

This is the pattern worth extracting. Put the neural component where being wrong is cheap. Put the symbolic component where being wrong is expensive.

The follow-up work

AlphaGeometry 2, reported in 2025, extended coverage to problems involving movement of objects and equations of linear angles, and pushed the solve rate on IMO geometry problems from 2000 onward to around 84 percent. Combined with AlphaProof, a Lean-based system for the non-geometry problems, the pair achieved a silver medal score at the 2024 IMO, one point below gold. In 2025 several systems, including general-purpose reasoning models running without a bespoke geometry engine, reached gold medal scores.

That last detail complicates the story. If a general reasoning model can score gold without any symbolic deduction engine, does the two-component argument still hold?

Partly. The general models are considerably slower, use vastly more computation, and produce proofs in natural language that need human checking. They are not producing machine-verified output. What they demonstrate is that enough scale and enough reinforcement learning on reasoning traces can approximate the same capability without an explicit architecture for it. Whether the approximation holds up outside competition mathematics is not yet established.

What olympiad performance does not mean

Competition mathematics is a specific genre. Problems have known-length solutions, are guaranteed solvable, use a bounded set of techniques, and were written by humans intending them to be found. Research mathematics has none of those properties. You do not know whether the statement is true. You do not know whether the tools exist. You do not know whether the problem is worth the time.

Tim Gowers, who helped grade AlphaProof's 2024 output, made the point that the system's answers were correct but arrived through search rather than through the kind of structural understanding that generalises. A human solving these problems builds intuitions that transfer. Whether these systems do is an open empirical question and not one that a competition score answers.

The honest summary: machines now solve a category of hard, human-designed problems at the level of the strongest human competitors, using a method that separates guessing from checking. That is a real result. It is not the same as doing mathematics, and the people who built it are generally more careful about saying so than the coverage has been.

The Four Colour Theorem and the First Proof Nobody Could Read

2026-07-29T00:00:00.000Z

In October 1852, a student named Francis Guthrie was colouring a map of the counties of England and noticed he never needed more than four colours to keep adjacent counties distinct. He mentioned it to his brother, who mentioned it to Augustus De Morgan, who wrote to William Rowan Hamilton about it the same day. Hamilton was unimpressed and did not reply for a fortnight.

The question sat open for 124 years. It has a claim to being the first major theorem whose proof a human cannot check.

What the statement actually says

Any map drawn in the plane, divided into contiguous regions, can be coloured with four colours so that no two regions sharing a border segment get the same colour. Regions touching at a single point do not count as adjacent. Regions must be connected, which rules out awkward cases like a country with an overseas territory.

Translated to graph theory: every planar graph has chromatic number at most four. Place a vertex in each region, connect vertices whose regions share a border, and the resulting graph can be drawn in the plane without crossings. The claim is that four colours suffice for any such graph.

Three colours are not enough, as any map with four mutually adjacent regions shows. Five colours were proven sufficient by Percy Heawood in 1890, with an argument short enough to fit in an undergraduate lecture. The gap between five and four took another 86 years.

Kempe's wrong proof, and why it mattered

In 1879 Alfred Kempe published a proof. It was accepted, celebrated, and got him elected to the Royal Society. In 1890 Heawood found the error.

Kempe's method is still the foundation of everything that came after, which is why the failure was productive rather than merely embarrassing. His idea was reducibility. Suppose a counterexample exists. Take the smallest one, by number of regions. Every planar graph contains a vertex of degree five or less, a consequence of Euler's formula. So the minimal counterexample contains such a vertex. Remove it, colour the smaller map by minimality, then put the vertex back and try to find a colour for it.

For a vertex of degree three or four, this works directly. Degree five is where it gets difficult, and Kempe's argument for that case involved chains of alternating colours that could be swapped to free up a colour. Heawood produced a configuration where the chain-swapping argument breaks: two swaps interfere with each other.

The framework survived. What was needed was a much larger collection of configurations, each proven reducible, such that every planar graph must contain at least one of them. Such a collection is called an unavoidable set.

Appel and Haken

Kenneth Appel and Wolfgang Haken at the University of Illinois spent years, building on work by Heinrich Heesch, developing a discharging procedure that would generate an unavoidable set, and a computational test for reducibility. In June 1976 they announced a proof.

The unavoidable set had 1,936 configurations. Each had to be checked for reducibility by computer. The computation took roughly 1,200 hours on an IBM 370. The paper ran to 741 pages including microfiche supplements.

The University of Illinois mathematics department started stamping its outgoing mail with "FOUR COLORS SUFFICE".

The reception was mixed in a way that had nothing to do with whether people believed the result. It was about whether the object produced was a proof. A proof, in the traditional understanding, is an argument that a competent reader can follow and be convinced by. No human has ever verified 1,936 reducibility checks by hand, and no human ever will.

The philosopher Thomas Tymoczko argued in 1979 that the proof introduced empirical elements into mathematics: to accept it you have to accept that a particular machine ran a particular program correctly, which is a claim about the physical world rather than about logic. Others countered that a 741-page hand proof would also be beyond any single reader's ability to verify with confidence, and that the machine is more reliable than the reader.

The errors were real

The critics had a concrete point, not just a philosophical one. The original proof contained errors. Appel and Haken published corrections. Further errors were found in 1986 and addressed. In 1989 they published a book-length version with a full corrected discussion.

In 1996, Neil Robertson, Daniel Sanders, Paul Seymour and Robin Thomas produced a new proof with a smaller unavoidable set of 633 configurations and a cleaner discharging argument. They wrote that they had been unable to verify the Appel-Haken proof themselves, which was part of their motivation. Their version was still computer-dependent, but the computation was far more tractable and the human-readable part far shorter.

Gonthier closes it

In 2005, Georges Gonthier at Microsoft Research Cambridge completed a fully formal proof in the Coq proof assistant, in collaboration with Benjamin Werner. Every step, including the case checks, was verified by Coq's kernel.

This changed the epistemic situation substantially. The question is no longer whether a program that nobody has audited computed correctly. It is whether Coq's kernel, a small and heavily scrutinised piece of code, is correct, and whether the formal statement in Coq says what the English statement says. Both are checkable by a person in a reasonable amount of time.

Gonthier's work is also where the modern connection to AI-assisted mathematics starts. The techniques he developed for managing a formalisation of that size, particularly the small-scale reflection approach that became the SSReflect library, fed directly into Mathlib and the current generation of formalisation projects.

What this predicted

The four colour theorem was the first case of a pattern that is now common. A result is established by a computation too large for human inspection. The mathematical community accepts it provisionally, argues about its status, and eventually a formal verification settles the question of correctness while leaving the question of understanding open.

The Kepler conjecture followed the same arc. Thomas Hales proved it in 1998 using extensive computation. The referees at the Annals of Mathematics spent four years and reported they were 99 percent certain but could not fully verify the computational parts. Hales then spent over a decade leading the Flyspeck project to formalise it, completing in 2014.

The Boolean Pythagorean triples problem produced a 200-terabyte SAT solver proof in 2016, which was subsequently checked by a verified checker.

In every case the argument recurs: we now know it is true, and we still do not know why. Nobody can look at the four colour theorem's proof and extract an idea about planarity that explains the number four. The proof establishes the fact without illuminating it.

That is the real complaint, and it has never been answered. It is also the exact complaint people are now making about proofs produced by machine learning systems, which suggests the argument has another few decades to run.

The Benchmark Problem: Why Math Scores Keep Going Up and Meaning Less

2026-07-25T00:00:00.000Z

In 2021 a language model that scored 55 percent on GSM8K was a research result. By 2024 the benchmark was effectively saturated, with frontier systems above 95 percent. The natural reading is that models got roughly twice as good at grade-school arithmetic word problems in three years.

The natural reading may be right. It is also not something anyone can verify, and the reasons why are worth understanding in detail, because they apply to every benchmark number you will ever read.

The benchmarks

GSM8K is 8,500 grade-school word problems written by human contractors, released by OpenAI in 2021. Two to eight steps of elementary arithmetic each. It was designed to be easy for a person and hard for the models of the time.

MATH is 12,500 competition problems from AMC, AIME and similar sources, released by Dan Hendrycks and colleagues, graded by difficulty from one to five. Full worked solutions in LaTeX. Considerably harder.

MMLU includes mathematics subsets alongside 57 other subjects. AIME problems from recent years get used as a rolling benchmark because new ones appear annually. FrontierMath, released by Epoch AI in late 2024, consists of unpublished research-level problems specifically designed to resist the contamination problem. GPQA does something similar for graduate science.

What contamination means

A benchmark measures generalisation only if the model has not seen the test items. Language models are trained on scraped web text. GSM8K is on GitHub, on Hugging Face, in dozens of papers, in blog posts explaining individual problems, in forum threads where people paste problems and discuss solutions.

There is no clean way to exclude all of that. Model developers run decontamination filters, typically looking for n-gram overlap between training documents and test items. These filters catch exact copies. They do not catch a paraphrase, a translation, a problem with the numbers changed, or a discussion that states the answer without quoting the question.

Nor do they catch the fact that a benchmark's distribution leaks even when its items do not. If a thousand blog posts explain how to solve GSM8K-style problems, a model trained on them learns the genre. That is arguably legitimate learning. It also means the benchmark no longer measures what it was designed to measure.

The evidence that this is happening

Several studies have tried to separate genuine capability from memorisation, and the results are consistent enough to take seriously.

The GSM1K study from Scale AI in 2024 built a new benchmark matching GSM8K's distribution as closely as they could, with fresh problems written to the same specification. They then compared model performance on the two. Some model families performed nearly identically, which is the expected result if capability is real. Others dropped by up to thirteen percentage points. The drops correlated with how likely a model was to reproduce GSM8K items verbatim when prompted with their opening lines.

The GSM-Symbolic work from Apple researchers in 2024 took GSM8K problems and generated variants by changing names and numbers through templates. Performance dropped across every model tested. They also added a variant, GSM-NoOp, that inserted a clause which was topically relevant but logically irrelevant to the answer. Performance fell sharply, in some cases by more than half. A model that understood the problem would ignore an irrelevant clause. Many models incorporated it into the arithmetic.

Functional benchmarks, where each item is a template instantiated with fresh values at evaluation time, show similar gaps. The size of the gap is the closest thing we have to a contamination estimate.

Why this is nobody's fault exactly

It would be easy to read this as developers gaming benchmarks. Mostly they are not. The structural problem is that a public benchmark is a public document, and any public document becomes training data for the next model. A benchmark begins to decay the moment it is published.

The obvious fix, a private held-out test set, has its own failure mode. If you cannot inspect the items, you cannot check that they are correctly labelled, that the problems are well-posed, or that the grading is fair. Several public benchmarks have known error rates in their answer keys, in the range of one to five percent, which puts a ceiling on meaningful scores that nobody accounts for when reporting 97 versus 98.

There is also a subtler pressure. Benchmark scores drive funding, hiring, publication and coverage. Nobody has to cheat for that pressure to distort what gets optimised. If a metric is the target, work flows toward the metric.

How to read a benchmark claim

A few questions cut through most of the noise.

When was the benchmark released relative to the model's training cutoff? A model trained on data through 2025 evaluated on a benchmark published in 2021 tells you very little. The same model on a benchmark published after its cutoff tells you a great deal.

Was the evaluation run with the same prompting, tooling and sampling as the comparison points? Reported numbers often differ by ten points depending on whether tools were available, how many samples were taken, and whether the best of several attempts was scored. A pass-at-64 number and a pass-at-1 number are not comparable and are frequently presented side by side.

Is there a perturbed variant? If a paper reports performance on both the original benchmark and a symbolically perturbed version, and the gap is small, that is meaningful evidence. If the paper only reports the original, the number is an upper bound.

What is the human baseline, and was it measured the same way? Many benchmarks quote a human baseline collected under different conditions, sometimes from crowdworkers with a time limit, sometimes from experts with none.

What is being done

FrontierMath took the approach of commissioning new, unpublished problems from research mathematicians and keeping most of them private. Early frontier model performance was in the low single digits, which was itself informative: the gap between competition mathematics and research mathematics is much wider than benchmark saturation suggested.

Live benchmarks that draw fresh items from ongoing competitions, such as AIME and Codeforces, get around contamination for as long as they stay ahead of training cutoffs. They have a natural sample-size limit, since a competition produces fifteen problems a year and the variance on fifteen items is large.

Formal verification benchmarks like miniF2F and the Lean-based evaluations sidestep the grading problem entirely, because a proof either type-checks or it does not. They do not sidestep contamination.

The reasonable position

None of this means benchmark progress is fake. Models really did get much better at mathematics between 2021 and 2026, and anyone who has used them across that period can feel the difference without consulting a table.

It means the numbers are softer than they look, that differences of a few points between systems are usually noise, and that a jump from 92 to 96 on a saturated benchmark is not evidence of anything in particular. The useful signal now comes from problems that did not exist when the model was trained. Everything else is a measurement of how much of the internet ended up in the weights.

The Collatz Conjecture and the Limits of Brute Force

2026-07-21T00:00:00.000Z

Take any positive integer. If it is even, halve it. If it is odd, triple it and add one. Repeat. The conjecture is that you always reach 1.

Start at 27 and the sequence climbs to 9,232 before descending, taking 111 steps. Start at 97 and it takes 118. Start at anything anyone has ever tried and it reaches 1.

Paul Erdős said mathematics is not yet ready for such problems, and offered 500 dollars for a solution. Jeffrey Lagarias, who has written the standard surveys, has advised people not to work on it. The problem is genuinely, unusually hard, and it is hard in a way that illustrates something important about what computation can and cannot settle.

What has been verified

Distributed computing projects have verified the conjecture for every starting value up to roughly 2 to the power of 68, which is about 295 quintillion. The verification uses clever optimisations: you only need to check that a number's trajectory drops below its starting value, since everything smaller has already been confirmed, and large classes of residues can be eliminated by modular arguments before any iteration happens.

That is an extraordinary amount of checking. It also establishes nothing about the conjecture.

The reason is simple and worth stating plainly. There are infinitely many positive integers. Any finite verification, no matter how large, covers a proportion of them equal to zero. The set of unchecked cases after verifying 2 to the 68 values is exactly as infinite as the set was before anyone started.

When brute force does work

This is not an argument that computation is useless in number theory. It is an argument about the logical structure of a claim.

Computation settles a universal claim when the search space is finite, or can be reduced to a finite one. The four colour theorem was settled by computation because the discharging argument reduced infinitely many planar graphs to 1,936 configurations. That reduction is the mathematics; the computation is bookkeeping.

Computation also settles an existential claim, by producing an example. Euler conjectured that no fourth power is the sum of three fourth powers. Noam Elkies found a counterexample in 1988, and Roger Frye then found the smallest one by direct search. One example, and a 200-year-old conjecture is dead.

The Pólya conjecture, that most numbers below any bound have an odd number of prime factors, held for every value anyone checked for decades and fails first at 906,150,257. The Mertens conjecture was disproven in 1985 without anyone producing an explicit counterexample; the smallest one is believed to be beyond 10 to the power of 30 and possibly far beyond.

Those cases are the reason "verified up to a large bound" carries so little weight. Counterexamples to plausible conjectures are known to hide at scales no search will reach.

What we do know about Collatz

The problem is not untouched. There are real partial results, and they show what progress looks like when a full proof is out of reach.

Terence Tao proved in 2019 that almost all Collatz orbits reach a value smaller than any prescribed function that goes to infinity, however slowly. Informally, almost every starting number eventually gets small. The technique routes through a probabilistic model of the iteration and a transfer to the actual deterministic sequence.

This is genuinely close, and the gap is instructive. "Almost all" here means with logarithmic density one, which permits an exceptional set. A single starting value with a divergent orbit, or a single cycle other than the trivial one, would falsify the conjecture while being invisible to a density argument.

Other constraints are known. Any non-trivial cycle must have length in the hundreds of millions at minimum, from bounds derived through continued fraction approximations to the logarithm of 3 over the logarithm of 2. The count of starting values below a bound that reach 1 grows at least as fast as a specific power of that bound.

Why it resists

The generalised problem is undecidable. John Conway showed in 1972 that a natural generalisation, where the multiplier and offset vary by residue class modulo some fixed number, produces a system that can simulate arbitrary computation. There is no algorithm that decides, for an arbitrary such system, whether all orbits reach 1.

Collatz itself is one specific instance, so undecidability of the general family does not imply that Collatz is undecidable. But it explains the difficulty. The iteration mixes two structures that do not interact cleanly: division by two is transparent in base two, multiplication by three plus one is transparent in base three, and there is no representation in which both are simple. The sequence behaves like a pseudorandom process while being entirely deterministic, and there is no known handle on it.

What AI has and has not done here

Several groups have applied machine learning to Collatz-adjacent questions, mostly in the mode of searching for structure rather than searching for proofs. Neural networks have been trained to predict stopping times from binary representations, with moderate success, which is unsurprising since the first several steps are determined by the low-order bits.

More interestingly, automated conjecture-generation systems have produced candidate lemmas about the iteration, some of which turned out to be provable and none of which turned out to be useful. That distinction matters. A system that produces true statements is not the same as a system that produces progress. Mathematics is full of true statements nobody needs.

The formalisation angle has been more productive. Parts of the known partial results have been formalised in Lean, which does not advance the problem but does make the existing literature checkable. There is an active project to formalise Tao's almost-all result.

The lesson to carry

When you see a claim that something has been verified computationally, ask what logical form the claim has.

If it is existential, a computation can settle it outright and one example is a proof.

If it is universal over a finite domain, a computation can settle it, and the interesting work was the argument that reduced the domain to something finite.

If it is universal over an infinite domain, a computation can only refute it, never confirm it. Verification up to any bound is evidence about where to look, not evidence about whether the statement is true. It is entirely reasonable to find such evidence persuasive, since a conjecture that survives 2 to the 68 tests is more likely true than one that does not. It is not reasonable to call it settled.

Erdős was probably right that we are not ready. The useful part is understanding precisely which readiness is missing, and it is not compute.

How to Check an AI's Mathematics Without Being a Mathematician

2026-07-17T00:00:00.000Z

The central asymmetry in mathematics is that checking an answer is usually far easier than finding it. Factoring a large number is hard; multiplying the factors back is trivial. Finding an integral is hard; differentiating the result is mechanical. Constructing a proof is hard; following one is manageable.

That asymmetry is what makes AI-generated mathematics usable despite being unreliable. You do not need to be able to produce the derivation. You need a procedure for testing it. Here is one.

Step one: check the units and the type

Before looking at any arithmetic, check that the answer is the right kind of object.

If the question asked for a probability and the answer is 1.4, stop. If it asked for a length and the answer carries units of area, stop. If it asked for an integer count and the answer is 6.5, stop. If it asked for a function and the answer is a number, stop.

This sounds too simple to be worth a step. In practice it catches a meaningful fraction of errors, because type violations are exactly the kind of mistake a system optimising for plausible-looking text makes. The output is shaped like a solution. Whether it is the right shape is a separate question that nothing in the generation process enforced.

Dimensional analysis is the physics version of the same idea and it is more powerful than it gets credit for. Any physical formula must be dimensionally consistent, and checking consistency requires no understanding of the derivation at all.

Step two: test the boundaries

Take the answer and evaluate it at the extreme cases where you know what should happen.

If the answer is a formula in n, what does it give for n equal to 0, 1, or 2? You can usually work out those cases by hand, sometimes by direct enumeration. If a combinatorics formula gives 3 when the answer for n equal to 1 is obviously 1, the formula is wrong regardless of how elegant it looks.

If the answer is a formula involving a parameter, what happens as the parameter goes to zero or infinity? Does the behaviour match what the problem demands? A probability that goes to 2 as some parameter grows is a wrong probability.

If the answer involves a limit or an approximation, check the case where the approximation should be exact.

This step is close to free and it is the single most productive check available. Most incorrect closed-form expressions fail at n equal to 1.

Step three: substitute back

For anything that is the solution to an equation, put it in the equation.

Solved for x? Substitute and check both sides agree. Found an integral? Differentiate it. Found an eigenvector? Multiply by the matrix. Found a root of a polynomial? Evaluate the polynomial there.

This is complete verification, not a heuristic. If the substitution works, the answer is correct for that equation, full stop. There is no need to look at the derivation at all.

The failure mode to watch is a model that presents the substitution check itself and reports that it worked when it did not. Do the substitution yourself, or better, have a tool do it.

Step four: use a second system, not a second prompt

Asking the same model whether its answer is correct is nearly worthless. It will usually agree with itself, and when it disagrees the disagreement carries almost no information.

Asking a different system is better but not much better, since models trained on overlapping data share failure modes.

Asking a fundamentally different kind of system is what actually works. Put the expression into a computer algebra system. Write six lines of Python and check numerically. Run the combinatorial enumeration for small cases. These tools have no opinion about what looks plausible; they compute.

A worked example. Suppose a model tells you the sum of the first n cubes equals the square of the sum of the first n integers. That is true, but you should not take it on trust. Three lines of Python settles it for the first thousand values in under a second, and while that is not a proof, it converts an unverified claim into a claim you would bet on.

Step five: check the reasoning only if the answer survives

This ordering is deliberate. Reading a derivation is slow and it is where you are most likely to be persuaded by fluency. Do it last, after the cheap checks have already had a chance to fail.

When you do read, look for specific things rather than reading for overall coherence.

Watch for a step where a general claim is applied to a case that does not satisfy its conditions. This is the most common substantive error: invoking a theorem correctly stated but wrongly applicable, dividing by something that could be zero, taking a square root and silently discarding the negative branch, swapping the order of two limits.

Watch for a step that asserts something without justification and where the assertion is doing real work. Phrases like "it is clear that", "one can show", and "by symmetry" are sometimes correct shorthand and sometimes a gap. Ask specifically about them.

Watch for an answer that changes between the working and the summary. Models produce a derivation and then a concluding line, and these occasionally do not match.

The three sentences that matter

When you have a specific step you doubt, the useful prompts are narrow and adversarial.

"Justify the step from line four to line five in full detail, citing the specific theorem used and verifying its hypotheses hold here." This forces the model onto the specific claim rather than letting it restate the argument.

"Give a counterexample to the following claim, or prove no counterexample exists." Asking for a counterexample is a different search than asking for a proof and often surfaces the flaw.

"Verify this numerically with code and show the output." Any interface with code execution turns this into an actual check rather than another generated assertion.

Where this procedure fails

It fails on existence proofs that produce no object. If the claim is that some structure exists, and the proof is non-constructive, there is nothing to substitute back.

It fails on asymptotic claims. A statement about behaviour as n goes to infinity cannot be tested at n equal to 5, and the small cases can actively mislead. The Pólya conjecture holds for every value below nine hundred million.

It fails on anything requiring the full strength of the argument, which is most of research mathematics. At that level there is no substitute for reading, and the reading requires expertise.

For the vast majority of practical mathematics, homework, engineering calculations, statistical reasoning, financial modelling, the procedure is adequate. The answer either survives boundary checks and substitution or it does not, and the derivation matters much less than people assume.

The habit worth building

Treat every generated derivation as a conjecture with an attached argument, in that order. The conjecture is the part you test. The argument is context that helps you test it and occasionally teaches you something.

This is not a special posture for machines. It is what careful people already do with any result they did not derive themselves, including results from colleagues and from textbooks. The only thing that changed is the volume.

P versus NP for People Who Write Code

2026-07-13T00:00:00.000Z

The usual popular framing is that P versus NP asks whether problems that are easy to check are also easy to solve. That is accurate and it is not enough to work with. Here is the version that survives contact with actual programming.

The setup

A decision problem is a question with a yes or no answer, parameterised by an input. Does this graph have a Hamiltonian cycle? Is this Boolean formula satisfiable? Does this set of integers contain a subset summing to zero?

P is the class of decision problems solvable by an algorithm whose running time is bounded by a polynomial in the input size. Sorting is in P. Shortest path is in P. Primality testing is in P, proven in 2002 by Agrawal, Kayal and Saxena, which was a surprise at the time.

NP is the class of decision problems where a yes answer has a certificate that can be checked in polynomial time. If a graph has a Hamiltonian cycle, the cycle itself is a certificate, and checking it is a linear scan. If a formula is satisfiable, the satisfying assignment is the certificate.

Note what NP does not mean. It is not "non-polynomial". It stands for nondeterministic polynomial time, and the definition is about the existence of a checkable certificate, not about hardness. Everything in P is in NP, since you can ignore the certificate and just solve the problem.

The question is whether the inclusion is strict. Is there a problem with an efficiently checkable certificate but no efficient algorithm?

Why anyone cares

Because of completeness. Stephen Cook in 1971 and Leonid Levin independently showed that Boolean satisfiability is NP-complete: every problem in NP reduces to it in polynomial time. Richard Karp then produced 21 more NP-complete problems in 1972, and the list now runs into the thousands.

Complete means maximally hard within the class. If any one NP-complete problem has a polynomial algorithm, they all do, and P equals NP. If any one of them provably does not, none of them do.

So the thousands of NP-complete problems, which span scheduling, routing, packing, circuit design, protein folding models, game solving and puzzle generation, all stand or fall together. That is why the question is worth a million dollars and why it is one of the seven Clay Millennium Problems.

What the answer would mean

If P equals NP, with a practical algorithm, most modern cryptography breaks. Public key systems rely on the gap between finding a factorisation and checking one. So does much of the security infrastructure of the internet. On the other side, optimisation problems that are currently approximated would become exactly solvable, which would reshape logistics, chip design, and drug discovery. Scott Aaronson has argued that it would also mean mathematics itself becomes largely automatable, since finding a proof of bounded length is an NP search.

Almost nobody expects this. Informal polls of theoretical computer scientists put the fraction expecting P not equal to NP somewhere above 80 percent, and rising over time.

If P does not equal NP, which is the expected answer, nothing about daily practice changes. We would have a proof of something everybody already assumed.

Why it is so hard to settle

Three barriers are known, and each one killed a promising line of attack.

Relativisation, from Baker, Gill and Solovay in 1975. There exist oracles relative to which P equals NP, and other oracles relative to which it does not. Any proof technique that works the same way when the machines have access to an oracle cannot settle the question, because it would have to prove both. That rules out the diagonalisation arguments that had settled earlier separation questions.

Natural proofs, from Razborov and Rudich in 1994. Most circuit lower bound techniques share two properties: they apply to a large fraction of all functions, and the property they use can be checked efficiently. Razborov and Rudich showed that any proof technique with both properties would also break the pseudorandom generators that are widely believed to exist. So either those generators do not exist, which would itself be a shock, or this whole family of techniques cannot work.

Algebrization, from Aaronson and Wigderson in 2008, extended the relativisation barrier to cover algebraic techniques that had been developed specifically to get around it.

Each barrier does not say the question is unanswerable. It says a particular kind of argument cannot answer it. What remains is techniques that are non-relativising, non-naturalising and non-algebrising, and nobody has much idea what those look like.

The practical situation, which is stranger

Here is what a working programmer should actually take from this.

NP-hardness is a worst-case statement. It says no algorithm is fast on every instance. It says nothing about the instances you have.

Modern SAT solvers routinely handle industrial instances with millions of variables. Chip verification, dependency resolution in package managers, hardware model checking, and automated planning all run on SAT and SMT solvers in production. The problem is NP-complete and the solvers work anyway, because real instances have structure that the solvers exploit through conflict-driven clause learning, restarts and good heuristics.

Integer programming is NP-hard and commercial solvers handle enormous scheduling and routing problems daily. The travelling salesman problem is NP-hard and the Concorde solver has found provably optimal tours through tens of thousands of cities.

So the correct reading of "your problem is NP-hard" is not "give up". It is "there is no algorithm guaranteed fast on all inputs, so find out whether your inputs are adversarial or merely large". Usually they are merely large.

The genuinely hard instances tend to be constructed rather than encountered. Random instances near the satisfiability threshold are hard. Cryptographic instances are hard by design. Instances arising from a logistics problem at a real company are usually not.

Where machine learning enters

Two connections, both active.

The first is heuristic learning. SAT solver performance depends heavily on branching heuristics, which decide which variable to assign next. This is a decision problem with a clear objective and enormous amounts of available training data, and learned heuristics have produced measurable gains. The solver remains complete and its answers remain exact; only the search order is learned. This is the same architecture as AlphaGeometry: neural guidance, symbolic guarantee.

The second is the reverse direction. NP-hardness results are used to characterise what neural networks can and cannot do. Training a small network to optimality is NP-hard. Verifying that a network satisfies a robustness property is NP-hard. These results shape what verification tooling is realistic.

The claims to distrust

Every year brings several claimed proofs. Gerhard Woeginger maintained a list that reached over a hundred entries before he stopped updating it, split roughly evenly between claims in each direction, including several people who claimed both at different times.

The base rate on these is not encouraging, and the reason is the barriers. A proof that does not explain how it evades relativisation and natural proofs is almost certainly wrong, and the author usually has not addressed either. Language models, asked about the problem, will occasionally produce confident-sounding arguments in one direction or the other. Those arguments are worth exactly what any other unrefereed claimed proof of a Millennium Problem is worth.

If you want a single sentence to carry away: the question is open, the expected answer changes nothing practical, the barriers explain why fifty years of effort has not settled it, and NP-hardness in your own work is a statement about the worst case that your actual inputs probably do not achieve.

The Busy Beaver and What Cannot Be Computed

2026-07-09T00:00:00.000Z

Consider a Turing machine with n states, a two-symbol alphabet and a tape that starts blank. Some of these machines halt. Most run forever. Among the ones that halt, some run longer than others.

Define BB(n) as the maximum number of steps any halting n-state machine takes before stopping. This is the busy beaver function, introduced by Tibor Radó in 1962.

BB(1) is 1. BB(2) is 6. BB(3) is 21. BB(4) is 107.

BB(5) is 47,176,870.

BB(6) is greater than 10 raised to the power of 10 raised to the power of 10 raised to the power of 10 raised to the power of 7, and that is only a lower bound from a specific machine somebody found.

Why the growth is not just fast

Any function you can write down with a computer program is eventually exceeded by the busy beaver function. Not eventually exceeded by a bit. Exceeded permanently and by an unbounded margin.

The proof is short. Suppose f is computable and f(n) is at least BB(n) for all n. Then you could decide the halting problem: to determine whether an n-state machine halts, compute f(n), run the machine for that many steps, and if it has not halted by then it never will. But the halting problem is undecidable, so no such f exists.

That is the whole argument, and it is worth sitting with. The busy beaver function is not hard to compute in the way that factoring is hard. It is uncomputable, in the strict sense that no algorithm produces it. The values are perfectly well defined. Each BB(n) is a specific finite integer. There is simply no procedure that finds them all.

The connection to open problems

Because busy beaver values encode halting behaviour, they encode mathematics.

You can build a Turing machine that searches for a counterexample to Goldbach's conjecture and halts if it finds one. Whether that machine halts is equivalent to whether Goldbach is false. Someone has built such a machine with 27 states. So knowing BB(27) would settle Goldbach: run every 27-state machine for BB(27) steps, and if the Goldbach machine has not halted, no counterexample exists.

The same trick has been done for the Riemann hypothesis, with a machine in the low hundreds of states, later reduced considerably.

More dramatically, Yuri Matiyasevich, Stefan O'Rear and Scott Aaronson's collaborators constructed a machine of 748 states, later improved to 745, that halts if and only if Zermelo-Fraenkel set theory is inconsistent. Since ZFC cannot prove its own consistency, ZFC cannot determine whether that machine halts. Therefore ZFC cannot determine BB(745). The value exists; standard mathematics cannot reach it.

The threshold where busy beaver values become independent of ZFC is somewhere between 6 and 745, and narrowing it is an active pursuit.

BB(5), settled in 2024

For forty years the fifth value was a conjecture. Heiner Marxen and Jürgen Buntrock found a five-state machine running 47,176,870 steps in 1989, and nobody found anything longer, but proving that no five-state machine runs longer requires eliminating every non-halting candidate.

There are roughly 17 trillion five-state machines before symmetry reduction, cut to about 180 million distinct cases. Most are dispatched quickly. The difficulty is the residue: machines whose non-halting is not obvious and requires a proof.

The bbchallenge project, an open online collaboration begun in 2022, worked through them. Contributors built deciders, programs that prove a class of machines never halts, each covering a different behavioural pattern. Cyclers, translated cyclers, machines whose tape configuration satisfies a closed-form invariant, machines analysable through a finite automaton reduction.

A handful of machines resisted every automated decider and needed individual analysis. The most famous is Antihydra, a machine implementing a Collatz-like iteration on integers, whose non-halting depends on a statement about the distribution of a particular sequence that nobody has proven. It was ultimately handled by a bespoke argument.

In July 2024 the project announced that BB(5) equals 47,176,870, with the entire proof formalised in the Coq proof assistant. The formalisation matters here more than usual. The proof is a collection of programs run over hundreds of millions of cases, exactly the situation where a subtle bug would be invisible. Formal verification converts a claim about software into a claim checked by a kernel.

BB(6) is out of reach

The sixth value will not be determined. The current lower bound, established through analysis of a specific machine in 2025, involves a tower of exponentials.

The obstruction is not compute. It is that six-state machines can implement iterations whose behaviour depends on open number-theoretic questions. Several known six-state candidates run Collatz-like processes. Deciding whether they halt would require settling those processes, and nobody can.

This is the practical face of undecidability. Not an abstract limit at some distant horizon, but a wall that appears at six states and a two-symbol alphabet, which is about as small as a computational system can be.

Why this belongs in a discussion about AI

Because it draws a boundary that no amount of scaling crosses.

There is a persistent framing in which AI capability is a matter of resources: more parameters, more data, more compute, more capability. Within the space of computable functions that framing is roughly right, and progress has been startling.

The busy beaver function is outside that space entirely. No system that runs as a computation can produce it, regardless of size or architecture. A language model asked for BB(7) can output a number. That number will be wrong, and no training procedure could make it right, because the mapping from n to BB(n) is not a computable function to approximate.

This is not a claim that AI is limited in some vague philosophical way. It is a specific mathematical fact with a two-line proof, and it applies equally to human mathematicians running as physical processes.

What it does mean is that the discourse about machines eventually solving all mathematical problems has a well-defined false component. There are true arithmetic statements not provable in any given consistent formal system, that is Gödel, and there are specific finite integers that no algorithm computes, that is Radó. Both were established before the first neural network was trained and neither has an escape hatch.

The busy beaver function is the cleanest illustration because it is so concrete. BB(745) is a particular integer. You could write it down if you knew it. Nothing that computes will ever tell you what it is.

Symbolic or Neural: Choosing the Right Machine for the Job

2026-07-05T00:00:00.000Z

There are two kinds of mathematical software and they are almost perfect opposites.

A computer algebra system executes an algorithm. Ask Mathematica or SymPy or Maxima to factor a polynomial and it runs a factorisation algorithm whose correctness was proven before it was implemented. The answer is right. If the system cannot solve your problem, it says so, or it runs until you stop it.

A language model produces the most likely continuation of your text. Ask it to factor a polynomial and it produces something shaped like a factorisation. Usually correct for small cases. Sometimes confidently wrong. Never explicitly unable.

Both are useful. They are useful for different things, and the most common expensive mistake is using one where the other belongs.

What symbolic systems are actually good at

Exact manipulation of well-specified expressions. Integration, differentiation, series expansion, polynomial arithmetic, linear algebra over exact fields, solving systems of equations, simplification, number theory, combinatorial enumeration.

These are areas with decision procedures. The Risch algorithm decides elementary integrability. Gröbner bases decide membership in a polynomial ideal. Cylindrical algebraic decomposition handles quantifier elimination over the reals. Each of these is a theorem-backed procedure that terminates with a correct answer or a correct declaration that no answer of the requested form exists.

The value is not just correctness. It is that failure is visible. If SymPy cannot integrate your function, it returns an unevaluated integral. You know you did not get an answer. That property is worth a great deal and it is exactly what the alternative lacks.

Where symbolic systems fall down

The input has to be formal. Every symbol declared, every assumption stated, every domain specified. If you forget to tell the system that x is positive, it will refuse to simplify the square root of x squared to x, correctly, and you will spend twenty minutes wondering why.

Expression swell is the other chronic problem. Intermediate results in symbolic computation grow explosively even when the final answer is small. A determinant of a modest symbolic matrix can produce intermediate terms that exhaust memory. Gröbner basis computation is doubly exponential in the worst case and reaches that worst case on ordinary-looking inputs.

And they cannot read. A problem stated in English is not input. Translating a paragraph into a system of equations is exactly the step a computer algebra system does not perform.

What language models are actually good at

Translation between informal and formal. That is the core competence and everything else follows from it.

Given a paragraph describing a physical situation, produce the equations. Given a table of data, propose which statistical test applies. Given a vague sense of what you want, produce the Mathematica syntax that expresses it. Given an error message, explain what the system was objecting to.

They are also good at recall across an enormous surface. Which theorem covers this case, what is this identity called, where did this technique come from, what is a standard approach to this genre of problem. This is retrieval with fuzzy matching, and it beats searching for the name of something you do not know the name of.

And they are good at explanation. Taking a correct derivation and rendering it at a chosen level of detail is genuinely useful and low-risk, because the derivation is already known correct.

Where language models fall down

Long exact computation, for architectural reasons covered elsewhere. Consistency across a long derivation, where an error in step three propagates silently. Anything requiring the system to know it does not know.

That last one is the expensive failure. A computer algebra system that cannot do something returns an unevaluated expression. A language model that cannot do something returns a wrong answer in the same tone as a right one. The output distribution does not contain an "I cannot" for most mathematical questions, because the training text did not contain many people saying so.

The pairing that works

Use the model to reach a formal statement. Use the symbolic system to evaluate it. Use the model again to interpret the result.

In practice this means asking for code rather than answers. Instead of asking for the eigenvalues of a matrix, ask for the three lines of SymPy that compute them, then run those lines. You have then used the model for the part where it is strong, translation, and the algorithm for the part where it is strong, computation.

This is not a workaround. It is the correct division of labour, and it is what every serious tool-using deployment does. When an assistant with code execution answers a numerical question, it is running this pattern internally.

A decision table

Task	Use
Exact integral, derivative, series	Computer algebra system
Arithmetic on numbers longer than a few digits	Calculator or interpreter
Turning a word problem into equations	Language model, then verify the equations
Solving those equations	Computer algebra system
Finding the name of a theorem you half-remember	Language model
Checking whether a claimed identity is true	Numerical evaluation at random points
Explaining a known-correct proof at a different level	Language model
Producing a proof you will rely on	Proof assistant, with either system suggesting steps
Enumerating small cases of a combinatorial structure	Write the loop, run it
Deciding whether an approach is worth trying	Language model, as a cheap prior

The row that gets ignored most often is the third. People ask for the answer rather than the setup, and the setup is where the model adds value and where a human can still check the work.

Numerical verification deserves its own note

For any claimed identity between expressions, evaluate both sides at several random points. If they disagree at one point, the identity is false and you are done. If they agree at twenty random points to fifteen decimal places, the identity is not proven, but the probability of coincidence is negligible for anything you would encounter in practice.

This takes about four lines of code and catches most errors from either kind of system. It is underused relative to how cheap it is.

Two cautions. Choose points away from singularities and branch cuts, since complex-valued functions with branch cuts will legitimately disagree in regions where the identity holds only on a principal branch. And use enough precision that floating point error does not masquerade as disagreement.

The general shape

Every reliable mathematical workflow built on modern tooling has the same structure. Something creative and unreliable proposes. Something mechanical and reliable disposes.

AlphaGeometry does it with a language model and a deduction engine. Proof search in Lean does it with a neural tactic suggester and a type checker. A working engineer does it with a chatbot and a Jupyter notebook.

The pattern is identical at every scale, and the mistake is always the same: letting the proposing component's output through without passing it to the disposing component.

Machines That Generate Conjectures

2026-07-01T00:00:00.000Z

Most discussion of AI in mathematics concerns proof: given a statement, find an argument. That framing skips the part working mathematicians spend most of their time on, which is deciding what statement to consider in the first place.

A conjecture is a bet. It has to be plausible enough to be worth effort, hard enough to be worth stating, and connected to something that matters. Producing true statements is easy; a computer can produce them forever. Producing true statements that anyone should care about is the whole difficulty.

Several systems have tried, with results that are genuinely mixed in an instructive way.

Graffiti

Siemion Fajtlowicz began writing Graffiti in the 1980s. It looks at graph invariants, quantities like the independence number, the chromatic number, the average distance between vertices, and proposes inequalities between them. It tests each candidate against a database of graphs and discards anything with a counterexample.

Then it applies filters designed to discard uninteresting survivors. If a proposed inequality follows immediately from one already proposed, drop it. If it is weaker than something already on the list, drop it. Fajtlowicz called these the Dalmatian heuristic and spent considerable effort on them, because without them the output is unusable.

Graffiti's conjectures produced published papers. Several dozen, by mathematicians including Fan Chung, Ronald Graham and Paul Erdős. A number were proven, a number refuted, and the refutations were often as interesting as the proofs because they exhibited graphs nobody had thought to construct.

The lesson from Graffiti is that the generation step was never the hard part. Fajtlowicz's real contribution was the filtering, which encoded a great deal of human taste about what makes a graph inequality worth stating.

The Ramanujan Machine

A group at the Technion launched the Ramanujan Machine in 2019, aimed at a specific target: continued fraction expressions for mathematical constants.

Ramanujan produced formulas like continued fractions converging to expressions involving pi and e, often without proof, and often with no indication of how he found them. The project asked whether a search could find comparable formulas.

The method is numerical. Generate candidate continued fractions with polynomial coefficient patterns. Evaluate them to high precision. Compare against a table of constants and their simple algebraic combinations. When a match appears to many digits, you have a conjecture.

It found several previously unknown formulas, including expressions for the Catalan constant and for combinations involving pi and e. Some were subsequently proven, a few by the group and a few by outside mathematicians who read the output.

The honest assessment is that these are attractive identities rather than important ones. They do not unlock anything. But the project made a clean methodological point: high-precision numerical search over a structured space finds relationships that nobody would find by hand, and integer relation algorithms like PSLQ make the matching step reliable.

That technique predates the project. David Bailey, Peter Borwein and Simon Plouffe used it in 1995 to find the BBP formula for pi, which allows computing the nth hexadecimal digit without computing the preceding ones. That formula was found by numerical search and proven afterward, and it changed what people believed was possible about digit extraction.

Deep learning as a microscope

A different approach appeared in a 2021 Nature paper from DeepMind working with Geordie Williamson and with András Juhász and Marc Lackenby.

Here the model does not propose the conjecture. It is used to detect whether a relationship exists at all.

The procedure: take two collections of mathematical objects you suspect are connected. Train a network to predict one from the other. If the network cannot learn the mapping, that is weak evidence against a relationship. If it can, use attribution methods to identify which input features carry the signal. Then a human looks at those features and tries to formulate a precise statement.

Applied to knot theory, this produced a relationship between hyperbolic and algebraic invariants of knots, specifically involving the signature and geometric quantities, which Lackenby and Juhász then proved. Applied to representation theory, it produced progress on the combinatorial invariance conjecture for Kazhdan-Lusztig polynomials, with Williamson identifying structure in the network's attention that suggested a new construction.

The framing that stuck was that the network functions as a microscope. It does not do the mathematics. It points at where the mathematics is.

This seems more durable than conjecture generation as such, because it keeps the human in the role humans are still best at, deciding what is meaningful, while using the machine for the part humans are bad at, detecting weak signal in high-dimensional data.

The pattern-matching problem

There is a persistent failure mode across all of these systems, and it is worth naming.

An automated conjecture generator will produce enormous numbers of true statements that are true for uninteresting reasons. Coincidences of small numbers. Consequences of a definition that nobody bothered to state. Special cases of known results, dressed differently enough that the system does not recognise them.

Filtering these out requires knowing the field, which means the system is only as good as the taste encoded in its filters, which means someone had to encode taste. Graffiti's success came from Fajtlowicz spending years on that encoding. Systems that skipped it produced output nobody read.

Large language models change this somewhat, because they have absorbed a great deal of implicit taste from mathematical writing. Ask a model whether a given identity is interesting and it will give an answer that correlates with what a mathematician would say, not because it has judgement but because it has read a lot of mathematicians expressing theirs.

Whether that constitutes taste or a good imitation of taste is not a question anyone can currently answer, and for practical purposes it may not matter.

What would count as a breakthrough

Not a system that produces conjectures. Several exist and they work.

A system that produces a conjecture which reframes a field. The kind of statement that makes people realise they were asking the wrong question. The Langlands programme, Weil's conjectures, the Taniyama-Shimura conjecture that turned out to imply Fermat.

Those conjectures were not found by noticing a pattern in data. They were found by someone seeing that two apparently unrelated areas had the same shape. No current system has produced anything in that category, and it is not obvious what training signal would produce one, because the whole point is that the connection was not visible in the existing corpus.

That is the frontier, and the honest position is that nobody knows whether it is reachable from here. In the meantime the microscope framing is doing real work, and there is a growing list of theorems whose discovery involved a neural network pointing at something a person had not looked at.

What Chain of Thought Really Does

2026-06-27T00:00:00.000Z

In 2022 a paper from Jason Wei and colleagues at Google showed that prompting a large language model to produce intermediate reasoning steps before its final answer raised accuracy on grade-school word problems from around 18 percent to around 57 percent. The technique was called chain-of-thought prompting. A follow-up found that appending the phrase "let us think step by step" achieved much of the same effect with no examples at all.

This got read as evidence that models can reason when encouraged to. That reading is roughly backwards, and the accurate version is more useful.

The depth constraint

A transformer processes its input through a fixed number of layers. GPT-scale models have somewhere between 50 and 150. Each layer performs a fixed amount of computation: attention over the sequence, then a feedforward transformation.

When the model generates a token, that is one pass through all the layers. Then the token is appended to the context and the next pass begins.

The consequence: the amount of computation available before emitting any single token is bounded, and the bound does not depend on how hard the question is. Asking for the answer to two plus two and asking for the answer to a twelve-step algebra problem allocate the same computational budget to the first output token.

This is a genuine, provable limitation. Transformers with fixed depth belong to a restricted circuit complexity class. There are problems, including composing a permutation many times and evaluating deeply nested boolean formulas, that cannot be solved in constant depth regardless of network width.

Some computations require a number of sequential steps that grows with input size, where each step needs the previous step's result. No amount of parallel work substitutes. Long multiplication is one. Iterating a function is another. Following a chain of dependencies through a graph is another.

What generating tokens buys

Every generated token is another full pass through the network, and the model can read everything it has already written.

So if the model writes an intermediate result, that result now exists in the context, and subsequent passes can use it as an input rather than recomputing it. The sequence of generated tokens becomes a working memory, and the number of sequential computational steps available becomes proportional to the number of tokens generated rather than to the depth of the network.

Theoretical work has made this precise. Transformers with chain of thought of polynomial length can simulate polynomial-time computation, which places them in a substantially larger complexity class than the same models answering directly. The scratchpad is not a metaphor. It is a tape.

That is what chain of thought does. It converts a depth-bounded computation into a length-bounded one, and length is much cheaper to buy.

Why the psychological framing misleads

If you believe chain of thought works because the model is thinking more carefully, several wrong predictions follow.

You would expect the written reasoning to describe what the model actually did. It often does not. Studies on faithfulness have shown that models produce reasoning chains that do not reflect the factors driving their answers. In one line of work, inserting a bias into the prompt, such as always marking option A as correct in the examples, changed the model's answers while the stated reasoning never mentioned the pattern and instead constructed a justification for the biased answer.

You would expect encouraging phrases to help through some motivational channel. They help through a distributional channel: the phrase shifts the model toward text that resembles worked solutions, and worked solutions contain intermediate steps.

You would expect that more elaborate reasoning always helps. It does not. On simple tasks, forced step-by-step reasoning sometimes reduces accuracy, apparently by introducing opportunities for error into a computation that fit in one pass anyway. There is also a documented effect where reasoning about tasks that humans perform better intuitively, such as certain visual or grammatical judgements, degrades model performance in a way that parallels the psychology literature on verbal overshadowing.

The reasoning models

Since 2024, a class of models has been trained specifically to produce long internal reasoning traces before answering, using reinforcement learning against verifiable outcomes. Mathematics and code are the natural training domains because correctness is checkable automatically.

These systems generate thousands of tokens of internal work on a hard problem, including backtracking, checking their own steps and trying alternative approaches. Performance on competition mathematics improved dramatically.

The mechanism is the same one. More tokens, more sequential computation. What changed is that the model was trained to use those tokens productively rather than prompted into it, and that the training signal came from whether the final answer was right rather than from imitating human-written solutions.

An important consequence: performance now scales with inference-time compute. Letting the model think longer produces better answers, up to a point, which is a different scaling axis from making the model larger. This has become a central design consideration.

The faithfulness problem did not go away. The visible trace is still not guaranteed to be the computation. Providers differ in whether they show it, and some show a summary rather than the raw trace.

Practical consequences

If you want a hard mathematical question answered well, give the model room. Do not ask for the answer only. Do not impose a length limit that forces compression. The tokens are the computation.

If you want to know whether an answer is right, do not read the reasoning as evidence. A well-formed chain of thought is not a proof that the process was sound. Check the answer directly by the methods that apply to any claimed result.

If you are checking a specific step, extract it and ask about it in isolation. A step embedded in a long chain gets carried by the surrounding fluency. The same step alone gets evaluated.

If the task genuinely requires exact computation, give the model a tool. Chain of thought makes arithmetic better. An interpreter makes it correct.

The honest summary

Chain of thought is a real and important technique whose effect size on mathematical tasks is large. The mechanism is that generating tokens buys sequential computation that the architecture otherwise cannot provide.

It is not introspection. The trace is output, and output is generated the same way all output is generated, by predicting likely continuations. Sometimes those continuations happen to be a faithful record of the computation. There is no mechanism guaranteeing it and there is evidence that it frequently is not.

Both things are true at once, and holding both is the whole of understanding what these systems do when they appear to reason.

The Largest Proof Ever Produced Was 200 Terabytes Long

2026-06-23T00:00:00.000Z

The question is easy to state. Can you colour every positive integer red or blue so that no Pythagorean triple, no three numbers a, b, c with a squared plus b squared equal to c squared, ends up all one colour?

Erdős asked a version of this and offered 100 dollars for an answer. In 2016 Marijn Heule, Oliver Kullmann and Victor Marek settled it. The answer is no. You can colour the integers from 1 to 7,824 successfully. At 7,825 it becomes impossible.

The proof is 200 terabytes.

How a colouring problem becomes a logic problem

Encode the question as a Boolean formula. Introduce one variable per integer, true for red and false for blue. For every Pythagorean triple within your range, add two clauses: not all three true, and not all three false.

Now the formula is satisfiable exactly when a valid colouring exists. Ask a SAT solver.

For 7,824 the solver finds a colouring. For 7,825 it proves no assignment satisfies the formula. That negative answer is the proof, and the size comes from what a proof of unsatisfiability has to contain.

Why the proof is so large

A satisfiable answer is compact. Hand over the assignment and anyone can check every clause in linear time.

Unsatisfiability has no such certificate. To establish that no assignment works, the solver must in effect account for the entire search space. Modern solvers do this through conflict-driven clause learning: when a partial assignment leads to a contradiction, the solver analyses the conflict and derives a new clause capturing why that region fails, then continues. The learned clauses accumulate.

The proof is the log of every derived clause, in a format called DRAT, which stands for deletion resolution asymmetric tautology. Each line records a clause and the fact that it follows from earlier ones by a specific inference rule.

The search for 7,825 was distributed across 800 cores on the Stampede supercomputer at Texas Advanced Computing Center, using a technique called cube and conquer. Split the problem into roughly a million subproblems by fixing some variables, solve each independently, combine. Two days of wall clock time. The combined DRAT log came to 200 terabytes, later compressed to 68 gigabytes.

The part that makes it trustworthy

A 200-terabyte log is not something a person can inspect, and neither is a 68-gigabyte one. The reason to believe it is that the log was checked by a separate, formally verified program.

The checker reads each line of the DRAT proof and verifies that the recorded clause genuinely follows from the clauses before it. This is a simple mechanical check. It knows nothing about Pythagorean triples, colourings or SAT solving. It knows resolution.

Crucially, the checker itself has been verified. Implementations exist whose correctness has been proven in Coq and in ACL2. So the trust chain is: the encoding is correct, which a human can check because it is a short piece of code; the checker is correct, which has been formally proven; the checker accepted the log, which is an empirical fact about a computation that anyone can repeat.

This is a stronger guarantee than a refereed paper offers. A referee reads a proof and forms a judgement. A verified checker reads a proof and applies a decision procedure. The 200 terabytes is not a weakness of the result. It is the cost of the guarantee.

The complaint that remains

Understanding. Nobody knows why 7,825.

There is no argument for the number. It is not a nice number. It does not factor interestingly. Its appearance is not explained by any structural feature anyone has identified. The proof establishes that the property fails there without indicating what changes.

This is the same complaint that greeted the four colour theorem in 1976 and it is exactly as unanswered. A proof can certify a fact without illuminating it, and the mathematical community has never fully decided how it feels about that.

Heule has been direct about it, describing the goal as answering questions that humans cannot answer, and treating the lack of insight as a real cost rather than a rhetorical concession.

Other results in this style

The Erdős discrepancy problem, at least in its low-discrepancy cases, was attacked the same way. Boris Konev and Alexei Lisitsa showed in 2014 that no infinite sequence of plus and minus ones has discrepancy at most two, with a SAT proof of about 13 gigabytes. The full conjecture was then proven by Terence Tao in 2015, with a human argument, partly motivated by the computational evidence.

That pairing is worth noting. The SAT result did not prove the theorem. It made people confident enough about the shape of the answer to invest in finding a real proof.

Schur number five was determined in 2017 by Heule, using two petabytes of proof and 14 CPU-years. The answer is 160.

Keller's conjecture in dimension seven was settled in 2020 by a group including Heule, again through SAT, again with a verified checker.

The pattern is now established. Take a combinatorial question with a finite search space at each size, encode it, throw a solver at it with enough symmetry breaking to make the space tractable, verify the log.

Where the AI question sits

SAT solving is not machine learning and these results involved no neural networks. But the connection is direct and increasingly practical.

Solver performance depends on heuristics: which variable to branch on, when to restart, which learned clauses to keep. These are decision problems with abundant training data and a clear objective, and learned heuristics have produced real gains. The solver stays complete and its output stays verifiable; only the search order changes.

That is the same architecture as every other successful mathematical AI system. The neural component chooses where to look. The symbolic component decides what is true. The proof is produced by the part that cannot be wrong.

There is a reasonable prediction here about the next decade. The class of mathematical questions settled by machine will keep growing, and the growth will come from better encodings and better search guidance rather than from a system that reasons its way to answers. The proofs will get larger and less readable. The verification will get more rigorous. And the gap between knowing a thing is true and understanding why will keep widening, which is a genuine loss that nobody has proposed a fix for.

Fifty Years of Multiplying Matrices, and the Week a Machine Improved It

2026-06-19T00:00:00.000Z

To multiply two 2-by-2 matrices, the definition requires eight multiplications and four additions. Every entry of the result is a sum of two products, and there are four entries.

In 1969 Volker Strassen showed it can be done with seven multiplications. His method uses more additions, which does not matter, because the construction is recursive. Apply it to blocks of a large matrix and you get an algorithm running in roughly n to the power of 2.807 rather than n cubed. For large matrices that gap is enormous.

Strassen's result was a shock. Everyone had assumed the obvious algorithm was optimal, and nobody had thought to check. It opened a research area that has run for fifty-five years and the exponent has been pushed down to around 2.371 through progressively more intricate constructions, none of which is practical.

For the specific case of 4-by-4 matrices, Strassen's method applied recursively gives 49 multiplications. That stood as the best known for over five decades.

Reframing as a game

In October 2022 a DeepMind team published AlphaTensor in Nature. Their move was to represent matrix multiplication algorithms as tensor decompositions and then treat finding a decomposition as a single-player game.

The bilinear map that multiplies two n-by-n matrices corresponds to a specific three-dimensional tensor. A decomposition of that tensor into R rank-one terms corresponds exactly to an algorithm using R multiplications. So minimising multiplications is minimising tensor rank, which is NP-hard in general and has an enormous search space.

The game: start with the target tensor. Each move subtracts a rank-one term. Reach the zero tensor and you win, with a score based on how few moves you used. This is now the same shape of problem AlphaZero handles, and the same machinery applies, with Monte Carlo tree search guided by a neural network trained through self-play.

The search space is far larger than chess or Go. The action space at each step is effectively infinite, requiring discretisation of the coefficient values. Training used synthetic examples generated by randomly decomposing tensors and presenting the result as a puzzle, which gives an unlimited supply of instances with known solutions.

The results

AlphaTensor rediscovered Strassen's algorithm for the 2-by-2 case, which is the expected sanity check.

For 4-by-4 matrices over the field with two elements, where arithmetic is modulo 2, it found a decomposition with 47 multiplications, beating the 49 that had stood since 1969. It also improved the known bounds for over 70 different matrix sizes.

It found a 5-by-5 algorithm using 96 multiplications, improving the previous best of 98.

The team also ran a variant that optimised for actual wall-clock runtime on specific hardware rather than for multiplication count, and found algorithms that ran measurably faster on TPUs and GPUs than the standard implementations, in the range of 10 to 20 percent for particular sizes.

The caveats, stated properly

The 47-multiplication result for 4-by-4 is over the field with two elements. That restriction matters. Arithmetic modulo 2 has properties that general fields do not, and the result does not automatically transfer to real or complex matrices, where 49 remained the record.

Within two weeks of publication, Manuel Kauers and Jakob Moosbauer at Johannes Kepler University Linz posted an improvement, finding a 5-by-5 decomposition with 95 multiplications using a different search method they had been developing. They were explicit that AlphaTensor's result prompted them to look again at a problem they had set aside.

Later work by the same group and others found further improvements, including a 4-by-4 algorithm over general fields using 48 multiplications, published in 2024, which finally beat Strassen's recursive bound in the general case.

So the honest history: a reinforcement learning system found a genuine improvement in a special case, and its publication triggered a burst of human work that produced further improvements by other methods within months.

What that sequence actually shows

Two readings are available and both have merit.

The deflationary reading: AlphaTensor was quickly outperformed by classical search methods that a small academic group implemented. The advance was real but narrow, and the field's response demonstrated that the problem was more tractable than fifty years of neglect had suggested.

The other reading: fifty years of neglect is the point. The problem sat because everyone assumed Strassen was optimal for small cases. A machine finding an improvement forced attention back onto it, and the subsequent human improvements would not have happened otherwise. Measuring the contribution only by the final record misses how the record moved.

Kauers has said something close to the second reading himself, noting that the AlphaTensor paper was what made him revisit the search.

The general lesson about search

Matrix multiplication algorithm discovery has properties that make it unusually amenable to this treatment, and they are worth listing because they tell you where else the approach might work.

The objective is a single number, the multiplication count, with no ambiguity about what better means.

Verification is instantaneous. Given a proposed decomposition, checking it is a tensor arithmetic computation. A wrong answer cannot survive.

Training data is free. You can generate valid decompositions by construction and use them as supervised examples.

The search space is discrete and combinatorial, with no continuous optimisation required once coefficients are discretised.

Any problem with those four properties is a candidate. Circuit minimisation has them. Some coding theory constructions have them. Certain combinatorial design searches have them. Problems that lack them, which is most of mathematics, are not addressable this way, and the absence of a crisp verifiable objective is usually what is missing.

Practical impact

Close to zero, so far, and it is worth being clear about that.

Real matrix multiplication libraries do not use Strassen's algorithm for most sizes. The reduced multiplication count is offset by worse memory access patterns, worse numerical stability, and the fact that highly tuned implementations of the naive algorithm exploit hardware in ways the recursive structure disrupts. Strassen becomes worthwhile only above a crossover point that depends on the machine, and even then it is used carefully.

The AlphaTensor variants that optimised directly for hardware runtime are more interesting practically, because they targeted the metric that matters. Whether they end up in production libraries depends on maintenance and numerical analysis work that is unglamorous and largely undone.

The result's significance is not that matrices multiply faster now. It is a demonstration that a well-posed mathematical search problem, with fast verification, can be handed to a reinforcement learning system and produce something a field had missed for half a century. That is a narrow claim and it is well supported.

AI Tutors and What Mathematics Teaching Actually Needs

2026-06-15T00:00:00.000Z

The pitch for AI tutoring is Bloom's two sigma problem. In 1984 Benjamin Bloom reported that students receiving one-to-one tutoring performed about two standard deviations better than students in conventional classrooms, which would move a median student to the 98th percentile. The obvious question was how to deliver tutoring at scale, and for forty years the answer was that you cannot afford to.

Language models make one-to-one interaction cheap. The inference is that they should deliver Bloom's effect.

The inference has problems, starting with the finding itself. The two sigma result came from small studies with specific conditions, including mastery learning where students could not advance until they demonstrated competence. Replications have generally found smaller effects, often around 0.4 standard deviations for human tutoring, which is a good effect size and not a transformative one. The two sigma number has taken on a life independent of its evidence base.

But the more useful objection is about what tutoring does.

The bottleneck is not explanation

Mathematics teachers do not primarily fail because they cannot explain. Explanations are abundant. Every topic in the secondary curriculum has hundreds of good video explanations available free, and has for fifteen years.

What a good tutor provides is different. They notice which specific misconception a student holds, which is usually not the one the student describes. They decide when to let a student struggle and when to intervene. They calibrate difficulty so that work is hard enough to produce learning and not so hard as to produce despair. They notice when a student has stopped thinking and is pattern-matching. They maintain a model of what this particular student knows, over months.

Some of these a language model does well. Diagnosis from a worked attempt is genuinely strong; models are good at looking at a wrong solution and identifying the specific error. Generating a practice problem targeting a particular skill at a particular difficulty is also strong, and it is tedious work that teachers have limited time for.

Others it does badly by default, and the worst is the struggle question.

The desirable difficulties problem

There is a substantial literature, associated with Robert and Elizabeth Bjork, on desirable difficulties. Conditions that make learning feel harder and slower often produce better long-term retention and transfer. Spacing practice out. Interleaving problem types instead of blocking them. Testing yourself instead of rereading. Attempting a problem before being shown the method, even when the attempt fails.

The last one has a specific name, productive failure, studied extensively by Manu Kapur. Students who attempt problems they cannot solve, before instruction, outperform students who receive instruction first, on measures of conceptual understanding and transfer. The failure is doing work.

A system optimised to be helpful undermines all of this. Ask a language model a maths question and the default behaviour is to answer it, clearly and completely. That is the trained behaviour, reinforced by human raters who preferred complete helpful answers. It is precisely the intervention that removes the difficulty.

The student experience is smooth. Every obstacle dissolves on contact. Nothing is retained, because nothing was effortful, and the fluency of the explanation produces a strong feeling of understanding that is not correlated with the ability to reproduce the reasoning later. This gap between felt understanding and actual competence is well documented and it predates AI. Watching someone solve a problem feels like learning to solve it.

What the evidence says so far

The studies that exist are early and the results are not uniform.

A widely cited 2023 study of a Turkish high school found that students given unrestricted access to a chatbot for practice problems performed better on those practice problems and worse on a subsequent exam without access, compared to a control group. A version with a tutor-style system prompt that withheld direct answers did not show the same exam penalty.

Work on Khanmigo, Khan Academy's tutoring system built explicitly to withhold answers and ask questions instead, has reported more favourable results, though much of it comes from the organisation deploying it.

A 2024 randomised study at a large university using a tutoring system designed around research on learning found meaningful gains over an active control. The design mattered: the system used retrieval practice, spacing and prompted self-explanation.

The pattern across these is fairly consistent. Systems designed as answer engines produce worse learning than no system. Systems designed around learning science produce gains. The model is the same in both cases. The scaffolding is what differs.

What a defensible design looks like

Withhold the answer. The single most important design decision. When a student asks how to solve a problem, respond with a question that isolates where they are stuck. This has to be enforced in the system prompt and it has to survive pressure, because students will push for the answer and a model trained to be agreeable will eventually give it.

Require an attempt first. Do not engage with a problem the student has not tried. The attempt is where the diagnostic information is, and it is also where the learning is.

Space and interleave. A student who has just learned a technique should encounter it again in three days mixed with other techniques, not twenty more instances of it immediately. This requires tracking state over time, which most chat interfaces do not do.

Ask for explanation. Having a student articulate why a step works produces more learning than having them execute the step. Models are good at evaluating explanations and at noticing when one is hollow.

Check arithmetic with a tool. A tutor that makes arithmetic errors is worse than no tutor, and this is a solved problem.

Be honest about uncertainty. When a student's approach is unusual but possibly valid, the correct response is to work through it rather than to steer back to the standard method.

The teacher question

The realistic near-term effect is not replacement. It is a change in what teachers spend time on.

Generating differentiated practice sets, writing worked solutions, producing variants of a problem at three difficulty levels, drafting feedback on common errors: these consume large amounts of teacher time and are largely mechanical. Automating them returns hours per week.

The work that does not automate is the work that depends on knowing a specific child. Noticing that a student who was engaged in September has stopped. Deciding that this class needs to slow down. Knowing that a particular student's confusion about fractions traces to something from two years ago. None of that is in the transcript.

There is a version of this that goes badly, where the automated components are used as a reason to increase class sizes, and the relational work that was the actual value gets squeezed out. That is a policy choice rather than a technology outcome, and it is the one worth watching.

What to tell a student

If you are using these tools to learn rather than to finish homework, three rules cover most of it.

Attempt first, always, and get far enough to be genuinely stuck at an identifiable point. Then ask about that point rather than the problem.

Ask for a hint, not a solution, and say so explicitly. The model will comply if asked.

After you get an explanation, close it and reproduce the argument from memory on paper. If you cannot, you did not learn it, whatever it felt like. This step is the whole difference and it is the one everybody skips.

Hunting Primes with a Distributed Machine

2026-06-11T00:00:00.000Z

In October 2024 a former Nvidia employee named Luke Durant found a prime number with 41,024,320 digits. Written out at normal type size it would fill roughly fifteen thousand pages.

The number is 2 raised to the power of 136,279,841, minus one. It is the 52nd known Mersenne prime, and it was found by the Great Internet Mersenne Prime Search, a volunteer computing project that has been running since 1996.

The story of how you check whether a number that large is prime is a good illustration of how mathematical insight and raw computation combine, and it has almost nothing to do with machine learning, which is part of why it is worth telling.

Why Mersenne numbers

A Mersenne number is one less than a power of two. Written in binary it is a string of ones.

They matter for primality searching for one reason: there is a primality test for Mersenne numbers that is far faster than anything available for general numbers.

The Lucas-Lehmer test, developed by Édouard Lucas in 1878 and refined by Derrick Lehmer in the 1930s, works like this. To test whether 2 to the power p minus 1 is prime, for odd prime p, start with s equal to 4. Repeat p minus 2 times: replace s with s squared minus 2, reduced modulo the Mersenne number. If the final value is zero, the number is prime. Otherwise it is composite.

That is the whole test. It is deterministic, not probabilistic. It gives a definite answer.

The cost is p minus 2 squarings of a number with p bits. For the record prime that is about 136 million squarings of a 136-million-bit number, which is why this is a supercomputing problem despite being a simple loop.

The transform that makes it feasible

Multiplying two numbers with 136 million bits by the schoolbook method takes on the order of the square of that, which is unworkable.

The Schönhage-Strassen algorithm, and in practice the irrational base discrete weighted transform developed by Richard Crandall and Barry Fagin, reduce large-number squaring to a fast Fourier transform. Represent the number as a sequence of digits in some base, take an FFT, multiply pointwise, take an inverse FFT, propagate carries.

This brings the cost per squaring down to roughly n log n. It is what makes the search possible at all, and it is a genuinely deep piece of algorithmic mathematics sitting under what looks like brute force.

The floating-point FFT introduces a hazard. Round-off error can corrupt a result silently, and a single bad bit invalidates everything downstream. GIMPS clients monitor the maximum round-off error each iteration and abort if it approaches the danger threshold. Results are also verified by an independent run on different hardware with a different FFT size, and since 2020 by a much cheaper probabilistic verification scheme based on Robert Gerbicz's error-checking method, which catches errors during the computation rather than after.

The project

George Woltman released the software in 1996 and the project has run continuously since. At its core it is a work distribution server, a highly optimised client, and a lot of volunteers.

Seventeen primes have been found by GIMPS participants. The Electronic Frontier Foundation offered prizes for primes crossing digit thresholds, and GIMPS claimed the awards for the first ten-million-digit prime in 2008. Two further prizes, for one hundred million and one billion digits, remain unclaimed.

For most of the project's history the work ran on ordinary desktop CPUs. The 2024 find was different: Durant assembled a distributed GPU cluster across cloud providers in multiple countries, and the discovery was made on an Nvidia A100 in Dublin, confirmed on an H100 in Texas. It was the first Mersenne prime found on a GPU rather than a CPU, and it marks a shift in what the project's hardware base looks like.

What is not known

Whether there are infinitely many Mersenne primes is open. The Lenstra-Pomerance-Wagstaff conjecture predicts, from heuristic arguments about the density of primes, that there are infinitely many and gives an expected distribution of the exponents. The observed data fits the prediction reasonably well, which is evidence of the softest kind.

Whether there are any odd perfect numbers is also open, and it is connected. Euclid showed that if 2 to the p minus 1 is prime then 2 to the p minus 1 times that is perfect, and Euler showed every even perfect number has this form. So each Mersenne prime gives an even perfect number and there are exactly 52 known. Nobody has found an odd one, and it is known that any odd perfect number must exceed 10 to the power of 1500 and satisfy a long list of restrictive conditions.

Where a learning system could help, and does not

The obvious question is whether machine learning could speed the search.

For the primality test itself, no. The Lucas-Lehmer test is exact, deterministic and already optimal in structure. There is nothing to approximate.

For candidate selection there is a little room. Before running the expensive test, GIMPS trial-divides candidates by small factors and runs P-1 and elliptic curve factorisation attempts to find factors cheaply. Roughly two-thirds of candidates are eliminated this way, saving the full test. Deciding how much effort to spend on factoring before committing to the Lucas-Lehmer test is an optimisation problem with real structure, and the project's parameter choices are hand-tuned. A learned policy could plausibly do better. The gains would be measured in single-digit percentages.

That is the honest ceiling. This is a domain where the mathematics is exact, the algorithms are near-optimal, and the remaining work is engineering. There is no gap for a pattern-matching system to fill.

Why that matters

There is a tendency to assume that every computational problem has an AI angle. A great deal of computational mathematics does not.

The problems where learning helps are the ones with large search spaces, weak signals about which direction is promising, and cheap verification. Proof search has that shape. Conjecture generation has that shape. Matrix multiplication algorithm discovery has that shape.

Primality testing of Mersenne numbers has none of it. The search space is a list of exponents to try in order. The signal is a deterministic test. There is nothing to guess.

GIMPS has been running for thirty years on the strength of a nineteenth-century theorem, a twentieth-century transform, and volunteers who leave a program running. Occasionally the most sophisticated available approach is the one that was already there.

Mathlib: The Library That Is Slowly Eating Mathematics

2026-06-07T00:00:00.000Z

If you want to formalise a theorem, the hard part is rarely the theorem. It is everything underneath it.

To state that a continuous function on a compact set attains its maximum, you need continuity, compactness, topological spaces, order relations on the reals, the construction of the reals, and enough set theory to hold it together. If none of that exists in your system, you build it first, and you are six months in before you write the statement you cared about.

Mathlib exists so that nobody has to do that again.

What it is

Mathlib is the mathematical library for the Lean theorem prover. It began around 2017 and now contains well over a million lines of code and something in the range of two hundred thousand theorems, contributed by several hundred people.

The coverage is unusual for a formal library. Undergraduate mathematics is essentially complete: real and complex analysis, linear algebra, group theory, ring theory, field theory, Galois theory, point-set topology, measure theory, basic number theory. Graduate material is extensive and growing: category theory, algebraic geometry through schemes, functional analysis, Lie theory, homological algebra, analytic number theory.

The Lean community maintains a list tracking which theorems from a standard undergraduate curriculum are formalised, and it has been essentially fully ticked off for several years. The current frontier is research-level material.

Why one library rather than many

Earlier formalisation efforts fragmented. Different groups built their own developments of the same foundations, incompatible with each other, and results in one could not be used in another. Isabelle's Archive of Formal Proofs is a collection of independent entries. Coq's ecosystem has multiple competing libraries for the same structures.

Mathlib made the opposite bet: one library, one set of definitions, everything interoperating. If you prove something about topological groups, it composes with everything anyone else has proven about groups and about topological spaces, because there is only one definition of each.

The cost is coordination. A change to a foundational definition can break thousands of downstream proofs. Mathlib runs continuous integration over the whole library, and a proposed change that breaks anything must fix everything it broke. Large refactors take months and involve many people.

The benefit is compounding. Each contribution makes the next one cheaper. The de Bruijn factor for a new result in a well-covered area is now close to one: formalising takes about as much text as writing the informal proof.

The typeclass machinery

The technical device that makes the unification work is typeclasses, borrowed from functional programming and used far more aggressively than in any programming language.

A structure like "commutative ring" is a typeclass. When you prove a theorem about commutative rings, it applies automatically to the integers, to polynomial rings, to any structure that has been registered as one. The system searches for instances during elaboration.

This produces deep hierarchies. A normed field is a field and a normed space and a metric space and a topological space and a uniform space, with the relationships declared once and inherited everywhere. Getting these hierarchies right is a genuine design problem, and Mathlib has been through several painful reorganisations of its algebraic and topological hierarchies.

When it works, the result is that a lemma proven at the right level of generality is immediately available in every specific case. When it does not, you get instance resolution failures that are among the least pleasant error messages in software.

The projects built on top

Mathlib's value shows in what it enables.

The Liquid Tensor Experiment, formalising a theorem of Peter Scholze that he had asked the community to check, was completed in about eighteen months. That would have been impossible without the analysis and category theory already present.

The sphere eversion project formalised a version of Gromov's h-principle, a substantial piece of differential topology.

The polynomial Freiman-Ruzsa conjecture, proven by Tim Gowers, Ben Green, Freddie Manners and Terence Tao in November 2023, was formalised in Lean within three weeks of the paper appearing. Tao organised the effort and wrote about the experience, noting that the formalisation process surfaced places where the informal argument was imprecise.

The Fermat's Last Theorem project, led by Kevin Buzzard and funded by EPSRC, began in 2024 with the goal of formalising the Wiles-Taylor proof. It is expected to take many years and to require formalising large amounts of algebraic number theory and the theory of modular forms along the way. That infrastructure is arguably the real deliverable.

The machine learning connection

Mathlib is also the training corpus.

Every proof search system for Lean trains on it. The library provides hundreds of thousands of examples of goals paired with the tactics that closed them, which is exactly the supervised signal a tactic-suggestion model needs. Tools like LeanCopilot and various premise selection systems are trained on it directly.

More significantly, Lean plus Mathlib is a reinforcement learning environment. A model can propose a proof, the kernel checks it, and the check is a perfect reward signal. There is no reward hacking available: either the term type-checks or it does not. AlphaProof was trained this way, with a large volume of self-generated problems formalised into Lean statements.

This creates an unusual dynamic. The library was built by humans for humans, and it is now the substrate for training systems that will contribute back to it. Some Mathlib contributions are already machine-assisted, with a model proposing a proof that a human reviews and submits.

The community has had to think about what that means for review standards. A proof that type-checks is correct. It may still be badly named, poorly placed in the hierarchy, stated at the wrong generality, or unmaintainable. Mathlib's review process cares about all of those, and a machine-generated proof gets no exemption.

What it does not solve

The statement problem. Mathlib guarantees that a proof of a formal statement is correct. It cannot guarantee that the formal statement means what you intended.

This is where the remaining human judgement lives, and it is not a small residue. A subtly wrong formalisation is provable and useless. Reviewing a formal statement against its informal counterpart requires understanding both, and there is no automation for it.

Buzzard has made this point repeatedly in the context of the Fermat project: the difficulty is not proving things, it is stating them correctly, and stating them correctly requires knowing the mathematics.

The trajectory

Mathlib grows at a rate that has been roughly steady for years. Extrapolation is unreliable, but the direction is clear enough. More of mathematics becomes formally available, which makes formalising the next thing cheaper, which increases the rate.

At some point, plausibly within a decade, the question changes from whether a result can be formalised to why it was not. Journals have begun accepting formalisation as supplementary material. A few have discussed requiring it for results whose proofs are computationally intensive.

That would be the largest change to mathematical practice since the introduction of the referee, and it will have arrived through a volunteer project that started with someone deciding to formalise the natural numbers properly.

Base Rates, Bayes, and How to Read a Claim About AI

2026-06-03T00:00:00.000Z

A test for a disease is 99 percent accurate. One person in ten thousand has the disease. You test positive. What is the probability you have it?

The common answer is 99 percent. The correct answer is about one percent.

Out of a million people, a hundred have the disease and 999,900 do not. The test catches 99 of the hundred. It also produces false positives on one percent of the healthy population, which is 9,999 people. So among 10,098 positive results, 99 are genuine. Slightly under one in a hundred.

The test did not lie. It moved you from a one in ten thousand chance to a one in a hundred chance, which is a hundredfold update. It is just that a hundredfold update from a very small number is still a small number.

This is base rate neglect and it is the single most useful piece of applied mathematics for evaluating claims about anything, including claims about artificial intelligence.

The formula

Bayes's theorem says the probability of a hypothesis given evidence equals the probability of the evidence given the hypothesis, times the prior probability of the hypothesis, divided by the total probability of the evidence.

The version worth memorising is in odds form, because it is easier to use in your head.

Posterior odds equal prior odds times the likelihood ratio.

The likelihood ratio is how much more likely this evidence is if the hypothesis is true than if it is false. Multiply your starting odds by it.

In the disease example: prior odds are 1 to 9,999. The likelihood ratio for a positive test is 0.99 divided by 0.01, which is 99. Multiply: 99 to 9,999, or about 1 to 101. Same answer, less arithmetic.

Applying it to capability claims

Suppose you read that a system scored 90 percent on a mathematics benchmark. What should you conclude about whether it can do mathematics?

Set up the question properly. Hypothesis: the system has general mathematical capability of the kind the benchmark is meant to measure. Evidence: a 90 percent score.

The likelihood ratio requires asking how likely a 90 percent score is under the alternative hypothesis, which is that the system has narrow, contamination-assisted, benchmark-specific competence.

Under that alternative, a high score is also quite likely, because contamination produces high scores. So the likelihood ratio is close to one, and a close-to-one likelihood ratio means the evidence barely moves you.

This is why benchmark scores feel less informative than they should. The evidence is consistent with both hypotheses, so it does not discriminate between them.

Now suppose instead the evidence is a 90 percent score on problems written after the training cutoff, by people who did not publish them, in a format the system has not seen. Under the narrow-competence hypothesis that outcome is unlikely. The likelihood ratio is large and the evidence moves you a great deal.

The lesson is not that benchmarks are useless. It is that evidence is informative in proportion to how unlikely it would be if you were wrong.

Where the priors come from

The honest difficulty with Bayesian reasoning outside textbook problems is that priors are not given.

Two disciplines help.

Reference classes. Instead of asking how likely this particular claim is, ask what happened to similar claims. Of the claimed proofs of the Riemann hypothesis posted in the last twenty years, how many held up? Of the AI capability announcements that specified a timeline, how many hit it? The reference class rate is your starting point, and it is usually more informative than any argument about the specific case.

Sensitivity checking. If you cannot settle on a prior, try several and see whether the conclusion changes. If a claim survives from prior 0.01 to prior 0.5, the prior is not doing the work and you can stop arguing about it. If the conclusion flips, you have identified where the actual disagreement lives, which is progress.

The conjunction problem

A related failure. People systematically rate specific, detailed scenarios as more probable than the general categories containing them, because detail makes a story feel plausible.

Amos Tversky and Daniel Kahneman's original demonstration involved a character description followed by two options: that she is a bank teller, or that she is a bank teller active in the feminist movement. Most people chose the second, which is impossible, since every feminist bank teller is a bank teller.

This is directly relevant to forecasting. A detailed narrative about how AI systems will transform mathematics, specifying which techniques, in which order, with which consequences, is strictly less likely than the vague claim that AI will affect mathematics somehow. Each additional specific makes the conjunction less probable while making it feel more credible.

When you encounter a compelling detailed scenario, count the conjunctions. Five independent claims each at 80 percent gives you 33 percent for the whole story.

What this looks like in practice

Three habits follow.

When someone reports a striking result, ask what would have been reported if the result had been unremarkable. If the answer is nothing, you are looking at a filtered sample and the base rate is unknown. Selection effects account for most of the difference between what a field looks like from outside and what it looks like from inside.

When someone reports a percentage, ask for the denominator and the absolute numbers. A doubling of risk means one thing at a base rate of 40 percent and something entirely different at a base rate of one in a million. Reporting relative change without the base is the most common way to be technically accurate and materially misleading.

When you find yourself confident after reading one source, ask what the likelihood ratio actually was. A single article, however well written, rarely justifies a large update, because articles that argue a position exist regardless of whether the position is correct.

The connection back to machines

Language models are trained on text, and text is a filtered sample of the world. Interesting things get written about. Boring things do not. Successful projects publish. Failed ones often do not.

So a model's implicit priors are the priors of published text, which are systematically shifted toward the remarkable. Ask a model how likely a startup is to succeed and the answer reflects the distribution of startup stories that got told, not the distribution of startups.

This is not a flaw that better training fixes, because the filtering happened before the data existed. It is a reason to supply base rates yourself when you want a calibrated answer, and to treat any probability a model states without a stated reference class as an impression rather than an estimate.

The mathematics here is elementary. It fits on an index card. What makes it hard is that using it requires noticing that a question about probability is being asked, and most of the time nobody says so.