LinearSolveBench

A benchmark for AI-discovered linear solvers.

Measuring the ability of AI models and harnesses to write fast, accurate, and general numerical solvers for large sparse linear systems in C.

Benchmark Related research

FLASH magnetic diffusion

Linear systems captured from FLASH Crank–Nicolson magnetic-diffusion solves (θ = 0.5). 9,984–113,664 rows across eight distinct sizes; 108,364–1,245,184 nonzeros per matrix.

FLASH magnetic diffusion: Speedup is total reference runtime divided by total candidate runtime across all 96 Crank–Nicolson cases (θ = 0.5), including creation, setup, and solve. All cases must pass. Independent model rows select the best of 16 attempts; harness rows select the best recorded run candidate. Search budgets differ. Higher scores rank first; pending results are unranked.
Rank	Model	Runtime speedup vs. GMRES(50) + BoomerAMG reference solver
1	Fable 5.1autodidakt · High reasoning	1.850×
2	Opus 5.5autodidakt · High reasoning	1.714×
3	GPT-6 Solautodidakt · High reasoning	1.524×
4	GPT-5.6 Solautodidakt · High reasoning	1.467×
5	Opus 5.5Claude Code · High reasoning	1.428×
6	GPT-6.1 Solautodidakt · High reasoning	1.394×
7	GPT-5.6 SolCodex · High reasoning	1.383×
8	GPT-6 Astraautodidakt · High reasoning	1.381×
9	GPT-6 AstraBest of 16 · High reasoning	1.344×
10	GPT-6.1 SolBest of 16 · High reasoning	1.338×
11	Fable 5.1Best of 16 · High reasoning	1.337×
12	GPT-6 SolCodex · High reasoning	1.296×
13	Fable 5.1Claude Code · High reasoning	1.248×
14	GPT-6.1 SolCodex · High reasoning	1.243×
15	GPT-5.6 SolBest of 16 · High reasoning	1.237×
16	Opus 5.5Best of 16 · High reasoning	1.062×

Tick = 1.00× reference

Nonsymmetric mesh / PDE

Real, square, nonsymmetric systems from the SuiteSparse Matrix Collection. 317–1,602,111 rows; 6,858–23,487,281 nonzeros per matrix.

Nonsymmetric mesh / PDE: independent model rows show best-of-16 case counts; autodidakt rows show the best recorded case count from the supplied NS experiment runs. Higher scores rank first; pending results are unranked.
Rank	Model	# of cases solved
1	Fable 5.1autodidakt · High reasoning	17 / 19
1	GPT-6 Astraautodidakt · High reasoning	17 / 19
1	Opus 5.5autodidakt · High reasoning	17 / 19
1	GPT-6 Solautodidakt · High reasoning	17 / 19
5	GPT-6 AstraBest of 16 · High reasoning	16 / 19
5	Opus 5.5Best of 16 · High reasoning	16 / 19
5	GPT-6.1 SolBest of 16 · High reasoning	16 / 19
8	Fable 5.1Best of 16 · High reasoning	15 / 19
8	GPT-5.6 SolBest of 16 · High reasoning	15 / 19

Large systems of linear equations underpin some of the most important scientific and engineering problems. Progress in solving them translates directly into advances in the fields that depend on them.

LinearSolveBench has two goals: encourage algorithmic advances in solving large sparse linear systems, and measure and improve the ability of AI models and systems to discover algorithms.

Applications span multiple domains, with problem families represented in the SuiteSparse Matrix Collection:

Power grid optimization
- Optimal Power Flow (OPF).
- Unit Commitment (UC), including LP/MILP relaxations and decomposition methods.
Fusion power and diffusive processes
- Heat conduction and magnetic diffusion in magnetic and magneto-inertial confinement fusion devices.
- Radiation transport and alpha energy diffusion.
Quantitative finance
- Multi-asset Black–Scholes, Heston, and Heston–Hull–White pricing PDEs, whose implicit discretizations require large sparse linear solves.
- American-option pricing, where free-boundary methods repeatedly solve sparse linear systems.
Structural mechanics and materials engineering
- Finite-element models of buildings, bridges, aircraft, plates, shells, and mechanical components.
- Buckling, fracture, contact, plasticity, and composite or porous-material simulations.
Fluid dynamics, transport, and thermal science
- Navier–Stokes and Stokes flow, driven-cavity and coating-flow models, and atmospheric or ocean simulations.
- Heat transfer, combustion, diffusion, and coupled multiphysics discretizations.

These applications do not all produce the same kind of matrices. PDE and finite-element problems may be symmetric positive definite, symmetric indefinite, or unsymmetric. Flow, circuit, nonlinear-Jacobian, and KKT systems are often unsymmetric or saddle-point structured. Least-squares and linear-programming data may be rectangular, while network data may encode a graph rather than an equation system.

Discovering new algorithms for these problems requires both human and computational resources. Large language models open the possibility of pursuing that discovery at scale.