Retiring the word unproven

Since its first release, NumType’s README has carried a qualifier. The claim “TypeScript can do this” came annotated as newly tractable and unproven at scale, and that word, unproven, was the last statement in the project without a measurement behind it. The promise is that shape errors surface while you type. I could show that for projects the size of the demos. Whether it survives a large codebase, nobody knew, me included. This slice existed to replace the word.

The sweep

The probe is a generated corpus, swept along four axes that were fixed in the spec before anything ran: file count, chain length, tensor rank, and cold start. The generated code is calibrated against the retrieval demo, so the op mix is the one a real consumer produced rather than one designed to flatter. The spec also defines a found cliff as a result worth reporting, which removes any incentive to hide a bad number. Thirty-five sweep points ran; thirty-four of them measured clean, and one ended at a cliff, which gets its own section below.

The headline

Interactive latency holds everywhere. Across all thirty-four measurable points the warm hover median stayed between 0.044 and 0.106 milliseconds, and the spread barely cares what it is hovering over: a 250-file project, a chain of ten thousand operations carrying 2.68 million type instantiations, a tensor of rank 896. Typing never feels the scale. What does feel it is the cold start, the span from opening a file to its first diagnostics, which ranges from under two milliseconds on the file axis to ten seconds at the far end of the chain axis. That asymmetry is the practical rule the probe produced. A project may have as many files as it likes; a single file should not carry ten thousand chained operations.

What the flat line hides

The file axis produced the strangest table. With repeated shapes the instantiation count plateaus completely, and 250 files cost exactly what ten cost, because TypeScript deduplicates identical instantiations project-wide. With distinct shapes in every file the count grows linearly, at a flat rate near 3,800 instantiations per file. Both lines are good news, and the second one nearly went unmeasured. The obvious construction repeats shapes, and a sweep built only that way would have shown a horizontal line and reported effortless scaling, without anyone having measured wrongly. A second opinion on the spec caught the gap before the build, and the sweep got both sub-series.

Where the wall is

Rank is the one axis where growth is genuinely superlinear. The broadcast machinery recurses once per tuple position inside a single call, so the marginal cost doubles with every doubling of rank, and at rank 1024 the checker gives up mid-file.

// rank 896 still checks; rank 1024 dies
error TS2589: Type instantiation is excessively deep, possibly infinite.

Real NumPy-shaped arrays have single-digit rank, so the cliff says more about the machine than about the use case. It still matters, for an uncomfortable reason. At that point the checker rejects valid code, and the project’s degradation rule (wide instead of wrong) cannot apply there, because the checker dies before any degradation logic runs. The wall was always part of the design’s territory. Now it has coordinates.

One word retired

The README sentence that frames the research carried unproven from the first release on. It now carries the measurement. Interaction holds at consumer scale, the costs of growth land on the cold start, and the ceiling has coordinates. The numbers went through three independent verifiers before the sentence changed; what they caught on the way is written up in the research notes. One question stays open on purpose, and the README names it as well. Whether the machinery holds at a NumPy-sized set of operations is an axis no sweep has touched, and no measurement here speaks for it.

NumType is on npm; the sweep, its spec and the full tables are in the research notes on GitHub.