What Happens When You Ask AI to Extract Contract Metadata at Scale

A first benchmark on the gap between one-contract demos and contract-population workflows. Claude Opus 4.8, Sonnet 4.6, and GPT-5.5 tested on 100 SEC contracts.

If you ask a frontier model to pull the document title, party names, and effective date from a single contract, it will do so without issue (in most cases).

Relative to what was possible 10 years or even five years ago, the experience feels almost magical.

However, this is a significantly easier problem to solve than organizing an entire contract populati…