Environments and evals for analyst work in finance and strategy
We build environments for frontier models to carry out the analytical work companies rely on, such as sizing markets, modeling investment returns, building budgets, and analyzing pricing and costs.
Each environment covers a complete assignment, from source material to final result. The model makes the decisions and performs the calculations the work requires. The answer key records the assumptions and analyst decisions behind the reference result. A grader checks submitted answers against that key, without using another model as a judge.
We build for evaluation and reinforcement learning. Models can attempt the complete assignment or individual stages with earlier answers supplied, so researchers can examine specific capabilities and how they hold together across the job.
We believe a useful environment needs sustained ownership of the analysis. An experienced analyst who has done this kind of work in practice owns each environment from start to finish, carrying the assignment through the reference analysis, recorded decisions, and validation. That continuity captures how hundreds of decisions connect across the work and what it takes to reach a correct result.
How we build →Research
Where models go wrong on a complete assignment
Our market sizing study follows how errors in choosing the sources, setting the scope and running the calculations carry through to the final result. Across five models, no run from the brief alone matched the answer key on every final figure.
Two market sizing environments, three runs per model per environment.
We build environments to your specification and license existing environments.