Open research platform · Autonomous software engineering

Measuring what actually makes coding agents better.

SIA is a research platform for self-improving software engineering agents: a deliberately minimal baseline plus independently toggleable capabilities — repair, memory, reflection, retrieval, multi-agent roles — each evaluated in controlled ablations at matched budgets, on hardware anyone can reproduce.

Roadmap

A milestone is done only when its exit criteria are met and results are recorded in the experiment registry.

Research questions

Every capability module exists to answer a question. Negative results get recorded with the same rigor as positive ones.

Experiments

Experiments are pre-registered: hypothesis, method, and decision rule are fixed before the run.

Results

Populated automatically from run manifests in the repository — every number traces back to a replayable JSONL trace.

pass@1 by model and benchmark