Studies

When a coding model adds code, does it reuse an existing helper or copy nearby inline implementations?

Method. Paired clean and inconsistent code conditions, spanning synthetic files and pinned slices of JIG and Scout. Helper reuse and functional correctness were evaluated separately.

Finding. In the real-repository experiments, making the helper visible through an import changed reuse more than duplication in other files. Model choice also mattered, and one same-file example showed models following a nearby implementation.

Scope and limits. These experiments supplied repository context to coding models. They do not establish the behavior of agents that search and navigate a repository before editing it.

Can a more explicit rubric and separate feature scores improve Scout's keep/drop decisions?

Method. Human repeat grading first examined the rubric. Revised feature questions and fixed policy were then compared with the production classifier on fresh posts, with model answers hidden during grading.

Finding. On a blind 100-post holdout, JEV v5 plus fixed policy reached 77% keep/drop agreement with the reviewer's labels, compared with 64% for production.

Scope and limits. The result covers one reviewer and one 100-post batch for that comparison. It is an offline result and evaluates the classifier and policy together.