{"figure":"who-caught","title":"Who caught each incident, by month","page":"https://www.orbytlabs.ai/research/agent-evals","url":"https://www.orbytlabs.ai/research/agent-evals/who-caught.json","source":"app/(site)/research/agent-evals/incident-figures.generated.json (byMonth, window, beforeWindow), written by scripts/research/method-figures/agent-evals.ts from node scripts/check-incident-trailers.cjs --json, the trailers in docs/lessons-learned.md","caveats":["Every lesson from 73 on must carry a trailer. An incident written up in an earlier lesson, or never written up, may be missing from the table.","Most of the drawn incidents fall in September, so the other months rest on 5 incidents between them."],"not_shown":["One incident detected before the first month drawn and written up later (lesson 91, detected June 2, 2026). It has no column, because the table does not say what else those months held. The table below and the downloads list it."],"counted_on":"2026-11-08","cite_as":"Orbyt Labs, Who caught each incident, by month (2026-11-08), https://www.orbytlabs.ai/research/agent-evals/who-caught.json","window_from":"2026-09-12","window_first_entry":"73","newest_incident_detected":"2026-11-08","before_window":"lesson 91 detected 2026-06-02 by session","columns":[{"key":"month","label":"The month the incident was detected, YYYY-MM"},{"key":"machine_checks","label":"Recorded as caught by the Machine's own checks: a test, a guard or an outside model's check","unit":"incidents"},{"key":"model_session_or_founder","label":"Recorded as caught by a model session or the founder","unit":"incidents"},{"key":"unattributed","label":"Found in production; the trailer names no catcher","unit":"incidents"},{"key":"total","label":"All incidents detected in the month","unit":"incidents"},{"key":"drawn","label":"true for a month the chart draws; false for a month before the first lesson that must carry a trailer"}],"rows":[{"month":"2026-06","machine_checks":0,"model_session_or_founder":1,"unattributed":0,"total":1,"drawn":false},{"month":"2026-09","machine_checks":5,"model_session_or_founder":11,"unattributed":4,"total":20,"drawn":true},{"month":"2026-10","machine_checks":2,"model_session_or_founder":2,"unattributed":0,"total":4,"drawn":true},{"month":"2026-11","machine_checks":0,"model_session_or_founder":0,"unattributed":1,"total":1,"drawn":true}]}