We have released the code and supporting records for LLM Judges Have Dark Current, a joint study with Mitsubishi Heavy Industries, Research & Innovation Center. The repository contains the manuscript, evaluation records, and analysis code for arXiv v2.
What is available
The revised study examines three questions separately: whether a judge returns a readable verdict, whether it expresses a preference between identical answers, and whether it selects the specified target under a declared grading criterion. It pairs identical-answer checks with target-selection tests under the same evaluation settings.
The repository contains the manuscript PDF, recorded verdict metadata, analysis code, and supporting material for the figures and tables. The reproduction guide distinguishes file-integrity checks, inspection of saved records, and study-specific reanalysis, with instructions for each.
Research code is released under the MIT License. Author-owned manuscripts, figures, explanatory text, and numerical records are available under CC BY 4.0. LICENSE_SCOPE.md documents the terms and attribution.
Paper and release details
- Paper
- LLM Judges Have Dark Current: Stratified Null and Utility Tests for LLM Judge Reliability
- Public version
- arXiv:2606.15610v2, revised October 13, 2026 (JST)
- Research partner
- Mitsubishi Heavy Industries, Research & Innovation Center