This study asks whether large language models, tuned to match teaching frameworks, can measure classroom talk better and more fairly than standard classifiers that score single utterances and tend to miss reasoning voiced in non-standard dialects. The authors built a framework that runs the model through a judge, critique and refine loop with preference optimization, added a training objective using dialect style-transfer data, and tested it on 1660 elementary mathematics lesson transcripts.
- Detection of complex student reasoning chains rose by 14.2 percentage points on a macro-averaged measure combining precision and recall, compared with leading discriminative baselines.
- The debiasing step cut the false-negative rate for student contributions in African American Vernacular English by 18.4 percentage points.
- The automated discourse scores showed a statistically significant correlation of 0.10 with value-added measures of teacher effectiveness.
- The authors attribute the dialect gains to a contrastive objective that separates the reasoning in a contribution from surface features of how it is phrased.
The authors conclude that their approach captures high-inference constructs such as student reasoning and teacher uptake more accurately, and gives a fairer reading of students' intellectual contributions across demographic groups. They argue that equity-aware automated discourse analysis can stand in as a proxy for learning outcomes, resting that claim on the correlation with value-added models. The abstract does not state limitations.
