Skip to content
Ideas for a more human education
Small robots question a student about highlighted steps on a worksheet while a teacher reviews the grade.
Illustration: Human Learning Review, made with AI. It does not depict real people or events.
Research · Teaching & Learning

ProofPaths: An AI Agent Framework for Assessing Student Reasoning

Source: arXiv, . Summarized by Human Learning Review.

Summary

The paper asks how instructors can tell whether a student understands the reasoning behind a correct assignment when large language models can produce the solution for them. It answers by describing ProofPaths, an AI agent framework that pairs rubric-based grading with individualized, interactive interviews. It reports no results from use with students.

  • Several grading agents assess each submission against the instructor's guidance. They then exchange critiques and revise their judgments.
  • The agents' scores decide which review supplies the grade and the feedback the student receives.
  • An interview agent asks the student to justify steps in the submission and uses the answers to shape follow-up questions.
  • Each interview question is tied to a retrieved excerpt from the submission, so instructors can set the student's explanation beside the passage it cites.
  • The system tracks assessment progress and stores grading reviews and interview transcripts for every submission. Instructors can inspect them and override grades inside the assignment workflow.

The authors present the design as a way to probe understanding without the recurring staff effort that one-on-one interviews demand. The checks they describe are deterministic tests of the assessment controllers, which examine source handling and score aggregation. Other tests confirm limits on how many correction attempts the system will make.

Main findings

  • The authors argue that a correct submission can't show whether a student understands its reasoning, since language models can complete the work.
  • In ProofPaths, multiple grading agents critique one another and revise, and their scores select which review provides the grade and feedback.
  • An interview agent questions students about steps in their own submission, linking each question to a cited excerpt so instructors can compare explanation and text.
  • The framework is offered as a way to probe student understanding without the repeated staff effort that individual interviews require.
  • Instructors can inspect every stored review and interview transcript and override any grade within the assignment workflow.

How ProofPaths grades and interviews a student

  1. Agents grade against guidance

    Multiple grading agents assess the submission using the instructor's rubric guidance.

  2. Agents critique and revise

    Agents exchange critiques of each other's reviews and revise their judgments.

  3. Scores select the review

    Agent scores determine which review supplies the grade and feedback.

  4. Interview probes reasoning

    An interview agent asks the student to justify steps, citing submission excerpts.

  5. Instructor reviews and overrides

    Instructors inspect stored reviews and transcripts and can override grades.

Data and findings: arXiv, “ProofPaths: An AI Agent Framework for Assessing Student Reasoning.” Graphic: Human Learning Review.

This page is a short summary of a research paper. The full piece belongs to arXiv, and preprints may not have been peer reviewed.

Read the original at arXiv
A little room for reflection

A more considered
view of education.

Ideas worth sitting with. Practical questions worth asking. Human Learning Review, in your inbox.

ProofPaths: An AI Agent Framework for Assessing Student Reasoning: summary and main points | Human Learning Review