dorsal/arxiv
View SchemaMark My Works Autograder for Programming Courses
| Authors | Yiding Qiu, Seyed Mahdi Azimi, Artem Lensky |
|---|---|
| Categories | |
| ArXiv ID | 2601.10093vv1 |
| URL | https://arxiv.org/abs/2601.10093 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
Large programming courses struggle to provide timely, detailed feedback on student code. We developed Mark My Works, a local autograding system that combines traditional unit testing with LLM-generated explanations. The system uses role-based prompts to analyze submissions, critique code quality, and generate pedagogical feedback while maintaining transparency in its reasoning process. We piloted the system in a 191-student engineering course, comparing AI-generated assessments with human grading on 79 submissions. While AI scores showed no linear correlation with human scores (r = -0.177, p = 0.124), both systems exhibited similar left-skewed distributions, suggesting they recognize comparable quality hierarchies despite different scoring philosophies. The AI system demonstrated more conservative scoring (mean: 59.95 vs 80.53 human) but generated significantly more detailed technical feedback.
{
"annotation_id": "7cf48888-840d-4cfa-83cd-53b4f53fbb91",
"date_created": "2026-02-17T05:53:24.219000Z",
"date_modified": "2026-02-17T05:53:24.219000Z",
"file_hash": "61e1d311f388ba52f61a3aa3f3579f2a873b78c5fe86e653d98d190a8f8a516e",
"private": false,
"record": {
"abstract": "Large programming courses struggle to provide timely, detailed feedback on student code. We developed Mark My Works, a local autograding system that combines traditional unit testing with LLM-generated explanations. The system uses role-based prompts to analyze submissions, critique code quality, and generate pedagogical feedback while maintaining transparency in its reasoning process.\n We piloted the system in a 191-student engineering course, comparing AI-generated assessments with human grading on 79 submissions. While AI scores showed no linear correlation with human scores (r = -0.177, p = 0.124), both systems exhibited similar left-skewed distributions, suggesting they recognize comparable quality hierarchies despite different scoring philosophies. The AI system demonstrated more conservative scoring (mean: 59.95 vs 80.53 human) but generated significantly more detailed technical feedback.",
"arxiv_id": "2601.10093",
"authors": [
"Yiding Qiu",
"Seyed Mahdi Azimi",
"Artem Lensky"
],
"categories": [
"cs.SE"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "Mark My Works Autograder for Programming Courses",
"url": "https://arxiv.org/abs/2601.10093",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "d141eb6f-85c4-4ecf-967c-6e232bbde0e3",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}