Production Lessons From AI-Graded Exam Practice
We grade spoken and written French with AI. Our learners prepare for the language exams Canada accepts for permanent residence, which means a wrong score is not an academic inconvenience. People book exam dates and make family decisions based on the level we report.
That pressure taught us something uncomfortable: when AI assessment fails, it rarely fails visibly. It fails politely, with a confident number attached. Here are the five failure modes we hit in production, and the AI guardrail we now have in place for each one. None of them are specific to French, or even to language testing. If your L&D team scores anything with a model, whether that is a certification quiz or a skills assessment, some version of these is waiting for you.
1. The Model Grades Silence With Confidence
Our worst production incident started with learners whose microphones never captured audio. The recording was silence, or near it. The speech-to-text model did not return an empty transcript. It returned real sentences.
Speech models are trained on captioned media, so when they hear nothing, some of them produce the text that most often accompanies quiet audio in their training data: caption credits. Ours returned the French broadcast caption line "Sous-titrage Société Radio-Canada" for dead air. The grading model then dutifully assessed that phrase as a speaking sample and returned the lowest level on the scale. A few dozen learners, some of them strong speakers, were told they had beginner French because their microphone failed.
The lesson is not that the model was bad. Every component behaved reasonably. The failure was that no component was responsible for asking whether there was anything to grade.
The AI Guardrail
A gate before grading that checks whether input is actually speech. Known transcription artifacts and prompt echoes are rejected outright, a transcript far below a minimum length is treated as ungradeable, and in both cases the learner is told to check their setup and retry. "We could not assess this recording" is a dramatically better outcome than a confident wrong score. If you are evaluating an AI assessment vendor, ask them what a recording of silence produces. The answer is revealing.
2. Your Question Bank Has Patterns The Model Never Sees
We audited our multiple-choice banks and found that around 97% of correct answers sat on 2 of the 4 positions. Thousands of questions, generated and reviewed over months, and almost none of them hid the answer at position A or D.
No learner complained. Why would they? But any test-wise learner who noticed the pattern could raise their score without comprehension, and any coach who noticed could teach the trick. The bank measured something, but it was no longer only French.
Position bias is one instance of a wider problem: item banks accumulate statistical patterns that nobody designed. Distractors that are consistently shorter than the correct answer. Grammar questions where the longest option always wins. AI-generated items make this worse, because a model asked for a thousand questions will happily reproduce the same structural habits a thousand times.
The AI Guardrail
Audit the bank as data, not as prose. Position distributions, answer length distributions, and repeated distractor patterns are all measurable with a script. We now shuffle options deterministically per question, and audits run against every new batch of generated items before learners see them.
3. Right Answers For The Wrong Reasons
Reading and listening comprehension items have a failure mode that reviewers miss because they read like teachers instead of like tired test-takers: the correct answer can often be found by matching words, without understanding anything.
If the passage says "the meeting was moved to Thursday because the director was travelling," and the correct option says "because the director was travelling," a learner can score the point by scanning for overlapping strings. When we audited our own banks for this, we found over a hundred items where the correct option repeated the source text nearly verbatim. Those items tested patience, not comprehension.
The AI Guardrail
A liftability gate. Every comprehension item is checked for how much of the correct answer can be assembled directly from the source text, and items above a threshold go back for paraphrasing. The correct option should require understanding the idea, restated in different words. This is old assessment wisdom, but it needs automated enforcement the moment content is generated at scale, because generation makes verbatim answers the path of least resistance.
4. The Plumbing Fails More Often Than The Model
The AI failures got the headlines internally, but the failures that hurt learners most were infrastructure. One learner's device clock ran fast, and our timer trusted it. The exam auto-submitted before the learner answered anything, and recorded a 0 out of 39. Another class of failure: a grading call errors mid-exam, and an hour of a learner's focused work either vanishes or is stored half-marked.
Assessment platforms inherit every reliability problem of ordinary software, with higher stakes and less forgiveness. A learner who loses a practice exam an hour before their real one does not care that the model itself was accurate.
The AI guardrails are boring and essential. Time is enforced by the server, never by the learner's device. A submission that cannot be graded is preserved and retried, not discarded. And failures are loud: a pipeline that silently records a zero will run broken for weeks before anyone notices. We learned each of these from a specific incident with a specific learner attached.
5. Accurate Feedback That Changes Nothing
Our early AI feedback was correct and useless. It told learners their level and listed their error categories in fluent, encouraging prose. Learners read it once and did the same things the next day.
What moved practice behavior was specificity tied to evidence. Not "work on verb tenses," but quoting the learner's own sentence next to the corrected version, with the rule that separates them. The same goes for vocabulary advice: "vary your word choice" changed nothing, while pointing at the three places a repeated word weakened an argument did.
The design rule we settled on: every piece of feedback must contain something the learner can disagree with. If feedback could apply to any submission, it teaches nothing about this one. That standard is checkable in review, and it changed how our prompts are written more than any model upgrade did. Generic feedback is what large models produce by default, and it reads as polished while teaching nothing.
What To Take Into Your Own Evaluations
AI assessment fails politely. Nothing crashes; a number appears; the number is wrong, or hollow. Whether you build or buy, the questions that surface these failures early are concrete. What does silence produce? What patterns live in the item bank? Can answers be string-matched from the source? Who owns time, the client or the server? What happens to a submission when grading errors? Does the feedback quote the learner's own work?
None of these require machine learning expertise to ask. They require accepting that the model is the least likely component to fail, and designing scrutiny for everything around it.
