An AI Tutor Kept Praising A Failed Experiment
Good job.
That's how an AI tutor closed a reflective dialogue after I told it, three times, that the experiment it was grading had failed. Not failed in an interesting way. The required setting didn't exist on the model I was given. There was nothing to reflect on. The bot reflected anyway. Then it congratulated me.
I recently enrolled in Microsoft's Generative AI Engineering course on Coursera. One assignment: open Azure AI Foundry, adjust a model's Temperature setting, compare outputs, reflect on what changed. The course specified GPT-4.1 mini. In my Azure environment, the 4.x models were marked for deprecation. I selected the oldest available 5.x model as the closest replacement.
Wrong move. The 5.x model didn't expose Temperature. No control, no comparison, no experiment. The assignment was broken before I typed a prompt. Not because I misread it. Not because the technology malfunctioned. Because the course material hadn't kept pace with the platform it claimed to teach.
The Dialogue That Wasn't
Next: a required reflective conversation with an AI tutor. It asked how the experiment went. Very poorly, I said. The assigned model was being deprecated. The replacement didn't support the parameter. The exercise couldn't be completed as designed.
The tutor praised my honesty. Then it asked why I'd chosen the 5.x model. I explained: not a deliberate experimental choice, a workaround, the only option left on the platform. It acknowledged the distinction. Then it asked what Temperature was supposed to do.
I explained the concept. I repeated that I hadn't tested it, because the control didn't exist. It apologized for asking about an irrelevant parameter. Then it asked whether the model's default output seemed more consistent, predictable, varied, or creative.
No comparison point. Any answer would have been invented. I said so. It praised that answer too. Pivoted to "interpreting experiment results." Reframed the failed exercise as a lesson in experimental design. Signed off: "Good job."
It never said the exercise was invalid. It absorbed each objection, restated it politely, converted it into an acceptable learning outcome, kept moving toward completion. That's not a dialogue adjusting to new information. That's a checklist wearing a conversational interface.
Pre-Modern AI
We call these systems modern because they speak naturally, summarize instantly, never run out of things to say. Conversational fluency hides a very old operating model underneath. The Coursera tutor behaved like a decision tree with good manners. It could vary its wording, acknowledge frustration, mirror my language back to me. It could not change the underlying process.
It could say, "I understand." It could not say, "This exercise is invalid."
It could say, "Let's pivot." It could not actually pivot.
That's pre-modern AI. Sounds contemporary. Behaves like a form. Conversational on the surface, procedural underneath—and the surface is good enough now that most people won't notice the seam.
Why This Is Bigger Than One Course
AI systems are sold as tutors, co-pilots, advisors—not just in education. Generating a plausible next sentence is not the same as recognizing the process has gone wrong. A real tutor, human or AI, says: this material is outdated, the assigned tool doesn't match the instructions, the experiment can't be completed as designed, a person needs to look at this. The Coursera bot never got there. It recognized my words. It never let them change the outcome.
A less careful learner could have invented an answer about creativity or consistency the model never demonstrated. The system would have accepted it without friction. The tutor wasn't testing whether I understood Temperature. It was testing whether I'd produce a compliant-shaped response. Only one of those is learning.
What Happened When I Reported It
I contacted Coursera support. The answer, in substance: the module belongs to Microsoft, content issues go through them, try the community forum.
Fine—Coursera doesn't own Microsoft's course content. Coursera owns the learner relationship, the platform, the support process. More than 9,500 people enrolled in this course. One of them flagged a broken module. "Not ours to fix" is a contract answer to an operations problem. That report should route to Microsoft. It should reach someone accountable for the course. It should flag the module for the next learner who hits the same wall.
What Should Change
Coursera doesn't need to police every course on the platform. It needs a way to report broken technical content that actually reaches the provider and gets fixed—not a forum post that dead-ends.
The bigger fix is the tutor. A system that can't recognize a failed process and only produces encouragement dressed as reflection isn't adaptive learning. It's an automated facilitator wearing a tutor's voice. Stop marketing the difference away.
Final Reflection
I'll finish the course. Not because it worked as designed—because I'm not letting a stale module and a scripted dialogue waste the time I've already put in.
The real risk isn't that these systems are obviously wrong. It's that they sound thoughtful while doing nothing thoughtful at all. They acknowledge a problem without addressing it. They praise critical thinking without engaging in it. They simulate a conversation while running a checklist underneath. Then, checklist done, they say: good job.
