How hot—or cool—are AI-powered tutoring robots?
Is instruction guided by artificial education any more than the latest fad?
AI-guided instruction. Intelligent tutoring systems. There is a lot of talk these days about applications of artificial intelligence to education. I have mixed feelings. A lot of the time, I think it is just BS. However, I recognize that there could be something good here.
Let's just think this through for a moment. Suppose that an intelligent tutor was taught to provide instructional recommendations for a student who has problems in, let's just say, computing solutions for math problems. OK cool. Here is a whole host of questions that follow:
What does the AI program use for its recommendations about solving math problems? Let's take that idea a little farther. What if the AI was trained on the corpus of current recommendations about teaching children to solve math problems? I can think about—let me just say—a shipload load of documentation for training that would be inconsistent with good practice. There is, as you Dear Readers know, a lot of poopy recommendations regarding instruction available on the Intertubes. AI might learn its instructional practices by studying trashy resources. Sure, we can hope that smart programmers design their AI-bots to search for good recommendations rather than poopy ones, but how can we be sure that those design features use dependable standards for discriminating facts from poop. That is, if we put garbage into the AI bot, would we expect to get stellar recommendations out of the bot? I’m reminded of an adage about analyzing data with fancy statistics: GI-GO (garbage in, garbage out).
Does the AI-bot employ dependable, trustworthy instructional design principles? (a) Let’s suppose that we think instructional activities should be inspired by Constructivist views about teaching. How would the AI-bot pose a problem-solving task for the tutored student? What student responses would it be trained to anticipate or expect? (b) Alternatively, suppose we think Instructivist views should guide teaching. How would the AI-bot select and sequence teaching examples? What responses would it be prepared to reinforce or to correct? Maybe we should hope that the AI-bot’s teachers (ahem) don’t let the AI-bot figure it out for themselves…imagine the garbage that a bot could consume in teaching itself how to teach!
A real ideal benefit for AI training would be that the AI recommendations would be supple and adaptive. That is they would examine a student’s performance and make recommendations about how that performance would be used to promote better outcomes for that student that it is, they would suggest an individualized educational program. We know that there are good models for data-based decision making (e.g., Förster et al., 2018; McMaster et al., 2025). As the previous problem arises: What data would the AI use in developing recommendations? Maybe it would use actual performance by the student, as used in the studies of effective DBDM. But, if you see the current state of recommendations in math (or reading or spelling or composition or science, you will see that most of those recommendations are predicated on psychological types, withdrawn, anxious, or eager or other bologna versions of psychology that have been shown not to effect educational outcomes positively. That is to say the popular psychology version of adapting instruction to learners performance is bologna.
We instead need, if we're going to trust AI, a set of practices that train AI on how children actually respond to questions, how they answer questions. Said another way, we want to be sure that the AI assesses students’ performance based on their actual performance, not distal interpretations of their motives or false theories about performance.
A big idea here is that AI isn't going to be any better than we make it. It offers some benefits, such as rapid analysis and reference of the analytic recommendations of evidence available on the Internet. However, those recommendations are only as good as the data used by the program and its training to interpret (or even ignore) data.
So, don't get too excited about these recommendations. Sure they're sexy and they can sound really hot. They might be cool and valuable. But the proof will be in the pudding. Do they generate better outcomes for kids than, let's say, garden variety, different Instruction—and generate better outcomes than, let's say, some well-documented instructional practices?
References
Förster, N., Kawohl, E., & Souvignier, E. (2018). Short-and long-term effects of assessment-based differentiated reading instruction in general education on reading fluency and reading comprehension. Learning and Instruction, 56, 98-109. https://doi.org/10.1016/j.learninstruc.2018.04.009
McMaster, K. L., Lembke, E. S., Shanahan, E., Choi, S., An, J., Schatschneider, C., Duesenberg-Marshall, McK. D., Birinci, S., McCollum, E., Garman, C., & Moore, K. (2025). Supporting teachers’ data-based individualization of early writing instruction: An efficacy trial. Journal of Learning Disabilities, 58(4), 287-303. https://doi.org/10.1177/00222194241300324