Matthew, what was particularly scarring about the personalisation wave in the mid 2010s? Currently looking into/ building in that area (AI tutors, learning maps) and would be keen to avoid previous mistakes.
Long term, I am quite bullish on the personalization path and love to see folks like you building in the space. The key is keeping your product/marketing promises in line with what you can actually deliver to the student. If I could encourage you to do one thing, it would be to constantly measure - ideally with a neutral third-party, though that can be expensive at the outset - your student outcomes. If you do a good job helping students, the business side will follow.
Thanks, appreciate the response and a good reminder to make sure we invest in measuring performance. I'll try not to manufacture any quotes like the following anytime soon:
"We think of it like a robot tutor in the sky that can semi-read your mind and figure out what your strengths and weaknesses are, down to the percentile."
Agreed re: Hechinger, which is why I wanted to give the article airtime. Trying to keep testing myself, even if I disagree pretty strongly with the premise!
Matthew, what was particularly scarring about the personalisation wave in the mid 2010s? Currently looking into/ building in that area (AI tutors, learning maps) and would be keen to avoid previous mistakes.
Thanks for the links btw - reading every week!
Great question! Mary Jo Madda does a good job covering two of the ringleaders - Knewton and AltSchool - from that era, though they were by no means the only companies making claims about personalization that didn't hold up well. https://www.edsurge.com/news/2019-09-07-knewton-was-acquired-for-less-than-17m-and-former-ceo-brian-kibby-has-left
Long term, I am quite bullish on the personalization path and love to see folks like you building in the space. The key is keeping your product/marketing promises in line with what you can actually deliver to the student. If I could encourage you to do one thing, it would be to constantly measure - ideally with a neutral third-party, though that can be expensive at the outset - your student outcomes. If you do a good job helping students, the business side will follow.
Thanks, appreciate the response and a good reminder to make sure we invest in measuring performance. I'll try not to manufacture any quotes like the following anytime soon:
"We think of it like a robot tutor in the sky that can semi-read your mind and figure out what your strengths and weaknesses are, down to the percentile."
🔥🔥🔥
Hechinger does a decent job explaining that we don’t understand the mechanism of the math tool.
I... also tend to distrust WWCH. They are trying hard at an important task, but they are also so bad, in so many respects
Agreed re: Hechinger, which is why I wanted to give the article airtime. Trying to keep testing myself, even if I disagree pretty strongly with the premise!
The tool probably doesn’t work, or doesn’t work for the reasons anyone thinks! Ed research is deeply confounded almost all the time