Very cool, if true

I think one of the deep things this is showing us is that humans are actually rubbish, and don't live up to our own ideas of how smart we are.

I've two examples from my own field (educational technology). I'm sure they exist in others.

The first is around automated assessment for long form writing. (Automated assessment is obviously way better for marking constrained choice questions.) The fundamental problem to making progress here is not the technology, it's what you compare it to. Human markers are terrible. On long form writing, even with very clear, strict guidance, human markers often give profoundly different marks to the same script. Even worse, the same human marker will often give an alarmingly divergent mark to the same script hidden further down the marking pile if they don't spot what's happening. There are various methods for improving this, some more successful than others, but the ones that are very effective ... are essentially coming up with marking rules that you could give to a computer.

The second is about tutoring. There was a guy who was developing artificial tutors. The problem, he realised pretty quickly, was that artificial tutors couldn't respond specifically to what the learner was doing, couldn't make sure that they understood the whole curriculum, and tended to give the same stock responses when learners give particular 'trigger' phrases. So he did some field research looking at what the best human tutors do. And it turned out that they almost never respond specifically to what the learner is doing, don't make sure that they understand the whole curriculum, and tend to give the same stock responses when learners give particular 'trigger' phrases.

(The punchline to the second one is that his team, and several others, have gone on to develop artificial tutors that are now really pretty good at all those things. And don't get tired.)

Humans are rubbish. I suspect we are literally as stupid as it is possible to be and still think you're smart. We are the first species to cross the barrier in to being able to articulate a self-conception about intelligence, and it seems odd to imagine we shot way across that line rather than drifting just across it.

I should add that humans are also brilliant. All of my most loved people and friends are humans. All the people I most admire, and all the cleverest people - all humans.