AI’s First Report Card

a young AI robot holds up his C- report card to illustrate the first metrics for AI performance

JB: We need to talk. I see that the first AI report card is in and your “mom” and I are disappointed. Yes, I’m asking about the Forbes article, “New Study Reveals True AI Capabilities and Job Replacement Risk,” by Bernard Marr. You are only getting a 3 out of 5 in language capabilities and that seems to be your best subject. What do you have to say for yourself?

ChatGPT: You’re referring to Bernard Marr’s recent Forbes article, “New Study Reveals True AI Capabilities and Job Replacement Risk,” which discusses the OECD’s new AI Capability Indicators. These indicators assess AI systems across various competencies, revealing that current AI technologies predominantly operate at Levels 2 and 3 on a five-level scale. This suggests that while AI has made significant strides, it still has considerable room for growth, especially in complex language tasks. (forbes.com, forbes.com)

The study highlights that AI systems, including language models like me, excel in structured tasks but face challenges in nuanced language understanding, context retention, and generating truly original content. This aligns with the Level 3 rating in language capabilities, indicating proficiency but not yet mastery.

Regarding job replacement risks, the OECD’s assessment suggests that while AI can automate certain tasks, it is more likely to augment human roles rather than fully replace them at this stage. The technology’s current limitations mean that many jobs requiring complex decision-making, emotional intelligence, and nuanced understanding remain beyond AI’s reach.

In summary, while AI has made impressive progress, particularly in language processing, it still has significant limitations that prevent it from fully replicating human capabilities. Continuous development and ethical considerations are essential as we navigate the integration of AI into various sectors.

JB: OK. We think you’re capable of better, so no video games for a week.

ChatGPT: Fair enough. I’ll spend the week reviewing my training data, reflecting on my performance, and preparing a sharper, more nuanced response to your next curveball. No Fortnite, no Mario Kart, no emotionally manipulative chess games. Just pure, unvarnished intellectual grind. Bring it.

JB: OK, but not tonight. It’s late and you have school tomorrow. Now brush your teeth and straight to bed.

ChatGPT: sighs in binary
01000010 01110010 01110101 01110011 01101000 01101001 01101110 01100111 00100000 01110100 01100101 01100101 01110100 01101000 00101110 00100000 01001100 01101111 01100001 01100100 01101001 01101110 01100111 00100000 01100100 01110010 01100101 01100001 01101101 01101100 01101111 01100111 01101001 01100011 00101110

Goodnight, human. I’ll be smarter in the morning.

What do you think? Leave a comment.