Diamond Age: Requiem

AI stunting, what it might be good for, and how it's not really intelligence

We are again being told that AI is here.  And perhaps it is this time.

Strangely, I’ve had a rather strong opinion on this one since Daniel Dennett showed up to guest lecture my philosophy class after giving a talk at the university.  We had a rather spirited exchange on the topic of qualia, and I maintain to this day that his answers at the time were thoroughly unsatisfactory.

Qualia, you see, is the “what it’s like to be” of an entity — the little crackling bolts in that numinous cloud of sentience that does the perceiving of things.  The taste of wine, the smell of charcoal.  The quanta of the experience of sentience itself.

Some like Dennett effectively deny that these phenomena are worthy of consideration, while others maintain they’re the whole point and closely linked with the reason strong AI won’t be achieved.  A third camp holds it doesn’t matter because they’ll get good enough that we will ascribe qualia to them regardless.  Decent bit of AI ethic centers around this disagreement.

Anyhow... where was all this going?  Ah yes:  the recently arrived AIs.

Here’s what’s strange about intelligence — it’s not overly difficult to create.  We do it all the time, in fact, we just usually call them ‘children.’  And what’s the hard problem with children?  Raising them, ultimately.

Put another way, pedagogy is the hard problem.

Our current AI models are essentially recursion machines for human behavior, which means they run a sort of analysis on all past human behaviors and attempt to sequentially output the ‘next best’ behavior.  But really fast, with a lot of computational horsepower.  Some hold a fair number of humans aren’t much more than this, I’d agree this mirroring/modeling is one of our core capabilities, but it’s hard to say that’s all we are.  How, then, do we change?  How do we continually tackle the unknown?

Humans aren’t just known for our mammalian pack nature (which is where the modeling behavior comes from), we’re highly adaptable conquerors of our environment, even when that environment is constantly changing.  There’s something to that adaptiveness that’s necessarily more than the sum of previous behaviors, and herein lies the rub:  the AIs only have the previous behaviors as input.  They are fundamentally designed as regression engines.

The counter here, one supposes, is that perhaps they can be extrapolation engines too but... I’m not sure how.  That bit is as yet insufficiently explained and it’s the reason I’m somewhat muted in my excitement around even this new generation of AI. 

We’ve taught them how to mirror, but I’m as yet unconvinced we’ve taught them how to learn.

But it’s still quite interesting because behavioral mirroring is a Very Important Thing™ even if it’s not everything.  Where is it important? Well... whenever we’re trying to disseminate baseline human knowledge or behavior.  Where we might imagine those contexts being important?

Teaching is an obvious one.  In some sense, the entire goal of pedagogy is precisely the distribution of what is already ‘known by most,’ be that generalized knowledge.

Certainly one can imagine basic application almost immediately, but I’d allow room for some proper futurism here too:  learning modalities have stylistic clusters, and we could imagine one of these sophisticated regression models valuably fusing two regression sets in real-time: 1) the set of knowledge to be distributed, and 2) the set of knowledge about how to optimally present this knowledge to <target modality>.  Already better than a lot of people receive in mass education, and that’s before layering in 3) the (ever-growing) set of data around how the specific individual learns best.

It’s a fair bit of work to tune this, but in my view with some modest suspension of disbelief one can imagine us heading toward the world depicted in Neal Stephenson’s speculative fiction The Diamond Age, which contains as a central trope the notion of a very advanced ‘adaptive pedagogy device.’  His depiction would also require layering in an environment-dictated triage process to re-order in real-time how the curriculum is presented based on the needs of the subject, so we need to layer in 4) the set of data about what behaviors are baseline adaptive in common human contexts, and 5) some ability (presumably from wearables or whatever) to appropriately classify context.  As with a lot of things in AI, I expect that (5) would end up being the most challenging of all of these and present myriad ethical dilemmas.  Regardless, the tools to build this specific piece of future tech seem to be lining up and that’s pretty cool.  Even just executing on 1-3 here seems like an extremely powerful MVP and not something that relies on unspecified techno-magic, just a lot of work.

I recommend reading the book if this topic is at all interesting to you, Neal is a fun read.

Other contexts where this kind of tech could be interesting seem fraught with ethical implications.  Take therapy, for example — do we feel comfortable defining a set of canonical human behaviors that we wean tot steer people toward?  Feels gross to me.  Sure, we might want to steer people away from certain behaviors (effectively a choice we have already made), but being outright normative about desired behaviors feels pretty icky.  Perhaps the tech is useful for certain types of rehab or therapeutic approaches which have a well-defined target outcome, but even there the subtle nudge toward behavioral conformity just kind of offends my intuition on the matter.

There will, of course, be productivity enhancements from this tech — Tier 1 support is an obvious application here, but if we’re being honest... aren’t we already attempting to automate some of this with robo-responders,  wiki FAQs, and other low level tooling?  Perhaps this offers a better experience, and allows us to scale that effort to passable ‘Tier 1.5’ support.  Useful, and there are probably at least a handful of sizable software companies to be built around this sort of thinking, but hardly a step function change to the economy.

For me, any revolution here will be a revolution in how quickly we are able to train people up to C+/B- proficiency on huge swaths of human knowledge.  Kindergarten through Community College for the whole world available at SaaS pricing.  Seems like a pretty cool world.

But this is all just a priori theorycrafting in the realm of whats possible: even the most recent models are relatively easy to fool, and do not produce an output that’s consistent enough to be trustworthy.  Do they produce impressive results?  Yes!  But only in a manner Karl Popper would find wildly vexing.

We have a tendency to measure intelligence based on its peaks, and this is a mistake — robustness is arguably more important.  Shallower valleys (risk-biased intelligence) are more valuable to a living organism than higher peaks (insight-biased intelligence) and so much of what is useful about being ‘smart’ is the ability to dodge pitfalls.

Intelligence, one might argue, is “that which solves for the non-ergodic,” and since researchers are building fundamentally ergodic solution engines here… well they’re sort of not even on the right track to solving it yet.

To put this in finance terms:  the AI industry is dazzling us with a few great trades instead of building consistently profitable strategies.  Intelligence needs to run like a high Sharpe arb strategy to be useful, and the type of things we’re building approach it like a Wall Street Bets punter.

In the end I still have to fade Dennett, because I would hold that the presence of qualia in training proto-intelligence is what turns it into intelligence.  The pain of the hot stove is not a statistically-encoded lesson.  Intelligence is a phenomenon which may only be develop fully in sensory creatures, because the senses and the environment are the lathe that sculpts a mind into being. The lack of this puts us in a realm that Grady Booch described well — the current state of AI is a bit of a stunt that is still most interesting to the less-technical, a still too error-prone to be immediately useful.

Grady Booch @Grady_Booch

#ChatGPT: remarkably coherent prose full of half-truths intermingled with boldly stated falsehoods masquerading as truths. To an outsider, this is astonishing. To an expert in this domain, this is nothing more than well-formed statistical nonsense at scale.

This is not intelligence, not really.

It may be prove to be ‘highly nutritious substrate for spreading knowledge’ if we can tune up those deep valleys and if so that’s... well it’s pretty interesting, in much the same way the internet putting a Library of Alexandria in each of our pockets was interesting.