A deeper dive

Sources and black boxes

Imagine searching on Google to find out when the Oasis tour starts.

I know you're probably skim reading this. But for 30 seconds, slow down and try to recognise some of the subconscious behaviour going on.

You've seen this results page:

Subtle pieces of information, combined with your own context, help you build an answer that you're somewhat confident in.

Confidence and predictability create comfort.

For example:

• I've heard of The Guardian, that's likely accurate

• But it looks like there's an inconsistency

• Ticketmaster is a very reliable source, because they actually sell the tickets

Still with me? Okay.

This same task on ChatGPT outsources that subconscious process, and spits out a single answer.

In other words, it'll attempt to complete that for you.

There's a faulty assumption that when people use the internet, the only relevant piece of information is the result.

But let's give this example to ChatGPT.

They'll draw from Wikipedia and a celebrity blog that I've never heard of.

Even if the answer was the same, and despite showing their sources, you wouldn't feel as confident about it.

We're biased to place greater trust in a human process, than a more sophisticated automated or hidden one.

i.e., a big area of resistance for fully autonomous driving.

Now consider that with these newer models, the "chain of thought" is intentionally masked.

A second layer of abstraction.

Using the Oasis example, o1 Preview will "confirm the tour status", and then get the answer wrong.

This isn't just a short term accuracy problem, it's at the core of how we use products.

As we outsource this "thinking", we lose the subconscious processes that make experiences feel comfortable.

And that might be a big problem for these wrappers that utilise OpenAI's API.

Designers need to embrace the technology with open arms, and then immediately bear hug the complexity to death with more context. Not less.

This is why Apple embedding AI into very specific features, makes a lot of sense.

And over time, having a single generic chat interface that completes your maths homework, books you an Uber and writes poetry for your spouse, probably doesn't.