Status-Q

Current AI systems excel at generating large amounts of text.  You can give ChatGPT a few bullet points, and it will turn them into a paragraph, an email, an essay. So we’re all going to get a lot more text in the future.

AIs will soon force us to confront the fact that we live in a society where, for many situations, large amounts of text are required, expected, or interpreted in some way as being preferable to brevity.   (On that subject, one of the nicest phrases I’ve heard in AI-related discussions recently is, “Why would I want to read something that somebody couldn’t be bothered to write?” )

Anyway, you generate this text from your bullet points, and then you send it to a colleague.   But they’re swamped with the number of other people who are doing the same.  So they use an AI to summarize your email back down to bullet points.   This process of expanding and then contracting made me think of a heat pump, or a refrigerator.   Except that in this case, the expansion phase produces a lot of hot air.

This also means there are two very fallible intermediaries inserted into your communication channel. The process reminded me of two stories from my childhood.  

The first was in the late 70s, when I first saw (and loved) Star Wars.  I always wondered, though, why C3PO always spoke to R2D2 in English, and R2D2 always responded in beeps, clicks and whistles.  It was clear that he could understand English, and I realised even then that this was a much harder problem than generating it.  Why, amidst all that technology, had nobody thought of fitting him with a small loudspeaker and a speech-synthesis chip?  On the other hand, perhaps R2 was the smart one: was the English language really the best way for two machine intelligences to communicate?

The second story was a (possibly aprocryphal) one about an early computer-based system that could do English-Russian language translation; a very challenging task at the time.  They gave it the phrase “Out of sight, out of mind” and asked it to turn it into Russian.  They then took the output and told it to translate back into English. The result? “Invisible Idiot”.

Wouldn’t it be better if you just sent your bullet points to your colleague directly?

My thanks to Kit Hodsdon, who, responding to yesterday’s post, pointed out that there was a name for a phenomenon related to something I discussed there: the Gell-Mann Amnesia Effect.

This term was first coined by Michael Crichton, and to quote the Wikipedia page linked above, it describes “the tendency of individuals to critically assess media reports in a domain they are knowledgeable about, yet continue to trust reporting in other areas despite recognizing similar potential inaccuracies.”

The page is worth reading for more info. For example:

‘The Gell-Mann amnesia effect is similar to Erwin Knoll’s law of media accuracy, which states: “Everything you read in the newspapers is absolutely true except for the rare story of which you happen to have firsthand knowledge.” ‘

And what about the name? Crichton said he had once discussed the effect with the physicist Murray Gell-Mann “and by dropping a famous name I imply greater importance to myself, and to the effect, than it would otherwise have”!

Regular readers might assume that I spurn all things AI-related, and this is not the case. I do use and occasionally derive benefit from the tools that tend to come under this all-embracing phrase du jour. But it does sometimes seem as if, for general questions, the LLMs can throw up wrong answers as often as right ones, and, like others, I then feel a compulsion to point this out. I had one week recently where three different ‘AI’ systems, in three completely different contexts, gave me three answers that were demonstrably wrong on key points.

So it’s important to know the answer before you ask them a question, in which case…

The real problem is the certainty with which assertions are made. There’s no expression of doubt, no humming and hawing, before the response.

Having been listening on my recent travels to the wonderful audiobooks of Patrick O’Brian’s stories, I thought it would be fun to ask ChatGPT about archaic naval terminology – did it know what was meant by ‘Three points off the larboard bow’.

It started with a cheery phrase along the lines of “Ah! Nautical terms! Always a source of interest.” and then went on to give various bits of authoritative information, even with helpful diagrams, but including the assertion that ‘A point in this context is one-eighth of a compass’s 360 degrees, so one point is 11.25 degrees.’

Mmm. Do a little calculation and you’ll see the problem.

I’m paraphrasing slightly, because when I went back to copy and paste the exact text, it had conveniently lost its history, and when I asked again, it made no mention of the ‘one-eighth’. (There are actually 32 ‘points’ on a compass, so the 11.25 degrees bit was correct. Four points are 45 degrees.) It’s a worrying thought: perhaps the system could detect that I was surprised and go back and cover its mistakes!

Anyway, it did get the rest of it right. ‘Larboard’ is an old word for ‘port’, which was abandoned in the mid-19th century because it sounded too similar to ‘starboard’. If yelled from the mast-top in the heat of battle in a gale, I guess this could cause problems. So ‘three points off the larboard bow’ means (roughly) 34 degrees to the left of the direction in which the ship is pointing.

Many, many years ago, when a couple of the projects I was working on started to get some press coverage, I remember noticing that, in general, every single article about my work contained errors. (There were a couple of exceptions — the Economist was one — but they were notable for their rarity.) These were generally unimportant mistakes, and they were made by well-intentioned journalists working for reputable papers back in the day when they could afford to do real research but who were, after all, humans. It was an important lesson for me in my youth: you can detect the errors in the things you know about, but remember that every article you read about anything is probably similarly flawed.

This is why I think it is important to keep taking note of the times we can detect wrong answers, because they will also happen in places that we cannot so easily detect. I wonder if adding more human-like phrases to the output of LLMs will enable us to take what they say with a pinch of salt, too? (Sea salt, of course.)

I realise that I should be slightly more cautious about my use of phrases like ‘today’ and ‘this morning’ in my posts, since quite a few of my readers receive them by email the following day. In the days of the Trump regime, phrases like ‘The news looks a bit better this morning’ have a very limited lifespan!

Ah well. Here’s yesterday’s (or the day before yesterday’s) Matt cartoon:

I came across an amazing article by Marcin Wichary at the weekend, about a font you’ve probably never heard of: Gorton. And that’s despite the fact that you probably encounter it very regularly.

The Hardest Working Font in Manhattan is a long web page, but even if you don’t read the whole thing, there’ll be plenty to catch your interest if you skip bits from time to time.  There are even some interactive demos.  But do read most of it, too.

It’s always good to be reminded, amidst so much of the online rubbish we see today, just how much brilliant work is also out there.

A sombre mood here in our half-American household. The phrase that keeps running through my head is…

Elect me once, shame on me.
Elect me twice…