Baldur Bjarnason:

You know what Generative AI is in terms of how it presents to you as software: clever chatbots that do or say things in response to what you say: your prompt. Some of those responses are useful, and they give you an impression of sophisticated comprehension. The models that generate text are fluent and often quite engaging.

This fluency is misleading. What Bender and Gebru meant when they coined the term stochastic parrot wasn’t to imply that these are, indeed, the new bird brains of Silicon Valley, but that they are unthinking text synthesis engines that just repeat phrases. They are the proverbial parrot who echoes without thinking, not the actual parrot who is capable of complex reasoning and problem-solving.

A zombie parrot, if you will, that screams for brains because it has none.

The fluency of the zombie parrot—the unerring confidence and a style of writing that some find endearing—creates a strong illusion of intelligence.

Every other time we read text, we are engaging with the product of another mind. We are so used to the idea of text as a representation of another person’s thoughts that we have come to mistake their writing for their thoughts. But they aren’t. Text and media are tools that authors and artists create to let people change their own state of mind—hopefully in specific ways to form the image or effect the author was after. […]

These language models are interactive but static snapshots of the probability distributions of a written language. […]

That’s what distinguishes biological minds from these algorithmic hindsight factories …