Episode 15 - Information, a demon, and what life is

Tomlin Hawthorne

I'm thinking of something. It could be anything at all. An object, an animal, a person, a place. You're allowed to ask me questions, and I'll answer only yes or no.

Tomlin Hawthorne

You get twenty questions.

Tomlin Hawthorne

How many different things could you possibly tell apart with just twenty yes-or-no answers?

Tomlin Hawthorne

A hundred? A thousand?

Tomlin Hawthorne

Just over a million.

Tomlin Hawthorne

Each good question cuts the possibilities in half. Animal or not. Bigger than a breadbox or not. And halving twenty times in a row takes you from a million down to one. If you played perfectly, you could identify any one of a million things.

Tomlin Hawthorne

That's the power of a yes or a no. Today we're going to find out that a yes-or-no has a physical cost, that it's bound up with entropy, and that it's the reason a very small demon failed to overthrow the laws of physics. And at the end we'll ask what you are.

Tomlin Hawthorne

Last time you sorted films into two piles: the ones you can tell are running backwards, and the ones you can't. The pendulum and the spinning coin look fine either way. The milk pouring itself out of the tea does not. And what the second pile shares is that something is spreading out: milk into tea, a wave into foam. Entropy going up. That's the arrow.

Tomlin Hawthorne

This is the final episode of Part Three, and it begins with a threat to everything we've just built. Today's question: can intelligence beat the Second Law?

Tomlin Hawthorne

The threat comes from James Clerk Maxwell. He's a Scot, born in 1831, and he will be the hero of Part Four, because he's the man who worked out what light is. Einstein kept his picture on the wall. But he also had a mischievous streak, and in 1867 he put a small bomb under the Second Law, in a letter to a friend.

Tomlin Hawthorne

Here's what he imagined.

Tomlin Hawthorne

Take a box full of air, all at one temperature. Divide it in two with a wall. In the wall there's a tiny door, just big enough for one molecule to pass.

Tomlin Hawthorne

Now remember what temperature is. It's the average speed of the molecules. But it's only an average. In any gas, some molecules are going faster than average and some slower. It's a crowd, with sprinters and dawdlers.

Tomlin Hawthorne

And beside the little door, Maxwell places a being. Very small, very sharp-eyed, very quick. Maxwell called it a neat-fingered being. His friend Kelvin, who had a better ear for publicity, called it a demon, and the name stuck.

Tomlin Hawthorne

The demon watches the molecules. When a fast one comes toward the door from the left, he opens it and lets it through to the right. When a slow one comes from the right, he lets it through to the left. Otherwise he keeps the door shut.

Tomlin Hawthorne

He does no work. The door is weightless and frictionless. He's not pushing anything. He's just a doorman, with a policy.

Tomlin Hawthorne

And after a while, what have we got? All the fast molecules on the right. All the slow ones on the left. The right side is hot. The left side is cold.

Tomlin Hawthorne

We've made a temperature difference out of nothing. Heat has flowed from a lukewarm box into a hot half and a cold half, with no effort. And once you have hot and cold, as Carnot told us, you can run an engine.

Tomlin Hawthorne

So: free power, for ever. The Second Law, broken by a creature with good eyesight.

Tomlin Hawthorne

Maxwell wasn't trying to build a perpetual motion machine. He was making a point, and it's the point from two episodes ago. The Second Law isn't like Newton's laws. It's statistical. It's about what crowds do when nobody's watching each member. And he wanted to show that if you could deal with molecules one at a time, the law wouldn't hold.

Tomlin Hawthorne

Fine. But the trouble is, the demon is not obviously impossible. It doesn't have to be a goblin. It could be a tiny machine, a valve with a sensor. And if any such thing could be built, the heat death is cancelled and the whole of thermodynamics has a hole in it.

Tomlin Hawthorne

So physicists set out to find what's wrong with the demon. It took them a hundred and fifteen years.

Tomlin Hawthorne

Before I tell you the answer, have a go. Where's the catch? What is the demon getting away with?

Tomlin Hawthorne

Most people say: he has to see the molecules. And seeing costs something. He'd need a torch. He'd have to shine light on them, and that light is energy, and it'd heat things up, and that would cancel out his gain.

Tomlin Hawthorne

That's a very good answer. It was the official answer for about thirty years, from the 1920s to the 1950s. A Hungarian, Leo Szilard, started that line of thought, and it's in all the older textbooks.

Tomlin Hawthorne

And it's wrong. It turns out you can, in principle, measure where a molecule is and how fast it's going at as small a cost as you like. Measuring isn't the problem.

Tomlin Hawthorne

The real answer came from two men at IBM, which was not, at the time, known for its contributions to the philosophy of physics. Rolf Landauer in 1961, and Charles Bennett, who finished the job in 1982.

Tomlin Hawthorne

Here's the argument. The demon has to decide, for each molecule, whether to open the door. To do that he has to know something about it: fast or slow. He has to hold that fact, at least for a moment. He needs a memory.

Tomlin Hawthorne

Now, he sorts billions upon billions of molecules. Each one is a fact to be stored. His notebook fills up. And no notebook is infinite.

Tomlin Hawthorne

So at some point, to keep going, he has to wipe the slate. He has to erase what he's written, to make room.

Tomlin Hawthorne

And Landauer proved something that nobody had suspected. Erasing information has an unavoidable cost. Every time you wipe one yes-or-no from a memory, one bit, you must release a certain minimum amount of heat into the surroundings. You can't get round it with cleverer engineering. It's a law of nature.

Tomlin Hawthorne

And when you add up the heat the demon has to give off in wiping his memory, it exactly makes up for the entropy he removed by sorting the molecules. To the last decimal. The books balance. The Second Law stands.

Tomlin Hawthorne

I think it's a wonderful resolution. The demon isn't defeated by the effort of opening the door. Or by the effort of looking. He's defeated by forgetting.

Tomlin Hawthorne

But what's come out of this is much bigger than the rescue of a law.

Tomlin Hawthorne

We started with something that sounds abstract. Information. A fact, a yes-or-no, a bit of knowledge. That sounds like the least physical thing there is.

Tomlin Hawthorne

And it turns out to be physical. Information is always stored in something: a mark on paper, a magnet on a disc, a charge in a chip, a connection in a brain. And handling it has a price, in heat, set by the same laws that govern steam engines.

Tomlin Hawthorne

Landauer had a slogan for it: information is physical. Your phone gets warm for many reasons, most of them down to imperfect engineering. But underneath them all is a floor that no engineering can remove, and it's there because computing means erasing, and erasing means heat.

Tomlin Hawthorne

Now let me bring in the other half of the story, which was happening at the same time, a few miles away.

Tomlin Hawthorne

Bell Telephone Laboratories, New Jersey, 1948. A thirty-two-year-old engineer called Claude Shannon publishes a paper called A Mathematical Theory of Communication.

Tomlin Hawthorne

Shannon was a singular character. He rode a unicycle along the corridors at Bell Labs, sometimes while juggling. He built a mechanical mouse that could learn its way through a maze. He built a machine whose only function, when you switched it on, was to reach out a hand and switch itself off. And in that one paper he more or less founded the age we live in.

Tomlin Hawthorne

He was working on a practical problem. The telephone company wanted to know how much stuff you can push down a wire. And Shannon realised that before you can answer that, you have to say what the stuff is. What is it that a message carries? How do you measure it?

Tomlin Hawthorne

His answer: a message carries information to the extent that it tells you something you didn't already know. Information is surprise. It's the removal of uncertainty.

Tomlin Hawthorne

And the unit is the answer to one yes-or-no question where you had no idea which way it would go. He called that a bit. Twenty questions, twenty bits, a million possibilities.

Tomlin Hawthorne

Here's a lovely consequence. English is very wasteful. If I write: f, u, c-n, r-d, t-h-s. You can read that. I've left out half the letters and you lost nothing, because the missing letters weren't telling you anything. You could have guessed them. Shannon measured it: something like half of written English is redundant. That's why you can understand a friend in a noisy bar, and why text can be compressed. A file that's been compressed is just English with the predictable parts squeezed out.

Tomlin Hawthorne

Now here's the part that matters for us. When Shannon wrote down his formula for how much information is in a message, how much uncertainty it removes, the formula was the same one that's on Boltzmann's tombstone. The same shape exactly. Counting the possibilities, on the scale of zeros.

Tomlin Hawthorne

There's a well-known story, possibly improved in the telling, that he wasn't sure what to call his new quantity. And the great mathematician John von Neumann advised him: call it entropy. For two reasons. First, your formula is already used under that name in physics. And second, and more important, nobody really knows what entropy is, so in any argument you'll always have the advantage.

Tomlin Hawthorne

But it's not a coincidence of formulas. Think back to the coins. The macrostate is what you can see: about half heads. The microstate is the full detail. And entropy counts how many microstates are hiding behind the one summary.

Tomlin Hawthorne

So you can say it this way. Entropy is how much you don't know. It's the number of yes-or-no questions you'd have to ask to find out exactly what every molecule is doing, given only what you can see from outside. High entropy means the summary tells you very little. Entropy is missing information.

Tomlin Hawthorne

That's why the demon nearly got away with it. He was using information, knowledge of each molecule, to lower entropy. He was trading one for the other. And the trade turned out to be exactly fair.

Tomlin Hawthorne

Now. Life.

Tomlin Hawthorne

Dublin, February 1943. Europe's at war; Ireland is neutral. And Erwin Schrödinger, one of the founders of quantum mechanics, an Austrian who'd left after the Nazis took over, is living there in a somewhat irregular household and giving a set of public lectures at Trinity College. The audience includes the Irish prime minister. The title is: What Is Life?

Tomlin Hawthorne

It's an odd thing for a physicist to attempt, and he says so. But he asks a physicist's question. A living thing is a lump of matter. And lumps of matter, by the Second Law, run down. They spread, they cool, they come to equilibrium with their surroundings. That's what a dead thing does, rather quickly. So how does a living one avoid it? How does it stay so improbable for so long?

Tomlin Hawthorne

He gives two answers, and they were both prophetic.

Tomlin Hawthorne

The first: a living thing keeps its own entropy low by exporting entropy to its surroundings. It takes in order and gives out disorder. He put it that an organism feeds on negative entropy. You'll recognise this. It's the Sun and the Earth from two episodes ago. We eat concentrated, tidy food and give off heat and waste. We're each a small Carnot engine that has learned to maintain itself.

Tomlin Hawthorne

The second answer is the remarkable one. How does a living thing know how to rebuild itself? How does a cell pass its plan on to its daughters, faithfully, for millions of generations, without it blurring? Schrödinger reasoned that the plan must be stored in a molecule. A big one, stable enough to last, but not a simple repeating pattern like a salt crystal, because a repeating pattern can't say anything. It must be what he called an aperiodic crystal. A structure with no repetition, in which the arrangement of the atoms is itself a message. A code-script, he called it.

Tomlin Hawthorne

This was 1943. Nobody knew what genes were made of. Ten years later, Watson and Crick found the double helix, and it's just what he described: a long, stable molecule whose sequence spells out instructions in a four-letter alphabet. Both of them said that Schrödinger's little book had helped set them on the path.

Tomlin Hawthorne

So here's a physicist's answer to what you are. You're a pattern that persists. Not a thing, exactly. The atoms in you are replaced, most of them, every few years. You're more like a whirlpool, or a flame: a shape that matter flows through. You keep your shape by taking in order and shedding disorder. And you carry a memory, a set of instructions, written in bits, that lets the pattern be copied.

Tomlin Hawthorne

Entropy, information, and a code. That's not all there is to say about a living thing. But I find it extraordinary that it's sayable at all.

Tomlin Hawthorne

The myth of the week. It's that information means meaning.

Tomlin Hawthorne

In Shannon's sense, it doesn't. His measure is perfectly indifferent to whether a message is true, or important, or sense at all. In fact a page of random letters carries more information, by his measure, than a page of Shakespeare. Every letter of the gibberish is a surprise. Shakespeare is full of patterns you could have predicted.

Tomlin Hawthorne

Shannon knew this perfectly well and said so on the first page. Meaning, he said, is not the engineer's problem. It's yours.

Tomlin Hawthorne

And the question to take away. It comes from John Wheeler, a great American physicist who liked a slogan. Late in his life he came to think that information might be more basic than matter. That at the bottom of everything there are no particles and no fields, just yes-or-no answers. He put it in three words: it from bit.

Tomlin Hawthorne

Is that physics? Is it philosophy? Or is it a metaphor that's got out of hand?

Tomlin Hawthorne

Before you decide, read Borges's story The Library of Babel. It's seven pages. A universe that consists of a library containing every possible book: every arrangement of letters, the true ones and the false ones and the nonsense. Everything that can be written is on the shelves. And so, Borges sees, the library tells you nothing at all. I think it's the best thing ever written about information, and he wrote it seven years before Shannon.

Tomlin Hawthorne

Your challenge. Play Twenty Questions with someone this week. But score it. Every time the answer to your question is a surprise to you, a genuine fifty-fifty, you've gained a full bit. Every time you were fairly sure what the answer would be, you've gained much less. See how few questions you can win in. And notice that the skill of the game is exactly the skill of asking the question whose answer you can least predict.

Tomlin Hawthorne

That's the end of Part Three. We have atoms, heat, entropy, time and information. Next time we start Part Four, on a stormy night by Lake Geneva, with an eighteen-year-old novelist, a dead frog's leg, and the discovery that empty space is not empty.