Episode 13 - Entropy counting the ways

Tomlin Hawthorne

If you have a pack of cards nearby, pick it up and give it a good shuffle. If you haven't, imagine one.

Tomlin Hawthorne

Now look at the order the cards are in. Here's my question. Has that exact order ever come up before? In any shuffle, of any pack, in any card game, in the whole history of the world?

Tomlin Hawthorne

No. Almost certainly not. And it won't come up again, either. Not if every person on Earth shuffled a pack every second from now until the Sun burns out.

Tomlin Hawthorne

The number of ways to arrange fifty-two cards is an eight followed by sixty-seven zeros. That's in the same league as the number of atoms in our galaxy. Every time you shuffle properly, you're holding something that has never existed before.

Tomlin Hawthorne

Today that pack of cards is going to explain why eggs don't unscramble, why coffee goes cold, and why everything, eventually, runs down.

Tomlin Hawthorne

Last time, a rubber band on your lip. If you tried it, you found it warms when you stretch it and cools when you let it go, which is the opposite of what most people guess. We'll be able to say why by the end of today. Hold on to it.

Tomlin Hawthorne

Now. This is the episode I've been pointing at since the series began. And I want to open with a challenge that was issued to people like you, specifically.

Tomlin Hawthorne

In May 1959, a man called C. P. Snow gave a lecture in Cambridge called The Two Cultures. Snow was an unusual figure. He'd trained as a physicist, he'd become a successful novelist, and he was a senior civil servant. So he spent his days with scientists and his evenings with writers, and he'd noticed that the two groups had more or less stopped speaking to each other.

Tomlin Hawthorne

And he told a story against the literary side. He'd been at plenty of parties, he said, where cultivated people enjoyed themselves remarking on how badly read scientists were. A couple of times he'd lost patience and asked the room a question back. Could any of them tell him what the Second Law of Thermodynamics says?

Tomlin Hawthorne

Nobody could, and they weren't pleased to be asked. And yet, Snow said, in scientific terms it was about as basic as asking whether they'd ever read any Shakespeare.

Tomlin Hawthorne

That stung in 1959 and it still stings. So let's deal with it. In the next fifteen minutes you're going to learn the Second Law of Thermodynamics. Not just what it says. Why it's true. And I'd suggest that puts you ahead of a fair number of scientists.

Tomlin Hawthorne

We got to the first version last time. Heat flows from hot to cold, and never, on its own, the other way.

Tomlin Hawthorne

But it's wider than heat. Remember Thomasina, in Arcadia, stirring her rice pudding. The jam spreads out into the pudding, and if you stir backwards, it doesn't gather itself up again. A glass falls off the table and smashes. The pieces never leap back up and assemble. You can scramble an egg. Nobody has ever unscrambled one.

Tomlin Hawthorne

There's a whole class of things that happen in one direction only. And here's why that's a real puzzle. Think about what the egg is made of. Atoms. And when two atoms collide, they obey Newton's laws. And Newton's laws work perfectly well backwards. Film two billiard balls colliding, run the film in reverse, and what you see is a perfectly good collision. Nothing looks wrong.

Tomlin Hawthorne

So every single thing the atoms do is reversible. And the egg is nothing but atoms. So why isn't the egg reversible?

Tomlin Hawthorne

That question defeated everyone for decades. The man who answered it was Ludwig Boltzmann. You met him last time on the losing side of the argument about atoms. He was a big, bearded, short-sighted Austrian, a wonderful lecturer, a decent pianist, moody, generous, and prone to terrible depressions. He's the hero of this episode. And his answer is so simple that the first time you hear it, you may think you've missed something.

Tomlin Hawthorne

Here it is, with the cards.

Tomlin Hawthorne

Take a brand new pack, straight from the wrapper. It's in perfect order. Ace to king of spades, then hearts, and so on.

Tomlin Hawthorne

Shuffle it. What do you get? A mess. Shuffle again. A different mess. You can shuffle for the rest of your life and you'll never see it go back to perfect order.

Tomlin Hawthorne

Why not? Is there a law of nature against it? Does shuffling prefer mess?

Tomlin Hawthorne

No. The shuffle doesn't care. Every arrangement of the cards is exactly as likely as every other. The perfectly ordered one is just as likely as whatever you're holding right now.

Tomlin Hawthorne

It's simply that there's one way for the pack to be in perfect order. And there are an eight with sixty-seven zeros ways for it not to be. When you shuffle, you're picking an arrangement at random. And nearly all of them are messes.

Tomlin Hawthorne

Order isn't forbidden. It's outnumbered.

Tomlin Hawthorne

That's the whole idea. Everything else is detail. But let me make it more exact, with some coins, because we need one distinction.

Tomlin Hawthorne

Suppose I've got a hundred coins in a box. I shake the box and look in.

Tomlin Hawthorne

Now there are two different ways I could describe what I see. I could give you the full details: coin one is heads, coin two is tails, coin three is tails, and so on, all the way to a hundred. Or I could just give you the summary: forty-eight heads.

Tomlin Hawthorne

Physicists have names for these. The full details, every single coin, is called the microstate. The summary, what you'd notice at a glance, is the macrostate.

Tomlin Hawthorne

Here's the point. How many microstates give you the summary "all hundred heads"? One. There's only one way to do it.

Tomlin Hawthorne

How many give you "ninety-nine heads, one tail"? A hundred. The tail could be any of the coins.

Tomlin Hawthorne

How many give you "fifty heads, fifty tails"? About a hundred billion billion billion. A one with twenty-nine zeros.

Tomlin Hawthorne

So when I shake the box, I'll never see a hundred heads. Not because anything prevents it. Because the fifty-fifty summary can happen in vastly, absurdly more ways. Shake the box and you land, every time, somewhere near the middle.

Tomlin Hawthorne

Now do it with the egg. An egg has, let's say, a million billion billion molecules. There's a certain number of ways those molecules can be arranged that we'd look at and say: that's an intact egg, yolk in the middle, white around it. And there's a number of ways we'd look at and say: that's scrambled.

Tomlin Hawthorne

The second number is bigger. Not twice as big. Bigger by a factor that has more zeros than there are atoms in the egg.

Tomlin Hawthorne

So when you take a fork to it and jostle the molecules about, they wander from arrangement to arrangement. And practically every arrangement they can wander into is a scrambled one. Could the fork, by sheer luck, jostle them back to a yolk? Yes. Nothing forbids it. You'd just need to keep stirring for a length of time that makes the age of the universe look like a blink.

Tomlin Hawthorne

And that is the answer to the puzzle. Each collision is reversible. But the scrambled states outnumber the neat ones so overwhelmingly that the traffic only ever goes one way.

Tomlin Hawthorne

This count has a name. The number of microstates that all look the same from outside. It's called entropy. Strictly, it's measured on the kind of scale we met in episode two, where you count the zeros and not the number itself. But that's what it is. Entropy is a count of the ways.

Tomlin Hawthorne

So here is the Second Law, and you can take this to your next gathering of the highly educated. In any process, left to itself, entropy increases. Things move from states that can happen in few ways to states that can happen in many. That's all. That's the whole of it.

Tomlin Hawthorne

If you go to the Central Cemetery in Vienna, where Beethoven and Schubert and Brahms are buried, you'll find Boltzmann's grave. There's a white marble bust of him looking stern. And above the bust, carved into the stone, there's an equation. S equals k log W.

Tomlin Hawthorne

Let me read it for you the way we read Newton's law in episode five. S is entropy. W is the number of ways. Log means: count the zeros. And k is just a fixed number that converts the units.

Tomlin Hawthorne

So the stone says: entropy is the number of ways, on the scale of zeros. That's what's on his tombstone. I can't think of another equation that anyone wanted on their grave.

Tomlin Hawthorne

Now, some things follow from this that I want you to see.

Tomlin Hawthorne

First. The Second Law is a different kind of law from Newton's. Newton says: this will happen. Boltzmann says: this will happen, because the alternative is too improbable to bother about. It's a law of large numbers. In principle, all the air in the room you're sitting in could, in the next second, rush into one corner and leave you gasping. Every molecule is free to go there. It's not against any rule. It's just that the arrangement "air spread through the room" outnumbers "air in the corner" by so much that you can stop worrying.

Tomlin Hawthorne

Second. Why does heat flow from hot to cold? Put a hot thing against a cold thing. The fast atoms knock into the slow ones and the energy gets shared out. Could it go the other way, the slow ones handing their energy to the fast? Yes. But there are overwhelmingly more ways to have the energy shared out than to have it bunched up on one side. So it shares. Carnot's law is card-shuffling.

Tomlin Hawthorne

Third, the rubber band. A rubber band is made of long, tangled molecules like cooked spaghetti. When you stretch it, you pull them straight. Straight is tidy: there are fewer ways to be straight than tangled. So you've lowered the band's entropy, by force, and that has to be paid for. It's paid in heat. The band warms. Let it go, the molecules tangle up again, and it cools. You felt the Second Law on your lip.

Tomlin Hawthorne

Now, the myth of the week, and it's really two.

Tomlin Hawthorne

The first is that entropy means disorder, so my desk has high entropy. You'll hear this constantly. It's a fair first sketch and it leads people astray. Messiness is a judgement. Your desk is messy to your partner and perfectly organised to you. Entropy isn't a judgement. It's a count. A better word than disorder is spread. Energy spread out, molecules spread out, among as many arrangements as possible.

Tomlin Hawthorne

The second myth is more serious. You'll sometimes hear it said that life, or evolution, breaks the Second Law. A tree builds itself out of air and water. A baby assembles itself from soup. That's order appearing from disorder. Isn't that entropy going down?

Tomlin Hawthorne

It's a good question, and the answer is rather beautiful.

Tomlin Hawthorne

The law says entropy increases in a system left to itself. Sealed off. And the Earth isn't sealed off. There's a very large hot thing in the sky.

Tomlin Hawthorne

Here's what happens. The Earth takes in energy from the Sun as visible light. And it sends exactly the same amount of energy back out to space, as infrared, as heat glow. It has to, or it would keep getting hotter. So in terms of energy, we break even. We keep none of it.

Tomlin Hawthorne

But the light that arrives comes in a fairly small number of big packets. And the heat that leaves goes out as a much larger number of small ones. About twenty going out for every one that came in.

Tomlin Hawthorne

Same energy. Twenty times as many pieces. That's energy that has been spread. That's a huge increase in entropy, shipped off into the dark.

Tomlin Hawthorne

And that is what life runs on. Not on the Sun's energy, exactly. We give all of that back. We run on the fact that it arrives concentrated and leaves dilute. Everything alive is a little eddy of order, paid for by a much bigger increase in disorder that gets radiated into the night sky. A tree is not a violation of the Second Law. A tree is one of the ways the Second Law gets done.

Tomlin Hawthorne

So is your thinking, at this moment. You're tidying up some ideas, and you're paying for it by turning your breakfast into warm air.

Tomlin Hawthorne

I have to tell you how Boltzmann's story ended.

Tomlin Hawthorne

He spent thirty years defending this. And the opposition wasn't mild. Mach and his allies didn't believe in atoms, so a theory built on counting the arrangements of atoms was, to them, a fantasy built on a fiction. Boltzmann felt, more and more, that he was one man alone against the tide.

Tomlin Hawthorne

In the summer of 1906 he took his wife and daughter on holiday to Duino, a village on the Adriatic coast near Trieste. He was sixty-two, his eyesight was nearly gone, and he was in the grip of one of his depressions. On the fifth of September, while his family was out swimming, he took his own life.

Tomlin Hawthorne

This was a year after Einstein's paper on the jittering grains. Perrin's measurements, the ones that would convince everybody that atoms are real, were two years away. He very nearly lived to see it.

Tomlin Hawthorne

There's a coda, which I offer you without comment. Six years later a poet came to stay at the castle at Duino, and began writing there the sequence that made his name. Rilke's Duino Elegies.

Tomlin Hawthorne

A question to take with you.

Tomlin Hawthorne

The Second Law is a law that could be broken. It never will be, in any way you'd notice. But there's no wall there. It's a matter of overwhelming odds.

Tomlin Hawthorne

Is that a law in the same sense as Newton's? Or is it more like the laws you meet in the social sciences? That prices rise when supply falls. That large populations behave predictably when single people don't. Nobody thinks an individual is forced to behave as the average does, and yet the average is utterly reliable.

Tomlin Hawthorne

Might Boltzmann's kind of law be closer to your kind than to Newton's? And if the deepest law about time and change is statistical, what does that do to the old picture of physics as the science of certainties?

Tomlin Hawthorne

Your challenge. Ten coins and a box, or a mug with your hand over the top.

Tomlin Hawthorne

Put all ten in heads up. Shake, tip them out, and count the heads. Write it down. Do that twenty times.

Tomlin Hawthorne

Look at your list. How often did you get all ten heads? How often did you get four, five or six? The pile-up in the middle is entropy. That's all it is.

Tomlin Hawthorne

Then ask yourself: what if it were a hundred coins? Or as many coins as there are molecules in an egg?

Tomlin Hawthorne

Next time, we follow this to its conclusion. If the Second Law is why eggs only scramble, is it also why time only goes forward? Why do you remember yesterday and not tomorrow? The answer is going to take us, rather unexpectedly, all the way back to the beginning of the universe.