Glonce

Blog post defines entropy for a Markov chain using Boltzmann's counting method

miscThe Glonce editorsHacker News🔥48Glonce rating48Impact45Novelty30Relevance60Surprise20Players15Credibility75
Blog post defines entropy for a Markov chain using Boltzmann's counting method

A blog post sets out to define entropy for a Markov chain, building from classical thermodynamics up to statistical mechanics. It starts from Clausius's 1865 definition: by decomposing a physical process into a chain of engines, Clausius showed that entropy always increases for irreversible processes and stays unchanged for reversible ones, such as Carnot's ideal engine, which is the second law of thermodynamics. Entropy itself cannot be read off a thermometer or a ruler, the post notes, but it stays useful because other, directly measurable quantities can be derived from it. The author has previously written, more vaguely, about 'life as entropy,' an idea motivated by Schrödinger's 1944 book What Is Life? and its concept of negentropy: life appears to maintain order by feeding on the energy around it and reducing its own local disorder. To make that idea concrete, the post works through toy models: build a simplified system, find a mathematically consistent way to define entropy inside it, then simulate the model to see how entropy evolves. The chosen example is Dyson's toy model of a cell, a Markov chain of N sites where each site can be empty, active or inactive; earlier work, described as the author's 'last post,' showed the model converges to an equilibrium, though the exact equilibrium probabilities are described only as 'a complicated function of fixed points.'

The obstacle is that Clausius's original entropy is a function of work and temperature, concepts that do not obviously apply to an abstract Markov chain, so the post turns to Boltzmann's approach instead. Boltzmann related entropy to the number of distinct configurations, or microstates, compatible with a system's observed macroscopic state, such as its temperature, pressure and volume. The post's illustration: a gas of identical molecules frozen at absolute zero, each fixed in place at zero velocity, has few possible configurations, while a hot gas has many, so temperature, entropy and the number of possible states all seem to rise together. Boltzmann's formula makes that connection precise: entropy equals the logarithm of the number of possible states, multiplied by a fixed constant, the Boltzmann constant. The post calls the resulting equation 'surprisingly simple,' but the underlying proof, built by modeling gas particles with a finite set of velocities and positions and then taking a limit, is 'not easy'; mathematicians, it says, are still working to make Boltzmann's argument fully rigorous.

A worked example grounds the formula: Curie's model of a magnet, where a fixed number of electrons are each spin up or spin down, and the system's energy comes from spins that are misaligned with an external magnetic field. For a small system of 5 atoms at energy E = 1, the post works out that the configuration must be three atoms spin up and two spin down; the number of distinct ways to arrange that split is 5 choose 3, which equals 10. Applying Boltzmann's formula in base 2 gives an entropy of log2 10, or 3.32 bits, for that macrostate. The post plots the same calculation across different energies to produce an entropy curve, though the resulting graph itself is not reproduced in the extracted text.

The post then carries the same counting method over to Dyson's Markov chain. As an illustration rather than the model's actual solved values, it supposes an equilibrium where a given site has a 1/2 chance of being empty and a 1/4 chance each of being active or inactive; spread across 8 sites, that works out to 4 empty, 2 active and 2 inactive sites. Counting the number of ways to arrange that split is a multinomial coefficient: choose which 4 of the 8 sites are empty, then which 2 of the remaining 4 are active, leaving the last 2 inactive, computed as the factorial of the total number of sites divided by the factorials of the count in each state, which here means dividing by 96 (4! times 2! times 2!). The post sets up this calculation and states the divisor but does not carry it through to a final entropy figure for the 8-site case, unlike the fully worked 5-atom example.

The post closes by naming, rather than answering, what comes next: how entropy evolves over time in such a system, plus general statements about entropy tied to the graph's topology and the conditions under which entropy increases, both promised for 'a later post.' It thanks David Pfau for discussion and help while taking responsibility for any mistakes, and cites two references: Clausius's 1865 paper in Annalen der Physik, and chapter 6 of Schrödinger's 1944 What Is Life?

Key facts

  • The post carries Boltzmann's entropy formula, the logarithm of the number of possible microstates times the Boltzmann constant, all the way through on a 5-atom toy magnet (Curie's model) at energy E = 1: three atoms spin up and two spin down can be arranged 5 choose 3 = 10 ways, giving an entropy of log2 10 = 3.32 bits.
  • It then applies the same counting method to Dyson's toy Markov chain model of a cell, whose sites can each be empty, active or inactive and which settles into one of three equilibrium states, two of them labeled 'life' and 'death.'
  • For an illustrative equilibrium, not the model's actual solved values, of 1/2 empty and 1/4 each active and inactive across 8 sites (4 empty, 2 active, 2 inactive), the post sets up the multinomial coefficient and states its divisor, 96, but does not compute a final entropy figure for this case.
  • The framing traces to Clausius's 1865 definition of entropy from the second law of thermodynamics, and to Schrödinger's 1944 What Is Life?, whose concept of negentropy motivated the author's earlier idea that life maintains order by feeding on energy and reducing its own disorder.
  • The post explicitly defers its harder questions, how entropy evolves over time and what the chain's graph topology implies for when entropy increases, to a promised later post, and thanks David Pfau for discussion and help while taking responsibility for any mistakes.

Why it matters

The post's real project is bridging two different-looking definitions of entropy: Clausius's thermodynamic version, tied to work and temperature and used to state the second law of thermodynamics, and Boltzmann's statistical version, which counts a system's possible configurations instead. That distinction matters here because an abstract system like a Markov chain has no temperature to plug into Clausius's formula, so extending the vaguer idea of 'life as entropy,' motivated by Schrödinger's negentropy, into something precise calls for Boltzmann's counting approach instead. The post does not claim a new physical result. It is explanatory: a step by step demonstration, backed by one fully solved small example, of how the counting method Boltzmann built for gases can be carried over to a toy model of a living cell.

Who it affects

This is written for readers already interested in statistical mechanics, information theory, or the origin-of-life literature that runs from Schrödinger's negentropy through toy models like Dyson's. It has no product, policy or market angle, and is not aimed at a general audience. The most direct audience is anyone following the author's own series on 'life as entropy': the post explicitly builds on 'my last post,' about the Dyson model's convergence to equilibrium, and sets up two questions, entropy's evolution over time and its relationship to the chain's graph topology, that the author has promised to answer in a future post.

How to use it

There is nothing to install or buy here: the value is the worked method itself. The reusable recipe the post demonstrates is Boltzmann's: fix a macrostate, count every microstate compatible with it, then take the logarithm of that count, times the Boltzmann constant, as the entropy. A reader can follow the same steps on their own toy model, the way the post does twice: fully, on the 5-atom magnet example, reaching 3.32 bits, and partway, on the 8-site Markov chain example, stopping at the multinomial coefficient's divisor, 96, without a final number. The post is one entry in an ongoing series, so following the full argument means also reading the author's earlier post on the Dyson model's equilibrium and watching for the promised follow-up on how entropy evolves.

How solid is it

The post presents no new experimental or peer-reviewed result. It is an explanatory synthesis built on established physics, Clausius's 1865 paper and Boltzmann's statistical mechanics, applied to a named toy model, Dyson's, that the post treats as already established from the author's own earlier work. The fully worked example, the 5-atom magnet, is carried through to a specific, checkable number, log2 10 = 3.32 bits. The Markov chain example is not carried as far: it uses a hypothetical equilibrium, 1/2 empty, 1/4 active, 1/4 inactive, rather than the model's actual solved probabilities, which the post says exist but does not state, and it stops at the multinomial coefficient's divisor, 96, without a final entropy value. The author is candid that the broader motivating idea, that life can be understood precisely as a form of entropy, is still being worked out: the post itself says that until now, what that idea actually means in detail has been unclear. It credits David Pfau for discussion and help while taking sole responsibility for any mistakes. On Hacker News the post drew 115 points and 9 comments within about 15 hours.

Risks and caveats

The central caveat is that the Markov chain worked example is illustrative, not real: the 1/2, 1/4, 1/4 equilibrium split it assumes stands in for the model's actual equilibrium probabilities, which the post says exist but describes only as 'a complicated function of fixed points,' without giving values. The calculation built on that assumption is also left unfinished, stopping at the setup and the divisor, 96, rather than a final entropy number, unlike the fully solved 5-atom example. The post's own promised payoff, general statements about entropy tied to the Markov chain's graph topology and the conditions under which entropy increases, is deferred to a future post and not delivered here. And the deeper claim that motivates the whole exercise, that 'life' can be given a rigorous, entropy-based definition, is presented throughout as a hypothesis still being developed, not an established result.