Reading Date: Dec 23, 2022 ~ Feb 8, 2023
Order out of Chaos
When dealing with complicate physical systems, we will make assumptions and consider ideal situations first, and then start adding flunctuations or secondary elements to our calculation to correct our results.
No fundamental physical law has the direction of time, except the increasing of entropy. Gas particles described by Newton’s Law has no direction, which means they work equally well when the entire system is run backward in time. But the statistical result gives the arrow of time, where it reflects on the increasing of entropy.
Poincare has showed that if we have a box of gas containing a definite number of particles, and they obey Newton’s laws, then after a sufficiently long interval of time, the box must return to its original state. Since there is finite number of state, and we will definitely come to some time when it is repeated. This is known as Poincare cycle time. This recurrent cyclic behavior is a direct result of the strict application of Newton’s laws, in which past and future are the same.
The Return of Chaos
Poincare had transformed the 3-body problem from mechanics and dynamics into a problem in topology and geometry to show the stability of our Solar System. From Poincare section, where the cut plane that the trajectory investigated inside this phase space must pass, if a trajectory starts at one point on the section and returns to that point, then no matter how complicated the behavior is, it must be periodic.
He also found that if the trajectory cuts the Poincare section even a tiny distance away from that starting point, the system can follow completely different pattern of behavior, with trajectory taking different routes. The key thing here is, systems that start out from nearly the same state can very rapidly evolve in entirely different ways. He discovered that many systems are sensitive to initial conditions, and that they move away from those starting points in nonlinear fashion. This limits our ability to predict the behavior of such system as we can never precisely measure anything. The sensitive nature to initial condition and we cannot precisely determine or measure it are the hear of chaos. Simple laws, nonlinearity, and sensitivity to initial conditions and feedback is where chaos and complexity emerge.
Chaos out of Order
Self-Similar Process
The definition of chaos we are going to discuss here is a completely ordered and deterministic, with one step following from another in an unbroken chain of cause and effect that is completely redictable. With simple rules and iterative process by which the system feeds back on itself, it has self-similar or self-referential structure. From this, it maps to chaos. Some examples like logistic map in biology, period-double bifurcation, and vortices that breaks into sub-vortices endlessly.
This brings out the the topic, fractal. It is created through simple rules and iterative process, and it has self-similar structure. Some examples: Cantor set and Sierpinski gasket. Sierpinski gasket is the attractor of a random process. Fractal, such seemingly complicated systems, can be produced or described by the repeated application of a very simple rule.
Fractal, Dimensions, and Power Law
Fractal is form by determinstic iterating steps. It is mapping from one point to another, and this brings us to topology. For example, logistic equation maps points on line is stretching and folding the line repeatedly. Eventually, we get something like a Cantor set (like a Horseshoe, so called Horseshoe attractor), and we will see the points jumping around randomly as we move on the line. This is the topology associated with the on set of chaos produced by period doubling. The line is a finite volume, but repetitive iterative process are infinite. So we can imagine that there are infinite number of layers of phase space contain within a finite volume of phase space. If we do the same kind of stretching and folding to attractors that don’t start from a straight line, for example, an attractor wrapped around a torus, we can end up getting a even more complicated fractal chaos.
How do we measure the dimensions of a fractal? If we divide a straight line by 3, and scale one of those sub-line by 3, which is 3 to the power of 1, we get a line identical to the first line. We say dimension of a straight line is 1. If we take a cube, and divide each of its directions by 3, and then pick a sub-cube, we need to scale it 27, which is 3 to the power of 3 to get the original one. We say the dimension of a cube is 3. For a fractal like Koch curve, we divide the length by 4 and then scale by a factor of 3, so the dimension is $log_3{4}$, roughly 1.3. This kind of relationship is called power law. Some scientist investigated metabolic rates of animals of different sizes. The exponent is not 3, but 2.25, even though we have sizes and take up space. We are more like something between volume and surface, specially extremely crumpled-up fractal surface within finite volumes.
From Chaos to Complexity
If a system is close to equilibrium, they generally reponds in linear fashion to changes in their environment. The dissipative system in the linear regime settles down not into the state of maximum entropy (which would be equilibrium), but to the state where enttropy is being produced as slowly as possible. Things exist is in a steady state in the linear regime, like human body maintains his or her integrity for many years using the flow of energy (or food).
To see how pattern and complexity comes from chaos, we can observe systems that are far from equilirium. For example, Benard convection, how equally heated pan forms convection pattern out of chaos.
Gravitational Field and Its Negative Energy
The gravitational fields make the Sun able to produce heat and is able to maintain us far from equilibrium, on the edge of chaos. Gravitational field has negative energy. Since energy stored in gravitational field that links two infinitely far apart objects are zero, once they start moving together under the influence of gravity, they gain energy from this zero gravitational field. It is a fundamental truth about the way the Universe works that gravitational fields have negative energy, and that for matter concentrated at a point, this negative energy exactly cancels out the mass-energy of the matter. This later became the cornerstone of the idea of how the entire Universe could have appeared out of nothing. The total energy of the Universe starts out at zero, and the tiny bubble was blown up into the Universe today. The state of maximum entropy for a box of gas is with the gas spread uniformly throughout the box at uniform temperature. But when the gravitational forces come in, they pull things together to make large clouds, making more order and at the same time reduceing the entropy. It is swallowing entropy, to go with its negative energy. Explaining why the Universe is not in thermodynamic equilibrium today.
Turing Process
Turing’s thoughts on the pattern-forming process is like two components having a series of self-similiar process with feedbacks. For example, chemical A and B form forms feedbacks, and the patterns could arise in a mixture of chemicals if A not only encouraged the production of more A, but also encouraged the formation of another compound B, which was an inhibitor of A that acted to slow down A. Once A and B diffuse through the mixture of chemicals at different rates, so that there would be more A than B in some parts and more B in A at other places. The competition between A and B was the key to pattern formation, and that diffusion rate of B is larger than A, so that positive feedback proccess of A is always a local phenomenon and A be inhibitd by B is a widespread phenomenon.
Turning pattern on mammals’ skin are form through Turing process as shown by Murray. The process can be described through wave traveling, if the surface is very small at the time the Turing process is triggered (embryo), no pattern can form at all, there is no room for interference pattern to show. At the other hand, if the room is too large, the interactions become too complicated to allow any overall patterns to emerge. The chemical diffusion processes produce stripes on smaller areas, and spots on larger areas.
Some other examples that involve Turing process are Belousov’s discovery of a changing color liquid (citric acid cycle). The cycle forms a pattern through a series of chemical reaction. If we add needed components and remove the trash, they form a never ending cycle. This of course obeys the law of entropy because we add and remove items. The orginal formulation of the second law is not the ultimate truth, and has to be reconsidered in nonequilibrium situations and where gravity is involved. The patterns describe above also have the period-doubling feature, as we keep increasing the flow of reactants. The periodic pattern becomes less obvious, and the system tips over the edge into chaos. Lotka’s mathematical equation describes these kind of hypothetical oscillating chemical system.
The above processes described using certain chemicals can also be affected by outer conditions (ex: temperature) when it hits a critical point, not just Turing process. Sometimes, small changes in the environment, or small mutations, can have big effects on the body that develops.
Earthquakes, Extinctions, and Emergence
Complexity is made up of simple stuff, and these simple pieces have to be connected together and interact with one another in the right way.
Earthquakes and Power Law
Earthquakes’ frequency in log scale - magnitude (log scale energy) plot is a line, which means for every 1000 earthquakes of magnitude 5, there are roughly 100 earthquakes of magnitude 6. This is power law, since if we scale up the magnitude, the corresponding frequecy also scales. There is also power law in fractal. They are both scale-free. Means the occurrence of earthquakes is also scale invariant, there is no difference betweeen large and small earthquakes. The number of the population of cities also obeys power law.
Power laws always mean that the thing being described by the law is scale invariant, so that earthquackes of any size are governed by exactly the same rules. Like frozen potatos smashing against the wall and break down to pieces in different sizes has no difference from two planetoids collided and break into asteriods of different sizes, this is just different versions (frozen potatos and planetoids) of the same kind of variation (different size pieces). Another way of describing all these kind of variations is the possiblity of occurance. Large events are more rare, and can be described by the frequency of happening is $\frac{1}{size^k}$. Or we can say that the size of an event is proportional to 1 over some power of its frequency. Generally, this is called 1/f noise (thinks of noise as variation. 1/f noise contains information.
Example of 1/f noise: the jagged line of light curve measure from a quasar, weather system also contains 1/f noise, traffic jams, stock market’s fluctuations follow a power law, so is extinction, etc. These implies large events (severe weather, traffic jams, stock market crashes) can happen out of the blue, as a consequence of small triggers.
This power law were observed in complex systems, which built on top of simple models. Per Bak and his colleges built a sand model that drops new grains of sand one at a time, to simulate systems that are in a kind of steady state, only being disturbed slightly as they feed off a steady flow of energy. The avalanches obeys the power law. Any grain can stay in the pile for any length of time, and no grains stay in the pile all the time. The whole system has an important influence on every component part, and every component part has an important influence on the whole system.
Sand Pile Module
How Kauffman imagines about complex systems brings out both complexity and networks in emerging complex systems from simplicity. We start off by numerous individual non-connecting buttons, and then pick two buttons randomly and link them together one at a time. The largest number of connections a button is connected to tells how complex the system is. The largest cluster grows slowly at first, more or less in a linear fashion. But when the number of threads approaches and then exceeds half the number of buttons, the size of the largest cluster increases extremely rapidly, as each new connection is added, because there is more chance to link to an existing small cluster. After this, the growth rate turns off, as adding more threads usually just increases the connection between buttons that are already connected. This is phase transition, which switch from boring simple stable state to a complex state just by a few connections added. The connection and button can be anything. Through his model, how life emerges from primordial chemical broth becomes inevitable as connections between chemicals increase.
Human and DNA
How DNA works together to form a human body can be pictured in the same way. Human body is composed of cells. The way cells work are that genes act to control the machinery of the cell, and that genes can affect one another, with one gene turning another one on or off. All of those genes are present in the DNA of every cell in the human body, but they are not all active in every cell in the body. Only the appropriate bits of DNA get switched on during the normal course of life. Kauffman uses an analogy in which the genes are lightbulbs wired together at random. They investigated the behavior of such large networks to see if there are “stable patterns” resulting from simple rules. The state cycles act as attractors for the system, and in some cases, they might be powerful attractors that whatever state you start the system in, it very quickly moves toward one of these state cycles. And this may be the key to cellular life.
The only systems that behave in a way that is both complicated enough to be interesting and stable enough to be understood are those in which each node is connected to exactly two other nodes. In these kind of system, each state cycle has a length equal to the square root of the number of nodes, out of $2^{number \ of \ nodes}$ possibilities. These state cycles are powerful attractors, without being sensitive to small disturbances. If it start it in randomly chosen pattern, it will settle down into a stable cycle, repeatedly visiting just those states in the same regular order. With $10^5$ different nodes, there will be $317$ different attractors (state cycle length 317), and with $30000$ nodes, there will be $173$ different attractors. There are between $30000$ ~ $10^5$ genes in the human genome, and there are $256$ different kinds of cell in the human body. After comparing the number of genes and the number of cell types in different organisms, it shows that the number of cell types does increase as the square root of the number of genes!
The Facts of Life
Darwinian Evolution and Landscape Model
Darwinian evolution theory applies is adequate for describing those that involve only a few species interacting with one another and their physical environment. It deals with stable situations. But we need something more to get the full picture of how evolution works, especially when there are many different species interacting in an ecosystem.
Ecosystem is constantly changing, and is kept in balance only because all of the components are evolving as fast as they can in order to keep up with the others. This is usually known as Red Queen Effect. If what happens to one species affects every other species, then it is impossible for even a pseudo-stable Red Queen situation to develop. This is like lightbulbs linking to some of their neighbors in previous section. These systems were referred to as the web of life. If the system starts with having connections to neighboring nodes, it will spread beneficial changes, wash away defects through natural evolution, and opening up more network. This pushes the system toward the edge of chaos. On the total opposite, from the chaos, where every group manages to insulate itself from the chaos to some extent by reducing the number of connections, so that they have chances to go through natural selection and evolve. This moves the system from total chaos to the edge of the chaos.
Both Darwinian evolution and this constantly changing coevolving system can be described through fitness landscape model. The peak and valley in the landscape represent good genes and bad genes. The peak acts as attractor. In Darwinian evolution, this landscape is stable, and it does not change, cause the species is only interacting with a few other species. But there is a problem, what if the peak is only local maximum, and there is a valley between maximum and this local maximum? The Red Queen effect (co-evolving system) will change the landscape where the species has interaction with. The flock of individuals constantly evolving to the nearest peak on the constantly changing landscape. It is not hard to imagine that if the landscape changes rapidly, nothing interesting will happen. On the edge of chaos, the landscape is always changing, but usually slowly, and this keeps opening up new evolutionary possibilities for individual species within the ecosystem.
The Web of Life and Extinction
Another important thing for pushing the web of life towards the phase transition on the edge of chaos is to consider mutation or evolution at extremal situation (extremal dynamics). Extremal situations here will be the least-fitted and best-fitted species. If they mutate the best-fitted species by giving new random fitness numbers (a number that represents the height of the landscape) to the best-fitted species and its other two connected partners, then every species in the system will occupy a high peak in the fitness landscape, which gives a stable ecological network and is ought to last for a long time. If we do it the other way around by mutating the least-fitted species and its neighbors, the overall fitness numbers drop. This model produces a situation in which intervals of stability and mass extinctions are separated, even though the same rules apply to both of them. Which is bad, and it doesn’t match the power law of actual extinctions.
If we also put predator and prey model, which is to add layers in the food chain in this system. We can see that predators who lose more of their food supply do less well than those who lose less. An extreme case could be a predator losing all of its prey, so that the predator itself goes extinct, affecting predators in the next layer up the food chain. This model organizes itself into a stage on the edge of chaos, with a critical state in which the disappearance of just one prey species at the bottom of the food chain can sometimes trigger a wave of extinction rippling the the upper layer. The scale of this extinction wave is scale invariant and obeys power law. This model also gives the same power in the real extinction of life on Earth.
Mark Newman did another simulation consider only outside effect by remove all the species having fitness number lower than a randomly chosen number and then fill in new species with a new random fitness number. He also consider the fitness landscape has changed by changing their fitness number. The result is that the system has the same pattern of extinctions obeys a power law similar to the real world in fossil record and in systems consider only evolutionary changes in the network alone. This explains why the actual pattern we see in the record of fossil remains distributed through time is so simple. All the effects produce the same pattern. So it is hard to say whether the extinction of dinoasours is caused by a meteorite or not.
Life Beyond
When joining an exploring life on Mars project, Jim Lovelock suggested that they should look for general attributes of life, not specific kinds of life, because life on Mars can be in completely different form on Earth. This is to look for entropy decreasing as living things consume external source of energy to feed on. The best way to look for entropy reduction is to measure the chemical composition of Mars’ atmosphere. If there was no life on Mars, the gases would be in a state of thermodynamics and chemical equilibrium, dominated by stable compounds like carbon dioxide. From this, he came to the idea that for Earth to stay in this seemingly stable atmosphere, something must be regulating it. The living things and non-living things form this complex network of interaction, even though each species is only trying to maximize its own benefit and chance of survival. Some examples like Daisyworld model showing how can Earth’s temperature stays in constant while Sun increasing its brightness from 40% to 135% comparing to today.
Another interesting example like how sulfur, an essential ingredient of life but is easily wash off from rivers, transmitted from the oceans to the land and why cloud can form above the ocean (there is no dust or seeds like the land) so that the sea water does not heat up. This is because marine organisms release sulfur (in the form of dimethylsulfide DMS) to keep the pressure inside the cell. It is acting as cloud condensation nuclei that help forming clouds. The clouds then makes the wind more vigorous and stir up the surface of the sea. The clouds and rain brings down dust from the continents, which is carried high in the atmosphere. This dust is rich in nutrient minerals that are essential for life, but itself do not have the right physical properties to act as cloud condensation nuclei.
The DMS also makes a prediction about ice ages on Earth. The amount of carbon dioxide in the air affects the temperature fluctuations. Then what makes the fluctuation of the amount of carbon dioxide? It is the biological activity of the oceans, that draw carbon dioxide out of the air and lock it up in the form of carbon compounds. The scientist also find that iron is one of the components that keeps plankton growing. So through the regulation between iron, DMS, and other mixing effects, the Earth can wobbles between ice age and interglacial period.