No Free Lunches: Evolution and Rationality

The chicken is likely the most dominant dinosaur to have ever lived, and is certainly the dominant dinosaur alive today, with perhaps as many as 35 billion chickens alive today, accounting for about 70% of all birds. To put that in perspective, on any given day there are 14 times as many chickens on earth as there were T-rexes that ever lived (~2.5 billion).

(Yes, all birds are dinosaurs. That’s how clades work. Also, you are a fish.)

The Chickens top speed: 9 mph, and can turn on a dime when on the ground. Respectable. But a far cry from the Peregrine Falcon which, according to Wikipedia, “stoops” at over 200 mph from thousands of feet in the air, and is theorized to be able to reach 388 mph if the conditions are right. And if you google intelligence, you will be bombarded by claims that chickens are impressively intelligent. But still, any respectable listicle won’t even put them in the top 10 most intelligent birds.

However, it’s not in spite of those facts that the chicken is dominant, but rather, because of those facts.

When I was a kid, I thought of evolution as a sort of progression where every “advance” unlocked new abilities and powers. Surely evolution wasn’t just producing different creatures over time, but better creatures. Afterall, humans were smarter than monkeys; progress! 

But this view is false.

There is nothing in the basic structure of the theory of natural selection that would suggest the idea of any kind of cumulative progress.

George C. Williams

Evolution isn’t the same as leveling up. For every creature that increased a stat, there is another creature that decreased their stat. The creatures alive today are not faster, stronger, more intelligent than the creatures that existed 65 million years ago. This isn’t Pokémon, and creatures aren’t leveling up. Rather, they are re-rolling their stats with every generation, and getting selected based on fitness to that particular environment, not on whether the stats are better by some human standard. In other words, optimality is always relative to a context.

This is the principle behind the No Free Lunch Theorem, which states

any two optimization algorithms are equivalent when their performance is averaged across all possible problems.

In this particular case, natural selection is our optimization algorithm. The same algorithms which took an ape and gave use humans also took a theropod and gave us the chicken. 

But perhaps more interestingly, you could also reverse this and say that the algorithm is the organism.

any two [organisms] are equivalent when their performance is averaged across all possible [environments].

For any given environment where the T-Rex is the dominant dinosaur, there is yet another where the chicken is the dominant dinosaur. In our real world case, the strongest environmental impact is human culinary practices, but the principle remains true regardless of what the cause of the environment is. 

The No Free Lunch Theorem is in some sense trivially true. It is not saying anything interesting really. It is simply stating the obvious that if you have a choice between A and B, there are worlds where A is optimal and worlds where B is optimal, and so an algorithm that chooses A is exactly as optimal as an algorithm that chooses B once you average out their results across all possible worlds.

But also, “all possible worlds” is doing a lot of work here. Surely there are other differences between A worlds and B worlds which would allow you to discern which world you are in. Such differences would mean that context sensitive algorithms will perform better. So perhaps “possible” is the wrong word when it comes to a real life evolutionary scenario. My intuition screams at me that across all probable, feasible, and likely worlds, speed, flight, and intelligence are adaptive in the vast majority. Chickens are not representative.

But still. Chickens do dominate.

This applies to rationality, too.

There’s an old rationalist saying; Shut-up and multiply. Don’t know what to do? Shut up and multiply probabilities and utilities. Don’t know what to believe? Shut up and multiply the probability of the hypothesis given the evidence by the prior, and then divide all that by the probability of the hypothesis given the evidence times the prior plus the probability of not the hypothesis given the evidence times the probability of not the hypothesis. 

Simple. Straight forward. I once programmed this (Bayes Formula) into my iPhone as a shortcut. Objectively true epistemology (supposedly) in the palm of my hands. 

But according to the No Free Lunch Theorem, for any given environment where “shut up and multiply” gives you the correct answer, there is another possible world where it does not. 

And if you don’t want to talk about possible worlds, you can still refer to actual scenarios where it is the case that rational models give you the wrong answer. As a concrete example, see ergodicity. I don’t know what percentage of scenarios we enter into are ergodic, but it’s not nil. And for Bayes Theorem, we might talk about its inability to handle epistemic uncertainty (unknown unknowns) all that well, which I would argue is a constant feature of most situations and by definition cannot be fully accounted for.

This is the problem with a simplistic understanding of rationality that focuses on biases from rational models. The greatest strength of such models, the ability to filter our seemingly irrelevant context, is also their greatest weakness. So the first step in any rational endeavor is not figuring out how to apply a rational model to the situation, but rather to ask oneself “which of all the possible worlds am I in? And is it one that calls for one of the rational models, or one that calls for something different?” And these questions require us to be embedded in, and hyper sensitive to, the context that rational models often ignore.

This isn’t a simple defense of heuristics, as heuristics are indeed often wrong precisely because they are overly sensitive to irrelevant context. Rather, it is acknowledgement that context is king. The optimal algorithm for any particular context cannot be universalized to all other contexts. Yes, sometimes you need to bust out all stops and get the biggest, fastest, most intelligent algorithm you have. But sometimes, you can use the algorithmic equivalent to a chicken and dominate.

How varied your judgment and decision tactics should be in the real world is perhaps a post for another time. But it’s worth noting how varied science can be. We don’t restrict ourselves to Bayes to solve every problem in science, and shouldn’t do so in life. To consider all alternative ways of thinking a “bias” does a disservice to the whole project of rationality. Thinking correctly is hard, and there is no free lunch.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s