Map and Territory: Fake Beliefs
This post is part of my notes on Eliezer Yudkowsky’s Rationality: A-Z, known as The Sequences.
Making Beliefs Pay Rent (in Anticipated Experiences)
It’s important to be able to notice in yourself whether or not you have a mental map of something.
Not all beliefs connect to sensory experiences, or anticipation thereof. Eg beliefs about the atoms that constitute a brick.
Empiricism asks what our beliefs predict (and importantly, what they predict must not happen). If a belief offers no such predictions, it is likely “floating”, disconnected from the rest of your belief network.
Every guess of belief should begin by flowing to a specific guess of anticipation, and should continue to pay rent in future anticipations.
A Fable of Science and Politics
Politics’ polarizing, partisan nature influences the lens through which its adherents experience the world. Remaining part of the tribe becomes more important than seeking the truth, even when the former bears great costs.
Belief in Belief
[…] poor hypotheses need to do fast footwork to avoid falsification.
You can hold different beliefs about the same thing, even when these are in opposition to one another - eg one belief may be “floating”, as above. Ideally, empiricism allows us to fix these conflicting beliefs, but this isn’t always possible.
It is often easier to believe that you ought to believe something, rather than to actually believe it (e.g. religion). There is usually some self-delusion there - people will not admit that they believe in belief.
Religion’s Claim to be Non-Disprovable
People used to believe religions, now they believe in religions.
Most people’s concept of rationality is determined by what they think they can get away with
Professing and Cheering
Beliefs can be expounded as a way of professing and cheering for a cause.
To be honest I didn’t think this one was very good - the notion that religious beliefs are irrational, and that this leads to bad outcomes when power is involved, seems to have been covered ad nauseam by plenty of other commentators, and I’m surprised Eliezer spent so much time on it.
Belief as Attire
Beliefs may also be worn as attire, a way of identifying as part of a group and being bound to it.
Pretending to be Wise
Belief is quantitative, and just as it is possible to make overconfident assertions relative to ones anticipations, it is possible to make under confident assertions relative to ones anticipations.
In other words, there are situations where one feigns uncertainty/neutrality to appear maturity/impartiality/superiority, but in fact, their beliefs imply they ought to express confidence.
E.g. a principal saying “it doesn’t matter who started the fight, it only matters who ends it” (it clearly matters who started the fight), or similarly a geopolitical Great Power insisting two groups stop fighting immediately (regardless of the aggressor or any sense of justice and fairness). Eliezer calls this “pretending to be wise”, but really it’s just epistemically lazy.
Neutrality is in fact a judgment, even when it is trying to avoid being one. It is saying that, having weighed the evidence, each argument is equally as likely to be true.
Rationalists are not above politics.
Applause Lights
Some things are to allow the speaker to be lauded, more so than to contribute to meaningful discussion - the equivalent of “Applause” lights in a TV studio. E.g. “We need to balance the risks and opportunities of AI.” - this is obviously true, but has no substance.