In Praise of Patterns

Caveat, this is a rough draft, but it’s been a rough draft for weeks. I just don’t have the time to flesh out everything right now, but I thought I’d submit it before my previous post was forgotten. The general points are pretty obvious, I encourage readers to offer more.

A few weeks ago I asked readers if they knew the definitions of induction and deduction. I really wasn’t interested in how many people knew the definitions, or how many people didn’t, or how many responded. I was curious actually about the ratio between those who knew only one definition. Here is what I found:

4 people knew only the definition for induction.
13 people knew only the definition for deduction.

I was expecting this. In part, one might think that Sherlock Holmes skewed this, but I did offer both definitions, and requested that readers answer with those definitions in mind.

My working hypothesis is that we live in an age where induction, generalizing from data sets, is problematic and suspect. On the other hand, deduction is not as verboten.

When people say:
You can’t generalize!
That’s just statistics!

They are attacking inductive ways of teasing out information from sets of data, even if they don’t know the term induction (much of statistics is formalizing induction). On the other hand, I don’t believe that deductive models are in such disrepute a priori, though they might disagree with the specific model, the intellectual public does not believe that the method is discredited as a whole (I used the term intellectual public, because the “common man” has less of a problem with induction-though that is probably a lagging indicator in my opinion).

So why did induction go into such disrepute? My reasoning is a bit confused because I don’t think there is a deductive model that is tightly constricted that can elaborate a linear causative chain, rather, there are a variety of factors, some of them logically conflicting, that have likely led to the decline of induction.

I-The prestige of natural science:

This might seem a very peculiar factor. But I think there is something to it. Human beings seem to have certain mental modules. We have a “natural” idea of numbers, of physics, etc. It is our “common sense,” which we all share (barring brain damange or a Post-Modern education).

What do we do when “common sense” conflicts with science? Today, we often look beyond our intuition. Some of the “common sense” reasons trotted out against rocket propulsion for instance seem ludicrous in hindsight.

Example: Space flight can not work because there is nothing for the exhaust to “push” against.

That might seem a funny example, and in a pre-Newtonian age, it might make sense, but the above quote was, I believe, from the early 20th century! Our “mental modules” were refined in the context of rather low velocities, and of course, drag and friction can be confusing when trying to simplify physical models. 300 years after Newton, his physics makes sense, but some of it does seem to defy “common sense” when we first encounter it as children. This is because our “natural physics” starts out with certain variables (eg; drag) and parameters (eg; low velocity) that when stripped away, reveal a set of general laws which might at first blush seems to be new and surprising.

I could go on in this vein, but Newtonian physics is very convetional when set against General Relativity. Everyone knows most of the “paradoxes” that are involved in high velocites or gravities, so I won’t repeat them here, but it is important to note that Relativity is still a classical theory that is accessible to common sense when you make a few assumptions and strip away your conventional variables and parameters.

The big problem I think comes when you hit Quantum Theory.

A week ago I was having dinner with a friend when I presented this hypothesis. His response was: “I know Quantum Theory and it kind of makes sense, I don’t see how it would make people reject common sense or induction.” But I have left a piece of information out: my friend does work in photonics at M.I.T..

If you type you get a lot of stuff. It seems to come in three main classes:
1) Conventional texts for the scientist.
2) Popular works attempting to “demystify” the field for the lay person.
3) Popular works attempting to further “mystify” topics by appealing to the weirdness of the Quantum World.

Schrödinger’s Cat has smeared its way through philosophy and now has found a home in spirituality and pop culture.

There are only a few things that people who haven’t taken some Quantum Physics courses will know about it:

1) It’s weird & defies “common sense.”
2) It describes the world’s “basic building blocks.”
3) It’s had practical applications, so you can’t dismiss it as pie-in-the-sky weirdness (superstring theory anyone?).

But, it is strange that I am saying that this undermines induction, because Quantum Theory is statistical and destroyed the deterministic universe of classical physics [on micro-scale] (the “God does not play dice” quote by Einstein is a jab at Quantum Theory). Quantum Theory “works,” and the results popped out by the equations predict the data, but we may always be ignorant of the irreducible Ground of Being or whatever you want to call it.

To sum up this point, the counter-intuitive results of natural science can sometimes be used against those who wish to generalize from data sets a *rough & ready model* that does not dot all the i’s. This ignores the fact that science itself is inductive, insofar as observations of data and hypothesis generation that lead to deductive models are crucial pieces of the puzzle.

II-The rise of “Theory”:

First, I would like to refer readers to or . Both cover the rise of “Theory” very well.

Since the 1960s there has been a rise in the Academy of theories like Post-Structuralism, Post-Colonialism, Feminist Theory, etc. etc. They are part of the broad family that detractors often term Post-Modernist. Their basic mode is deductive. They make a few core assumptions, for instance:
The male-female power struggle is central
The Western-non-Western power struggle is central
The Linear-non-Linear power struggle is central

etc. etc. etc.

You get the basic point. In many ways it is reminiscent of Marxism, all the data can fit into a central paradigm that reappears in all manifestations of human culture, literature, etc. etc.

Unlike the Marxists, from where I stand, the Post-Modernists tend not to make a show of adding corrective factors when facts do not fit theory. Where Marxists might attempt to explain why 19th century theories that were not predictive (Communism did not succeed in advanced capitalist nations) by adding new layers of theory, the Post-Modernists often hew strictly to the paradigm and re-shape “facts” to theory.

This where it gets very confusing, because most Post-Modernists love “quotations” because they emphasize “subjectivity” to the point where “solipsism” is a central virtue. My truth is my, your truth is your truth, and so forth. In practice of course, most Post-Modernists seem to behave so that their truth is their Truth and and their truth is your Truth.

A method that began as a correction on skewing of facts due to personal interpretation has compounded itself to the point that it is cannibalizing the object of study itself, “facts,” in pursuit of stream-lining the method. It is as if the function f(x) = y is far more fascinating than the x or the y!

This resembles science and engineering in a fashion in that generalized techniques are crucial, and the functions, models, are the true holy grails, with facts and discoveries being supporting players (discoveries in fact being the models!). But, the important part of science is falsification, and the extra addendum, that subjectivity is paramount, means that Post-Modernists tend to neglect this part of the equation (it’s not “false,” rather, you are using a certain mental mode that makes it seem false!).

Subjectivity, the rise of Truths, and implied facts from the propositions that issue out of these truths, has resulted in the overturning of “common sense,” insofar as the latter has a rough congruency with reality, and can be corrected or refined by reality (science). If you assume for instance that ideas, and social constructions, are the central organizing principles of the universe, then the following statement becomes very interesting:

Men are physically stronger than women.

The above is a “common sense” assertion, but when you strip away the precedence of facts, and look at the way the facts are stitched together, you see something different from what is visible at face value. For instance:

1) This individual believes that “Men” and “Women” are “objective” categories.
2) This individual thinks that one can make an assertion of “physical” “strength”.
3) This individual believes that “Men” and “Women” are separate from the self-perception of the individuals in question, rather, they can make the assertion and set the terms of debate.
4) What does the word “stronger” mean? After all, “women” tend to give birth, while men do not, who is “stronger?”

(by the way, the most common refutation of this assertion that I encountered in college was a girl saying, “I know men weaker than me!” You can fill in the rest of the conversation)

Blah, blah. You get the point. There are a few tendencies that I am illustrating above (oops! Imposing my world-view again!).

1) Ignore what the person says, analyse how they say it.
2) Analyze the interrelationships between words, and tweak meanings and see how the function pops out something different (ignore the fact that the person making the assertion has specific meanings in mind).
3) Undermine the generalization by fiddling with definitions and finding exceptions outside of the set defined by the definition that you concocted.
4) Generalize-on-the-sly (using words like “tend”) when it supports you insofar as it is aligned with an obvious bedrock Truth (patriarchy is bad, whites are oppressive, etc.).

Why can people get away with this?

1) Science is confusing and difficult to comprehend, but it works! So, the fact that “Theory” is confusing and difficult to comprehend does not mean it is invalid.
2) It is easy tear down (“deconstruct”), while to build is really hard.
3) It gives Total Truths. It is a religion and science analog (religion giving moral Total Truths while there is a perception that science gives Total Truths, when in fact it is far more provisional and inductive in practice than the end result that the public will see).
4) The Total Truths can be made to dove-tail well with politics.


III-Politics is easier without induction and generalization

Prong #2 is mostly an Ivory-Tower phenom. You encounter it in college all the time. For instance, here is a real-life example of the kind of thinking that I believe is caused by prong #2.

1) Friend asserts that there are few Asians at his college.
2) I assert that that might be a function of his liberal arts major, as Asians tend to focus on business or science.
3) Someone responds, “My roommate is Asian and is an English major. So your assertion is false.”
4) Didn’t they understand tend, or trend, or pattern?

In political/ideological debates, a deductive model of thinking is very useful, because you are sure about your position once you flesh out the truth implications (I know, I used to be a pretty rabid libertarian! Facts be damned, I know the Truth, that’s all that matters!). If that deductive model is under your control, so you can assert the axioms, all the better! The subjectivity of PoMo thought has recently been brought to the service of Left thinking. After all, if ideas, and deductive models from On-High are what matter, social engineering is a lot easier.


Trends, like:
Women focus on the home, men on the workplace.
Gay men are a small minority.
Some level of xenophobia is universal.
War is pretty universal.
etc. etc.

Can be discarded as “social constructions,” illusions caused by subjectivity. But what do you replace it with? After all, that leads to nihilism…well, they have a deductive model for you. Of course, there are axiomatic issues with this (subjectivity, truth claims, how do they go together? It’s like the logical positivist “verification principle,” you can’t verify the verification principle!). But politics, like inductive generalization, is messy enough that it can accept the contradictions.

The trend (oh, that word again!) is moving past Leftism and into the Right and general society:
Gays are socially constructed (yay! let’s change them!).
Social pathologies like pornography are the result of improper inputs.
Male promiscuity is the result of our immoral culture.
Evolutionary theory is a social construct of secular humanists!

On the level of the street, this sort of thinking, which rejects generalizations that do not make deterministic predictions, manifests itself in issues like racial profiling. People are terrified of being “incorrect,” and generalizations are out. In private they might give in to their vice of making predictions based on facts they see around them, from their perspective, but in public, they will assent to all sorts of ludicrous assertions so as not to seem Wrong-Thinking.

Also, let me bring up a point about statistics. The phrase, “It’s just statistics!” I think comes from the fact that most of the time we see statistics, it is a government or advocacy organization brandishing it to “prove” something. They tend to be shorn away from context, and framed through selection bias, omission of important information like standard deviation, mean vs. median, mode, etc. or proper description of how the events are recorded (for instance, “50% of marriages end in divorce” sounds a lot different from “the majority of people who get married do not divorce, but a minority that do marry do so multiple times , so the total number of marriages ending in divorce is around 50%”).

The importance of the utility of method:

Method is important. But, it is important in the context of what it gives you. Is it a proper model of reality? Does it allow you build a bridge that won’t collapse? Will it allow you catch airplane hijackers?

We are swarmed by data, we meet more people in a week than our ancient ancestors might have met in a lifetime, our mental modules are just not up to the task of dealing with the modern world without some effort and extension of its capacities through first aproximations. We need helpers, science, statistics, etc., ways of simplifying the bewildering complexity that is smashed against us every day.

That very complexity means that we might never be able to dot all the i’s. We might have to deal with first aproximations, provisional models, and some fuzzy and sloppy conclusions. They aren’t a reflection of the problems with technique, as much as with the complexity of the world around us (you know the saying, physics is so cool & easy because it is simple while sociology is difficult because it is so
complex. Yeah, you read that right!).

There are some things that our brains are good at. Thinking is one. Computation isn’t. A computer can take in some truths and split out results like it’s magic. On the other hand, human brains are excellent pattern detectors, with analysis capacity to sift the patterns if we so choose. We will make mistakes. But not availing ourselves of the one mental module that is still pretty state-of-the-art is pretty stupid.

Of course, that’s just an opinion!

Posted by razib at 03:13 PM

Posted in Uncategorized