If You're Happy and You Know It, You're Probably Wrong

I've posted before on the very common problem of people whose estimation of their own abilities doesn't match reality such as the life of the party who is anything but. I cited Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments by Justin Kruger and David Dunning of the Cornell University Department of Psychology.

Now, via Crooked Timber I've found another interesting paper on the topic, On overrating oneself... and knowing it by Adam Elga.

Elga begins with an anecdote about his friend, Daria, who believed in astrology. Elga attempted to convince her that astrology is a load of bunk, presenting her with a lot of evidence such as studies showing that the position of the stars at one's birth had bugger all to do with one's personality or prospects. The result was:

Daria agreed that the studies were significant evidence against the truth of astrology, and that she had no countervailing evidence of comparable strength. But that was not the end of the matter. "I still believe in astrology just as much as I did before seeing the studies", she said. "Believing in astrology makes me happy."

Elga moves on to discuss a number of studies which show on a body of psychological research which show that:

... people have inflated views of their own abilities and prospects. People (nondepressed people, at least) rate themselves as better friendlier, more likely to have gifted children, more in control of their lives, more likely to quickly recover from illness, less likely to get ill in the first place, better leaders, and better drivers than they really are. And that's just the beginning. There is a great deal of work documenting the persistent and widespread positive illusions (about themselves) to which people are subject.

Some interesting questions arise when we combine this finding with the claim that research, such as this recent US Gallup Poll, shows that in general, conservatives are happier than leftists (or liberals or progressives or whatever the hell you want to call them). The Gallup Poll analysis has this to say on "The Politics of Contentment":

Even when accounting for partisan differences in marital status and household income, Republicans are significantly more likely than Democrats and independents to be very happy.

Happiness by Political Party ID
	Republicans (%)	Independents (%)	Democrats (%)
Very happy	62	49	50
Fairly happy	35	47	41
Not happy	3	4	7

Why Republicans are happier is not clear, but the result has been the same in nearly every asking of this measure since 1996, including one reading under former President Bill Clinton, a Democrat, and three under Republican President George W. Bush. Only in 1996 did Republicans and Democrats express about equal levels of happiness.

Percentage Very Happy, by Party ID
	Republican (%)	Independent (%)	Democrat (%)
2003 Dec 11-14	62	49	50
2002 Dec 5-8	59	42	46
2001 Nov 8-11	42	33	37
2000 Oct 6-9	53	44	45
1996 Mar 8-10	52	46	50

[My emphasis]

This looks pretty impressive, especially the finding that 62% of Republicans surveyed are very happy compared to only 50% of Democrats. I'm assuming, of course, that when the report asserts that "Republicans are significantly more likely than Democrats and Independents to be very happy" (my emphasis again) they've at least run the data through Microsoft Excel's Chi-Squared test or something similar, rather than just eyeballing the percentages and saying "Wow! There's a good item for the press release."

In any case the statement "Republicans are significantly" (or otherwise, depending on whether that Chi-Squared test was run and what the result was) "more likely than Democrats and Independents to be very happy" is pretty sloppy; what it comes down to in the real world of verifiable results is that if you were to ask George W Bush if he was very happy, there's a 62% chance that he would answer yes, whereas if you asked Howard Dean the odds are only 50/50.

But if, in the US at present, you're more likely to find a very happy Republican than a very happy Democrat, Elga's paper points to an unhappy consequence; Republicans are significantly more likely to be self-deluding than either Democrats or Independents. Or, to translate this conclusion into the more rigorous form applied in the previous paragraph, if you were to ask either George W Bush or Howard Dean if they are self-deluding the probability that they would answer yes is zero, with a 62% chance that Bush's answer is incorrect while with Dean the odds are again 50/50. If Dean believes that he is not self-deluding he is significantly more likely to be correct (politically or ordinarily) in this believe than the Shrubster (the word significantly is employed here in the same sense - whatever that is - that it was employed in the Gallup organisation's report of its findings on "The Politics of Contentment").

If Republicans are more likely than others to be self-deluding, it's reasonable to suppose that they are also more likely to hang on to beliefs that make them happy, even if the weight of evidence is against their beliefs. Any suggestions about what these beliefs might be would be purely speculative so I don't propose to venture too far into that territory. While this line of argument might be appealing on a number of issues - such as the vexed question of whether the US would be better governed today if the Republicans had really run two short planks in the 2000 Presidential Election - it inevitably leads to unfounded accusations of arrogance and intellectual snobbery. Anyone else who is prepared to risk it is quite welcome to take up this line of argument as far as I'm concerned. There are much more interesting issues to explore.

At the very least it's possible that many of the Republicans who reported themselves to be Very Happy were in fact, Fairly Happy Republicans who had over-estimated their happiness levels. This is also true for the other political affiliations. Similarly, the Fairly Happy category for all three political affiliations probably includes a few self-deluding Not Happy people; the most trustworthy figures are those for the Not Happy category. It's unlikely that respondents who placed themselves in this category were affected to any significant degree by the tendency which Elga notes, for non-depressed people to overestimate their abilities and prospects. They may have been affected by the tendency of some depressed people to underestimate how well off they are but, if you're depressed enough to make that error, your opinion that you are not happy is probably well-founded.

Although the differences between Democrats and Republicans here look impressive, with over twice as many Not Happy Democrats as Republicans, I'm disinclined to regard them as significant. Assuming that the Chi-Squared test I referred to earlier was run, all that it would indicate on a contingency table with 9 categories is that the distribution of results between all 9 categories was too great to be attributed to random chance alone. And even to draw that conclusion you have to specify exactly how much of the variation you're prepared to write off as the result of random chance. If you're genuinely interested in the question "Are Republicans more likely to be happy than Democrats?", you need to conduct an entirely different kind of study, which supports the possibility of testing for significant differences between these two groups alone.

It's time to put aside these technical quibbles and move on to those interesting issues. For ease of later argument, I'm going to provisionally accept the following propositions as more or less true:

1: Consistent with the Gallup survey finding, Republicans are more likely than Democrats to be very happy.

2: As Elga notes, happy people are more likely to delude themselves about their abilities and prospects than less happy people.

3: Happy Republicans are more likely to delude themselves about their abilities and prospects than unhappy Democrats (this more or less follows from 1 and 2).

4: Happy Republicans are more likely than unhappy Democrats to hold beliefs in the face of strong evidence that they are false because those beliefs make them happy, just as Elga's friend Daria preferred to believe in astrology because it made her happy, despite strong evidence that it was a load of cobblers (this is either speculative or a truism).

5: The US Gallup survey finding reflect a fundamental difference in happiness levels between Lefties and Conservatives generally and are therefore applicable to other countries if you just change the names of the political parties (this is very speculative).

You might wonder what I mean by "more or less true" and "more or less follows". It's not very complicated; "more or less true" means (more or less) that the statement is, in my view, supported by enough evidence or argument to be taken as true. "More or less follows" in the case of 3 means that if you're prepared to be slack enough to accept the truth of 1 and 2 they more or less logically entail 3 and to hell with Baye's Theorem. So for the rest of this post we're just going to take it as read that propositions 1 to 4 have been established and get on with it. I can't say fairer than that now, can I? If you have problems with any conclusions I might draw later, at least you know where to start looking for the weaknesses in the argument.

[Incidentally, proposition 5 was added in after I had started on the next section of the post, beginning "Here are some big questions you might like to ponder a little". It's an assumption that is frequently made by other bloggers and the paid opinionistas whom they emulate. I doubt, therefore, that anyone will object to its being made. Usually it's an implicit assumption but I think it's much better if we drag it out into the open just this once. This paragraph was, of course, written after proposition 5 was added.

Later: If you're still not entirely happy with proposition 5, you're in good company. Neither am I, but it's not really my assumption. I've just borrowed it for the time being.]

Here are some big questions you might like to ponder a little. First of all, on the personal level, is it better to be a happy, self-deluding conservative or an unhappy, smart-arsed lefty? Would society as a whole be better off if everyone was a happy self-deluding conservative rather than an unhappy smart-arsed lefty? Could the human race survive a the geopolitical climate that would result, if every country implemented policies of reverse social-engineering to turn their unhappy smart-arsed lefty citizens into happy self-deluding conservatives? Is Trotsky, G. starting another bloody series that won't get past the first post or is there more of this crap to come?