Utility versus Preference

My rejection of utilitarianism is as follows: Nobody actually follows it.

What do I mean by this?  There are all these people out there who claim to subscribe to utilitarianism, Scott Alexander included!  Yes, this is more fall-out of that post.  In this case, the result of more research on the prospective idea that I was straw-manning his arguments.  As it transpires, I may have been.  This post elaborates why.

I will recall to you, however, that Scott Alexander argues that people are irrational.  Quoting section three-point-one-one of the Anti-Libertarian FAQ:

Old-school economics assumed choice to be "revealed preference": an individual's choices will invariably correspond to their preferences, and imposing any other set of choices on them will result in fewer preferences being satisfied.

...

But the past fifty years of cognitive science have thoroughly demolished this "revealed preference" assumption, showing that people's choices result from a complex mix of external compulsions, internal motivations, natural biases, and impulsive behaviors. These decisions usually approximate fulfilling preferences, but sometimes they fail in predictable and consistent ways

Revealed preference is actually a concept originating in Utilitarianism: Link

So people are irrational only with respect to Utilitarian philosophy.  Which is because people don't actually -follow- Utilitarian philosophy.

Quick question: Has Scott Alexander actually constructed a utility function?  Has any Utilitarian?  "Revealed Preference" was the Utilitarian answer to the question "Well, given that we haven't defined a Utility function, what should we substitute in its place?"  The answer: Preference.

This is where our expectations on preference to diverge.  According to Revealed Preference, preference should demonstrate an individual's underlying utility function; to the extent that it doesn't, Revealed Preference is in fact wrong.  Scott Alexander argues that this demonstrates that people are irrational; alternatively, it demonstrates that Revealed Preference isn't a valid mechanism by which to reconstruct people's utility function.  The argument that non-utilitarians aren't rational because their behavior doesn't conform to utilitarianism is... well, judge for yourself.

Utilitarianism is sophistry at its finest; a philosophy of morality which draws its validity from the mathematical and logical principles it is built on - that isn't operating on mathematical constructs.  Again, do you, our putative utilitarian, have a utility function?  If not, none of those mathematical or logical principles apply: They're rationalizations for the behavior you want to engage in anyways.  You're not balancing value A against value B against value C to compute the sum utility of some set of considered actions, you're pretending your brain has already done all that and the outcome of that is that your preferences already do the work that your philosophy says needs to be done.  That's what "Revealed Preference," as an idea, is really all about.

So I guess my response on section 3.1.1 was off the mark: I was in fact revealing my irrationality, because my preferences didn't take into account the utility the different options offered me.  But it's a conditional irrationality, conditioned upon the idea that utility is the rational thing to predicate my preferences upon.  In practice, I simply didn't care about either the utility or the option.

The million-dollar question then becomes, however, given that preferences don't reveal an underlying utility-calculation node in my brain (which I didn't expect to be there anyways), what exactly is revealed utility, and how do we calculate it?

If preferences aren't revealing, then preference-based logical arithmetic is just building castles on sand.

Not that there's anything wrong with a preference-based moral philosophy, mind.  You just lose out on how official and well-reasoned utilitarianism sounds.