This is a continuation of the previous post. Go read that first, if you haven’t already.
The Third Principle of Culture Engineering:
Humans have flexible values, which are chiefly defined by their engagement with other humans.
How do we know what to want? How do we know what counts as “desirable?” Most of it’s not baked in. We look at the people around us, and feel pressure towards caring about the things that they care about…or towards not caring about the things that they care about, for those of us who have picked up contrarian streaks…or towards any of a hundred other little preference-adjustments. We consume texts and other pieces of media, and absorb concepts that resonate with us.
A lot of our value-formation, as far as I can tell, is pure halo effect in action. (Or, even more bluntly, straight-up Pavlovian conditioning.) We look at a Good Thing that we already like or want, and all its traits become associated with that feeling of liking and wanting. Even traits that were previously value-neutral. And, soon enough, we have new suites of preferences built around those traits. Mustaches become sexy because that sexy singer grew a mustache. We develop strong anti-Fleem opinions because our despicable political opponents seem so enthusiastic about Fleem.
This is not to endorse pure blank-slate-ism. My limited exposure to modern developmental psychology suggests that, in fact, a number of important preferences — especially common root-level preferences that can manifest in a lot of different ways, like “it’s good to be high-status” — pretty much are baked in a lot of the time. But that goes only so far, and it’s not very far. Humans, in different times and places, have idolized a boggling array of outcomes. Members of the same species have invested their ego in becoming stylites, Imperial concubines, titans of industry, samurai retainers, and discursive rationalists. And they’ve all been happy to go far down their chosen paths, and none of them would want to trade with any of the others. (At least, you can easily find examples who wouldn’t want to trade.) Something is convincing all those people to want very different things.
To my mind, this Third Principle is the doom of preference utilitarianism as a practical way of approaching anything. It doesn’t make sense to treat people’s preferences as a given, because they’re not, they’re super-malleable. And if you hope to change the nature of human civilization, a large part of that is going to have to be teaching people to want the thing that you’re trying to build.
The Fourth Principle of Cultural Engineering:
There is no simple way to change people’s values.
This one is the most contingent of the Principles, by far — I’m not at all sure that I believe it in any kind of deep conceptual way. The universe is full of technologies and methodologies of which we as yet know nothing. Maybe, somewhere out there, there is a simple way to change people’s values! Who knows?
But, given what we’re capable of right now, the Fourth Principle is an important safeguard against overzealous application of the Third Principle. The fact that desires are malleable makes it tempting to think that they’re easily manipulable. But, well…just ask any parent who’s tried to make his kids want good and healthy things. Or any religious leader who’s tried to shepherd his flock towards righteousness. Or, um, any DARE instructor. The direct approach often doesn’t work. Figuring out which influences will influence is actually super hard.
So where does this leave us?
With a set of very general broad-brush guidelines for where to go:
- It’s important to help people do better than they would on their own…
- …which can theoretically be accomplished by inducing them to change their values into better ones…
- …but it’s also important to remember that this is not a trivial task.
- And also that, whatever you say or do, different people will react to it differently.
Following those guidelines leads us to culture engineering.