Wednesday, 24 October 2007

The road to Universal Ethics: Universal Instrumental Values

So here we are: thinking, eating, debating, loving, working, suffering, blogging, procrastinating and just generally living. Life is full of stuff, and it’s full of choices. Should I go to the party or should I stay home and work? Should I donate to charity? Should I publish? Should I kiss her? Sometimes you can’t choose to do it all, sometimes you’re forced to deny one thing in order to allow another. People (myself included) like to think that our choices are not arbitrary, and so we like to come up with abstract theories like Ethical Theories and Theories of Value to put our choices into an overall framework.

But of course, the question nags, how do you choose your theory? In fact what is the point of adopting a theory of choice-making if there are infinitely many such theories and no way to choose between them!

I have an image of a man turning up at a specialist electrical store:

Man: “Hi, I have a terrible problem: I just can’t seem to make choices. Can you sell me a choice-making machine?”

Attendant: “Certainly, sir. Over here we have a whole range of them. This one is the Categorical-1000 from Kant Technologies. That’s the Utilitarian-XAct and the Utilitarian-XRule, both from Mill and Bentham Ltd. Here we have one of the most modern machines: it’s the Veil of Ignorance from Rawls Industries; it makes very unbiased choices, but it does have the drawback of making you forget who you are for a few minutes, which some customers may find disconcerting. These are just a few of the many choice making machines we have here. Take your pick!”

Man: “NO! – I can’t – I can’t choose!”

Are we wasting our time discussing moral or ethical systems? I think not, and to illustrate why, I’ll continuine our trip through BlissWorld. Recall that BlissWorld is a world where people think that U = Sum(pleasure) – Sum(pain) is the absolute measure of goodness, and they seek to maximize U at all costs. In other words, they are hedonistic utilitarians. They have devised a type of machine called the Bliss Machine, which directly stimulates the pleasure centers of the human brain; using such a machine certainly is very pleasurable.

Before they hook every human being up to their Bliss Machine, the great moral thinkers in Bliss World start thinking about what course of action will actually maximize utility over all of time versus which action will produce a lot of utility now but not so much later. If they think carefully, they’ll realize that, according to their measure of goodness, two planets of blissed out people are better than one, hence they keep some people off the Bliss Machines to go and colonize mars. But once you’ve hit upon the idea of colonizing one extra planet, it surely must dawn on you that you shouldn’t really stop there. You should really go and colonize the entire universe; indeed you should go and do this as quickly as possible. Remember, in BlissWorld you should do whatever you can to maximize U = Sum(pleasure) – Sum(pain), this is the definition, if you like, of the word “should” in BlissWorld society.

The moral thinkers on BlissWorld go and instruct everyone that it is morally wrong to go and “bliss yourself out”, until such a time as the colonization of the universe is well under way. Even though the Bliss Worlders’ notion of intrinsic value is to maximize U, they now have a notion of instrumental value – including, amongst other things, to colonize the universe. Thus Bliss Worlders go and study physics and engineering, build spaceships, foster economic growth, think of new and better forms of government, and do all sorts of things that we would think of as fairly normal things. Indeed their society might look a lot like ours – because most of the things that they are doing are things of a “generally useful” nature – things like educating people, building communities, engaging in scholarly research, maintaining a healthy economy, etc. Remember though that people on Bliss World all have the ultimate aim of just hooking up to the pleasure machines that they invented. Their society is a little bit totalitarian. No-one is allowed to “waste time” on things that will not ultimately lead to the colonization of the universe or the production of Bliss Machines. People have very little personal freedom, and anyone who kicks up a fuss or tries to question the nature of BlissWorld is simply forcibly hooked up to a bliss machine for the rest of their life.

A little while later, the moral thinkers on BlissWorld realize that they may have made a bit of an oversight. Their utility function, U, says that they should maximize Sum(pleasure) – Sum(pain). Maybe there is a “super-bliss” machine which will allow a human to experience even more pleasure than a Bliss Machine does? Indeed, it may be possible to re-design humans so that they are capable of experiencing more pleasure. So the Bliss Worlders have to add these tasks to their list of instrumental values: they should get people to think of ways of enhancing the human mind and thinking of more intense forms of pleasure. The BlissWorlders now get some people to start studying human neurobiology, and other people to study the human pleasure from a psychological, cultural and even artistic point of view. BlissWorld is a hive of activity.

The BlissWorlders find themselves having to think creatively to solve the new problems they are facing. Most Bliss Worlders find it very hard to think out of the box, because they are used to towing the line – remember this world is a brutally totalitarian one. In fact the directors of research find that the best ideas often come from those people who are on the borderline of tolerability - the people who are most outspoken and who have the most individuality. The research directors go to the leadership of BlissWorld and ask for people within the research community to be given more personal freedom – for example to be allowed to socialize, to engage in artistic activity, to write books, to form political movements.

The leaders of BlissWorld agree to a pilot project in a remote lunar colony called Utility Vista. After a shaky start, the people at Utility Vista begin to perform way above average. An ordinary citizen called Kaluza Klein who works as a sewerage technician started reading physics books in his spare time (reading books in your spare time was strictly prohibited under the old rules), and came up with a startling new theory of space and time which had radical implications for the fate of BlissWorld. He worked out that our 3-dimensional universe may be sitting in a larger 26-dimensional spacetime, where the remaining 23 dimensions are compactified. It may be possible, he concludes, to tunnel through to a region of higher-dimensional space. This is important for the people of bliss-world, because in 3-dimensional space there is a limit to how quickly new space can be colonized, given by the volume of a spherical shell which expands at the speed of light. Finding new laws of physics would allow this limit to be broken, and would allow a huge increase in the number of people alive, and hence the value of U.

Encouraged by this progress, the BlissWorld government decides to allow more personal and political freedoms for all of its citizens. It allows people to do things that are only tenuously linked to the Ultimate Goal.

The interesting point of this example is that the intrinsic goals that the BlissWorlders are logically compelled to pursue start to look very sensible.

If you replaced the notion of ultimate intrinsic value U with some other intrinsic value V, it is quite possible that you’ll end up pursuing exactly the same instrumental values; those things that are just "generally useful".

It may be true that for a very wide class of notions of intrinsic value, you always end up with the same notion of instrumental value.

Let me call this hypothesis the Universal Instrumental Value Hypothesis

Let U denote a utility function which represents some idea of what is intrinsically valuable, and write I(U) for the notion of instrumental value that U gives rise to. For any notion of value which grows with the number of people alive, “Progress”(progress in physics, engineering, economics, communication, etc) always becomes an instrumental value. For example, if R = the number of people alive who have red hair, then I(R) includes “Progress” as defined above. If Z = number of prayers which are said to the god Zeus, then I(Z) also includes “Progress”.

Anyone whose instrumental value includes progress should obviously also include “Knowledge” and it should include “Creativity”, because one moment of creative genius can equate to a huge amount of progress, and it is an inherent property of creativity that you cannot predict in advance where that creativity will come from. Certain personal (and even political) types of “Freedom”, and “Diversity” are therefore also included – because a group of people who all think in the same ways are less creative than a diverse group.

Even some kinds of intrinsic value which make no reference to people will include instrumental values which require people. For example if P = “the number of paperclips in the universe”, then I(P) includes “Knowledge” and “Progress”. But then it also includes “Creativity”, and “Freedom” and “Diversity”.




4 comments:

acumensch said...

You touched on something that has always bothered me -- this business of choosing a heuristic device for choosing heuristic devices.

-- be it ethics, politics, epistemology, metaphysics, etc.

As far as BlissWorld goes, how can you be sure that utility maximizing ought to be the goal of the governors of BlissWorld? Perhaps that's for the inhabitants to decide individually. You mentioned that it was totalitarian, and I think that's right. And by doing that, you've escaped the political problem of heuristic uncertainty (Because you have a leadership committee who decides all these policies.) And this is assuming that some kind of leadership committee must choose a heuristic device.

But because utility maximizing is itself uncertain -- you ought to allow each member to freely engage in expeditions to Mars, etc.

That is to say, one way of escaping the political heuristic problem is by letting each person solve the problem themselves, letting each person choose a set of principles etc. The burden is then decentralized, and the subjectivity/objectivity problem is evaded.

Good post!

Roko said...

"As far as BlissWorld goes, how can you be sure that utility maximizing ought to be the goal of the governors of BlissWorld? "

I just chose a the first and simplest moral system that came to mind. The idea behind this is that although you have to choose SOMETHING to aim for in this world, and SOME set of rules for a society, in the end all those choices ought to lead through the same "universal" path. I call this "universal instrumental values". Thus the initial choice of utility function ought not to matter. This is inspired by the concept of a quotient in mathematics. In the language of maths, lots of seemingly different moral theories may all fall into the same equivalence class.

"But because utility maximizing is itself uncertain -- you ought to allow each member to freely engage in expeditions to Mars, etc."

Well said. This, I believe, highlights a very deep connection between ethics and uncertainty - stated bluntly it says that you should allow peopel freedom becuase free people will most effectively explore the space of possibilities.

Wei Dai said...

What happens when the paperclip-maximizing AI decides that it has learned enough about how to best convert the universe into paperclips and that it's time to actually do it? I'm not sure your line of thought leads anywhere, unfortunately.

Roko said...

"What happens when the paperclip-maximizing AI decides that it has learned enough about how to best convert the universe into paperclips and that it's time to actually do it?"

Yes, a good point. This is still very much a preliminary thought, and I'm not entirely sure of all the territory myself.

So, I have an argument that shows that, at least for a while, and at least to a certain extent, a large class of utility functions factor through what I call Universal Instrumental Values (UIV).

I am not suggesting that your actual behavior is completely independent of your utility function, because, as you say, eventually you'll want to start doing whatever [possibly quite silly] thing your utility function tells you to do.

Rather, I'm suggesting that UIV provides the cornerstone for a rather new approach to goal system design. Instead of having a fixed utility function/supergoal, you periodically promote certain instrumental values to terminal values i.e. you promote the UIVs.

This is quite a subtle idea, and really deserves a post of it's own, which I will provide!