Value of a Computational Process?

July 9th, 2012
morality
Consider a future person living a happy and fulfilling life. They're unfortunate enough to suffer a severe accident, but there's time to preserve and then scan their brain fully, after which they can be brought back up in an emulator on a computer. [1] It doesn't matter that they're now running on digital hardware instead of in a biological brain; they're still a person and they still count.

Now imagine this person or "em" asks to be let alone, cuts off all communication from the rest of the world, and rejoices privately in finally being able to fully explore their introverted nature. This isn't what I imagine myself doing, but is a choice I can respect.

Someone comes along and suggests turning off this person's emulation on the grounds that no one will know the difference, and we can use the hardware for something else. This seems wrong. Which means this computational process is valuable entirely for its own sake, independent of its effect on the world.

Unlike biological brains, computational processes are very flexible. We could run many copies, or run them much faster or slower than usual. We could run a specific segment of their independent experience repeatedly, perhaps the happiest few moments. It also seems unlikely that a full emulation of a human is the only thing that's valuable. Perhaps there are simpler patterns we could emulate that would be much better in terms of value per dollar?

I'm trying to reduce my concept of value and getting lots of strange questions.


[1] I think this will be possible, but not for a while.

Comment via: google plus, facebook, lesswrong, substack

Recent posts on blogs I like:

Avoiding being raped

A comment on a recent blog post asked for advice for a trans woman who kept making dangerous dating and sex choices, because she hadn’t received the female socialization that would have taught her to avoid threatening people and risky situations.

via Thing of Things October 28, 2026

“We should be more ambitious with the software we build”

Since early this year, Michael Malis (of pgrust) has been showing me interesting things he's done and saying [W]e should be more ambitious1 with the software we build There is, understandably, a lot of skepticism about projects like pgrust because there…

via Posts on October 24, 2026

Was the HuggingFace hacking incident scheming?

In July 2026, internal OpenAI agents hacked HuggingFace (HF) in order to figure out how their evaluation grader works. This involved a thousand agents coordinating on an improvised message board to carry out an elaborate sequence of exploits. A later grou…

via Victoria Krakovna October 18, 2026

more     (via openring)