Thoughts on Existential Risk

March 11th, 2018
ea
I recently talked to someone who wanted to know what I thought about existential risk (x-risk). The idea is, how do we keep humanity from being destroyed, or otherwise failing to meet its potential? Within the EA movement there's been some shift in focus from global poverty towards x-risk over the past few years, and as someone who's been advocating for and funding poverty-related organizations they thought I might have some thoughts.

Here's more or less what I said:

  • Future people matter, and we owe it to our children's children to keep from wiping ourselves out. As future people who aren't around yet to advocate for themselves, most decisions today aren't fair to them. A big contribution to global inequality is that the worst suffering is far away from the richest people, and another is that the people involved are less in a position to advocate for themselves: both of these apply even more strongly to future generations.

  • There could potentially be a lot of future people. The scale here is much larger than anything involving the people currently alive on earth.

  • I'm much more skeptical than most people I talk to, even most people in EA, about our ability to make progress without good feedback. This is where I think the argument for x-risk is weakest: how can we know if what we're doing is helping, and how can we do as much good as we can without that? This has been my main concern since I first started thinking about x-risk

  • Cause prioritization, evaluation, and thinking through potential future scenarios are all really hard, and I expect they benefit both from focused study and conversations with people who've similarly put in a lot of thought. As an EA who's gone into earning to give, however, it does make me feel somewhat disconnected. I could say "seems like community consensus is to prioritize x-risk more, despite the reasons that it doesn't seem tractable to me" but that's not how I work, and it's not really EA. I feel pretty conflicted about this, and it's an additional reason to be wary of earning to give.

  • Overall, I've been feeling unhappy about my giving since at least 2016. I think our 2017 donations were valuable, but flexibility is one of the main advantages of earning to give, and it's not one I'm making much use of. Good giving takes time, research, and investigation, and right now I'm not really able to put in that work.

I'm unsure enough that if I don't sort this out this year I might shift giving from GiveWell's top charities to a donor-advised fund. Though if I don't sort this out this year I'm not all that convinced that I'll know better next year either.

(I'm not considering ways of doing good things with my career other than earning to give currently; all of those options seem like they would involve moving or working remotely, neither of which I want to do.)

Referenced in:

Comment via: google plus, facebook, substack

Recent posts on blogs I like:

Mindful posting

It is important, for the preservation of American democracy and maybe the world, that the Democrats take the House and Senate in November.

via Thing of Things September 18, 2026

There's no point at which turning your brain off will work

In early 2025, I started seeing people turn off their brain as they use LLMs1. They would have an LLM take an action (summarize text, write some code, etc.), and just assume that it worked2. This generally didn't work in early 2025 and the result was …

via Posts on September 18, 2026

My Weirdy Creatures

Over the last few days, I've been working on some little creatres. But they're very weird. I call them "weirdy creatures". I've made quite a lot of them, I think maybe 5-6, not sure. Weirdy Predator This is a weirdy predator. They'…

via Anna Wise's Blog Posts September 13, 2026

more     (via openring)