• Posts
  • RSS
  • ◂◂RSS
  • Contact

  • Superintelligence Risk Project Update II

    July 18th, 2017
    airisk, giving  [html]
    This is the beginning of my third week looking into potential risks from superintelligence (kickoff, update 1) and I think I'm hitting diminishing returns. I'm planning to wrap up in the next day or so, and go back to figuring out what I should work on next.

    Last week:

    • Technical Distance to AGI: I hypothesized (incorrectly) that the main difference between ML researchers who thought we could vs couldn't work on AI risk now was how far off they thought AGI was, in terms of some combination of time and technological distance. Recommended comments: Jacob, Paul, Dario. I also made a 1:9 bet with Dave on whether we'll have driverless cars in the next 10 years.

    • Examples of Superintelligence Risk: I collected the examples I've seen of what a "loss of control of an AI" catastrophe might look like, and tried to figure out why the examples are much less realistic than we see of other existential risks like nukes or bioterror. Recommended comments: Eliezer, Jim, Paul.

    • Conversation with Bryce Wiedenbeck: I talked to an AI professor, main takeaway being that he thinks the technical distance to AGI is very high. Recommended comment: Dario.

    • I found Open Phil's notes on Early Field Growth interesting, especially their section on failure modes in cryonics and molecular nanotechnology. My takeaway was that heavy popularization of a new field prior to scientific success leads scientists on the border of the new field to take on an oppositional stance. The field gets starved of people who could do substantial technical work, makes minimal progress, and I think people also avoid the areas around its edges, like a chilling effect. I see superintelligence risk as just on the edge of this, where it could go either way. Which also makes me (weakly) think that Daniel Dewey's point on the relevant field-building effects of MIRI-style vs prosaic AI-style should maybe go farther. Specifically, you don't want safety to be thought of as a "we don't do that, those people are cranks" sort of thing, so it's a lot better if AI safety develops primarily as a field within ML.

    • Spoke to three other ML researchers, one of which I'm hoping to write up conversation notes from.

    • I had applied for an EA Grant when I thought I might spend longer on this, but withdrew after getting to the phone interview stage.

    • I spent most of Monday working on the house and running errands instead of on this project.

    (A big takeaway for me is that I don't like doing this kind of work very much. I think it's a combination of two things: it's isolated work (as I'm doing it) and it's a kind of thinking that I enjoy in moderation, but not for full time work. These two combine pretty strongly: this kind of thinking is much more enjoyable for me when working with someone else, where we can have a lot of conversations to clarify ideas and look for the best areas to make progress. David Chudzicki, one of my housemates, has been helpful here, and we've talked a lot, but it's still something I'm mostly working on alone.)

    Comment via: google plus, facebook

    Recent posts on blogs I like:

    Governance in Rich Liberal American Cities

    Matt Yglesias has a blog post called Make Blue America Great Again, about governance in rich liberal states like New York and California. He talks about various good government issues, and he pays a lot of attention specifically to TransitMatters and our …

    via Pedestrian Observations November 19, 2020

    Collections: Why Military History?

    This week, I want to talk about the discipline of military history: what it is, why it is important and how I see my own place within it. This is going to be a bit of an unusual collections post as it is less about the past itself and more about how we st…

    via A Collection of Unmitigated Pedantry November 13, 2020

    Misalignment and misuse: whose values are manifest?

    Crossposted from world spirit sock puppet. AI related disasters are often categorized as involving misaligned AI, or misuse, or accident. Where: misuse means the bad outcomes were wanted by the people involved, misalignment means the bad outcomes were wan…

    via Meteuphoric November 13, 2020

    more     (via openring)


  • Posts
  • RSS
  • ◂◂RSS
  • Contact