• Posts
  • RSS
  • ◂◂RSS
  • Contact

  • Conversation with Michael Littman

    July 9th, 2017
    airisk, giving  [html]
    A few days ago I talked to Michael Littman, a CS professor at Brown working in Machine Learning and Artificial Intelligence, as part of my project of assessing superintelligence risk. Michael doesn't think that work on long-term AI risk is valuable, and I wanted to get a better sense of why. In the same fashion as my conversation with Dario Amodei, here's a summary of what I learned from him.

    Michael believes that we'll probably have machines that are as smart as humans (AGI, for short) at some point. He doesn't think it's certain, and he's really interested in the question of whether human-style intelligence is achievable, but he thinks we'll probably figure it out eventually. At the same time, he sees this as very far off. Not just "far" in the sense of a long time coming, though he does think that's probably the case, but "far" in the sense of how much more technical and conceptual work is required before we get there.

    For example, I asked him what he thought of the idea that to we could get AGI with current techniques, primarily deep neural nets and reinforcement learning, without learning anything new about how intelligence works or how to implement it ("Prosaic AGI" [1]). He didn't think this was possible, and believes there are deep conceptual issues we still need to get a handle on. He's also less impressed with deep learning than he was before he started working in it: in his experience it's a much more brittle technology than he had been expecting. Specifically, when trying to replicate results, he's often found that they depend on a bunch of parameters being in just the right range, and without that the systems don't perform nearly as well.

    The bottom line, to him, was that since we are still many breakthroughs away from getting to AGI, we can't productively work on reducing superintelligence risk now.

    He told me that he worries that the AI risk community is not solving real problems: they're making deductions and inferences that are self-consistent but not being tested or verified in the world. Since we can't tell if that's progress, it probably isn't. I asked if he was referring to MIRI's work here, and he said their work was an example of the kind of approach he's skeptical about, though he wasn't trying to single them out. [2]

    I asked him what he thought of Concrete Problems in AI Safety (pdf), and Michael told me he hadn't read it in detail but had skimmed it. His response was that getting better at specifying what we care about is a great research problem, and an area he's working on. It's technology that would be useful to us today, and he's happy to see it developed better. He doesn't see it, however, as useful from a long-term AI safety perspective because we just don't know anything about what AGI would look like yet.

    At this point, I was trying to figure out if the key disagreement between him and, say, Dario or Paul was on whether Prosaic AGI was possible. I asked what he would think we should work on if he believed that all we needed to get to AGI was a lot more engineering work on current technologies without discovering anything fundamentally new. Unfortunately, this was far enough from what he actually believes that he found it really hard to get into the hypothetical. He thought that even in this hypothetical he probably wouldn't think "loss of control" issues were among the most urgent ones we have, but he really wasn't very sure what he would think if he were actually in that situation.

    [1] I'm not sure whether Paul Christiano coined this term, but he's who I've heard it from. For more detail on it, see the first section of Prosaic AI Alignment.

    [2] He followed up by email: "At root, I feel that the AI risk community is trying to extrapolate from our cartoonish current understanding of what intelligence is and how it works and making a deeply unsubstantiated leap that more raw computation will give us a great deal more of whatever is powerful about intelligence. This kind of extrapolation has been very successful in areas like physics, where we can infer the structure of far away galaxies and project the impact of what it means for more energy or gravity or temperature to be concentrated in one place. I think it's a mistake to think of intelligence as a physical quantity and to put any faith into similar extrapolations in the cognitive realm. It's simply too easy for us to misinterpret our own introspections about intelligence and the science of the production of creative new ideas is still very very young."

    Comment via: google plus, facebook

    Recent posts on blogs I like:

    Best Practices Civil Service

    I propose that transportation agencies hire people whose job is to keep abreast of global developments in the field and report on best practices. Which agencies should do it? Ideally, all urban ones. Very small ones should piggyback on large ones, or part…

    via Pedestrian Observations June 14, 2021

    Collections: The Queen’s Latin or Who Were the Romans? Part I: Beginnings and Legends

    Who were the Romans? How did they understand themselves as a people and ‘Roman’ as an identity? And what were the implications of that understanding – and perhaps more importantly the underlying reality – for Roman society and the success of the Roman Emp…

    via A Collection of Unmitigated Pedantry June 11, 2021

    It's ok to feed stray cats

    Before we had kids, Jeff and I fostered a couple of cats. One had feline AIDS and was very skinny. Despite our frugal grocery budget of the time, I put olive oil on her food, determined to get her healthier. I knew that stray cats were not a top global pr…

    via Giving Gladly May 15, 2021

    more     (via openring)

  • Posts
  • RSS
  • ◂◂RSS
  • Contact