• Posts
  • RSS
  • ◂◂RSS
  • Contact

  • Conversation with Bryce Wiedenbeck

    July 14th, 2017
    airisk, giving  [html]
    A few days ago I spoke with Bryce Wiedenbeck, a CS professor at Swarthmore teaching AI, as part of my project of assessing superintelligence risk. Bryce had relatively similar views to Michael: AGI is possible, it could be a serious problem, but we can't productively work on it now.

    Before our conversation he looked some at Concrete Problems in AI Safety (pdf) and Deep Reinforcement Learning from Human Preferences (pdf). His view on both was that they were good work from the perspective of advancing ML but very unlikely to be relevant to making AGI safer: the systems that get us to AGI will look very different from the ones we have now.

    One reason is that he saw a lot of learning from humans as being mediated by learning utility functions, but he sees utility functions as a very limited model. Economists and others use utility functions when talking about people because that's mathematically tractable, but it's a bad description of how humans actually behave. Trying to come up with utility functions that best explain human preferences or behavior probably solves some problems nicely and is helpful, but while Bryce wouldn't completely rule it out he thought it was very unlikely to get us to AGI.

    We tried to get more into why he thinks implementations for AGI will look vastly different from what we will have today, and couldn't make progress there. Bryce thinks there are deep questions about what intelligence really is that we don't understand yet, and that as we make progress on those questions we'll develop very different sorts of ML systems. If something like today's deep learning is still a part of what we eventually end up with, it's more likely to be something that solves specific problems than as a critical component.

    (This has been a common theme in my discussions with people recently: very different intuitions on the distance to AGI in terms of technical work required, and also on whether work we're doing today is likely to transfer.)

    Comment via: google plus, facebook

    Recent posts on blogs I like:

    It's ok to feed stray cats

    Before we had kids, Jeff and I fostered a couple of cats. One had feline AIDS and was very skinny. Despite our frugal grocery budget of the time, I put olive oil on her food, determined to get her healthier. I knew that stray cats were not a top global pr…

    via Giving Gladly May 15, 2021

    Collections: Teaching Paradox, Europa Universalis IV, Part III: Europa Provincalis

    This is the third part of our series (I, II) examining the historical assumptions of Paradox Interactive’s grand strategy computer game set in the early modern period, Europa Universalis IV (which is in turn the start of a yet larger series looking at sev…

    via A Collection of Unmitigated Pedantry May 14, 2021

    Randal O’Toole Gets High-Speed Rail Wrong

    Now that there’s decent chance of US investment in rail, Randal O’Toole is resurrecting his takes from the early Obama era, warning that high-speed rail is a multi-trillion dollar money sink. It’s not a good analysis, and in particular it gets the reality…

    via Pedestrian Observations May 12, 2021

    more     (via openring)


  • Posts
  • RSS
  • ◂◂RSS
  • Contact