Jeff Kaufman  ::  Blog Posts  ::  RSS Feed  ::  Contact

Superintelligence Risk Project: Conclusion

I spent most of July working on a project to evaluate risk from AI that is smarter than humans. I posted updates as I went [1] but intended to write up a summary with conclusions when I finished. Unfortunately things petered out at the end: I started applying to jobs, wanted to finish a bunch of house projects, and kept waiting for one final set of conversation notes. [2] The more I put off the final write-up the more I kept putting it off, and now it's September. So, in the interest of just writing the thing up, here's where I am now:

Summary: I'm not convinced that AI risk should be highly prioritized, but I'm also not convinced that it shouldn't. Highly qualified researchers in a position to have a good sense the field have massively different views on core questions like how capable ML systems are now, how capable they will be soon, and how we can influence their development. I do think these questions are possible to get a better handle on, but I think this would require much deeper ML knowledge than I have. more...

Using Docs for Running Notes

When I work I keep two kinds of notes:

  • Short summary notes aimed at people who are interested in what I'm doing (boss, coworkers, me way in the future trying to figure out what I was working on then)

  • Verbose running notes where I write everything down about what I'm currently doing. This can include commands I've run [1] and their output, anything that will help me reconstruct what I was working on, todo lists, or anything else that's helpful to be written down.

The primary difference between these isn't access (both are readable by any of my coworkers) but audience (who I expect to be reading them).

I've been using Google Docs for my running notes, and while it's not ideal it's pretty good. Some configuration options that I find help a lot:

  • Redefine "heading 1" to be "monospaced 10pt" and then Ctrl+Alt+1 becomes a shortcut for "code font". Ctrl+Alt+0 is still the default shortcut for normal text.

  • Turn off the default print layout (View > Print Layout) to remove the large gaps between logical pages. If I could remove the concept of "page" from this all together I would, since it will never be printed out.

  • Switch the document from "Letter (8.5" x 11")" to "Statement (5.5" x 8.5")" in Page Setup, with 0.1" margins. This lets me make the window nice and narrow. I'd be happier if the page would just re-flow as I changed the size of the window, but this is ok.

The main thing that's still missing is Emacs keybindings: Ctrl+A for beginning of line, Ctrl+D for end of line, Ctrl+D for delete, etc. On Mac this just works, and on Linux it works everywhere but Docs, but I don't know how to get it to work in Docs on Chrome on Linux.


[1] I also log all my shell history.

full post...

Rejoining Google

A week from Monday I'm starting back at Google. [1] I'm pretty excited about it, both the company and the team I'll be on. more...

Regularly Scheduled

Events that happen on a regular schedule lower the energy it takes to be social. Things like church every Sunday, a dance that's every 1st Saturday, or a dinner with friends on 2nd and 4th Wednesdays. In general, I think people would probably be happier if they had more things like this in their lives. [1] To help with this, I made a site:

The site lets you create an event, and then invite people: more...

Forking Currency

Lets say I start a new currency, the Credit, to compete with the Dollar. The traditional way to do this would be for me to offer to convert between Dollars and Credits, but let's say instead I give credits to existing Dollar owners. You show me a Dollar bill with a serial number I haven't seen before, I give you Credits.

I think this is a helpful analogy for thinking about the recent Bitcoin Cash fork. Anyone can make a Dollar fork, and for people who hold Dollars in their native form, paper, this isn't a problem: you can choose to participate in the fork or not, mostly based on whether the fork ends up being worth something. On the other hand, what if someone owes you a Dollar, perhaps because you loaned it to them, or you deposited it in their bank? In the Dollar case this is very clear: they have no obligation to interact with Credits at all, and if they happen to give you back bills that have already been registered for Credits that's your problem. If they registered all the bills they held and kept the Credits for themself, even that would probably be fine. more...

Yudkowsky and MIRI

In talking to ML researchers, many were unaware that there was any sort of effort to reduce risks from superintelligence. Others had heard of it before, and primarily associated it with Nick Bostrom, Eliezer Yudkowsky, and MIRI. One of them had very strong negative opinions of Eliezer, extending to everything they saw as associated with him, including effective altruism. more...

More Posts


Jeff Kaufman  ::  Blog Posts  ::  RSS Feed  ::  Contact  ::  G+ Profile