• Posts
  • RSS
  • ◂◂RSS
  • Contact

  • Evaluating Opus

    August 8th, 2020
    audio, singing, tech  [html]
    Several months ago, I proposed a bucket brigade approach to singing with friends over the Internet. Recently, Glenn and I have been working on an implementation (prototype source code). We can't use WebRTC (I think) because we need fine grained control over latency, so we're doing everything manually. Which raises the question of compression.

    CD quality audio, which your browser is happy to give you, represents a second of audio with 44,100 ("44.1kHz") 16bit samples. It's possible to send this raw over the internet, but that's a lot of bandwidth: 88,200 bytes per second in each direction, or 1.4 Mb/s. That's not completely nuts, but many people on Wi-Fi are in environments that won't be able to handle that consistently.

    One way to do better is to just throw away data. Instead of taking 44,100 samples every second, just take a quarter of that (11,025). Instead of using sixteen bits per sample, just send the eight most important ones. This is a factor of eight smaller at 175 kb/s round trip, but compare:

    (CD Quality: 44.1kHz, 16bit)

    (Reduced: 11kHz, 8bit, undithered)

    This doesn't sound great: there are some weird audio artifacts that come from rounding in a predictable way. Instead, people would normally dither it:

    (Reduced: 11kHz, 8bit, dithered)

    This sounds much more realistic, but it adds a lot of noise. Either way, in addition to sounding awkward this approach also uses more bandwidth then would be ideal. Can we do better?

    I wrote to Chris Jacoby, and he suggested I look into the Opus codec. I tried it out, and it's pretty great:

    (Opus: 64 kb/s)

    (Opus: 32 kb/s)

    (Opus: 24 kb/s)

    (Opus: 16 kb/s)

    All of these are smaller than the reduced version above, and all of them except the 16 kb/s sound substantially better.

    In our use case, however, we're not talking about sending one large recording up to the server. Instead, we'd be sending batches of samples off every, perhaps, 200ms. Many compression systems do better if you give them a lot to work with; how efficient is opus if we give it such short windows?

    One way to test is to break the input file up into 200 ms files, encode each one with opus, and then measure the total size. The default opus file format includes what I measure as ~850 bytes of header, however, and since we control both the client and the server we don't have to send any header. So I count for my test file:

    format size
    CD Quality 706 KB
    11kHz, 8bit 88 KB
    Opus, 32 kb/s 44 KB
    Opus, 32 kb/s, 200ms chunks 51 KB

    I was also worried that maybe splitting the file up into chunks would sound bad, either because they wouldn't join together well or because it would force opus to use a lower quality encoding. But it sounds pretty good to me:

    (200ms chunks)

    Another question is efficiency: how fast is opus? Here's a very rough test, on my 2017 Macbook Pro:

    $ time for i in {{1..1000}}
        do opusenc --quiet \
           --music --bitrate 32 \
           row-row-row.wav output.opus32
        done
    $ time for i in {{1..1000}}
        do opusdec --quiet \
          row-row-row.opus32 output.wav
        done
    

    The test file is 8.2 seconds of audio. Encoding 1000 copies took 68s, decoding took 43s, for a total of 111s. This is 74x realtime, and is the kind of embarrassingly parallel computation where you can just add more cores to support more users.

    Running on the server it looks like we can use the python bindings, and on the client we can use an emscripten web assembly port.

    Overall, this seems like a solid chunk of work, but also a good improvement.

    Comment via: facebook, lesswrong

    Recent posts on blogs I like:

    Governance in Rich Liberal American Cities

    Matt Yglesias has a blog post called Make Blue America Great Again, about governance in rich liberal states like New York and California. He talks about various good government issues, and he pays a lot of attention specifically to TransitMatters and our …

    via Pedestrian Observations November 19, 2020

    Collections: Why Military History?

    This week, I want to talk about the discipline of military history: what it is, why it is important and how I see my own place within it. This is going to be a bit of an unusual collections post as it is less about the past itself and more about how we st…

    via A Collection of Unmitigated Pedantry November 13, 2020

    Misalignment and misuse: whose values are manifest?

    Crossposted from world spirit sock puppet. AI related disasters are often categorized as involving misaligned AI, or misuse, or accident. Where: misuse means the bad outcomes were wanted by the people involved, misalignment means the bad outcomes were wan…

    via Meteuphoric November 13, 2020

    more     (via openring)


  • Posts
  • RSS
  • ◂◂RSS
  • Contact