• Posts
  • RSS
  • ◂◂RSS
  • Contact

  • Browser Audio Latency

    February 2nd, 2021
    audio, bucket_brigade, tech  [html]
    Any time you are working with digital audio, you have to handle latency: how long does it take from when the audio enters the system until it leaves again? Special purpose professional-level audio hardware generally has been optimized for extremely low latency, often adding as little as 2ms. Extreme low latency, however, has costs: scheduling must be very predictable, power consumption is higher, audio processing algorithms can look at fewer samples, and you have much less recovery time before causing an audible glitch.

    Web browsers are in an interesting position, where they have to work with whatever audio hardware the computer happens to have, and need to handle a very wide range of use cases.

    I've recently been working on two musical projects where I care a lot about latency, but in different ways:

    • Bass Whistle JS: a bass synthesizer that you control by whistling. To feel natural and responsive, it needs the lowest possible latency.

    • Bucket Brigade: a platform for making music with others over the internet. It doesn't need low latency, but it does need to know precisely how much there is so it can compensate.

    But how much latency is there? We can measure it: send a beep to the speaker, check how much time passes until we detect the beep in the samples coming from the microphone (example worklet). On my 2016 15" MacBook running OSX 11.1, if I do:

      var audioCtx = new window.AudioContext();
      var micStream =
        await navigator.mediaDevices.getUserMedia(
          {audio: true});
    
    I measure ~67ms end-to-end latency in Chrome, and ~55ms in Firefox. Instead of using default settings, though, we could tune for low latency:
      var audioCtx = new window.AudioContext({
        latencyHint: 0,
      });
      var micStream =
        await navigator.mediaDevices.getUserMedia({
          audio: {
             latency: 0,
             echoCancellation: false,
             noiseSuppression: false,
             autoGainControl: false,
          }});
    

    Note that we're asking AudioContext for an output latency of '0' and not interactive. While MDN documents interactive as "try to use the lowest possible and reliable latency it can", '0' is even lower. We are also turning off all of the smart processing options, which would add latency.

    This gives me ~19ms in Chrome and ~14ms in Firefox. Try it yourself.

    In the other direction, for Bucket Brigade, I tune for high latency with the hope that it will be completely consistent:

      var audioCtx = new window.AudioContext({
        latencyHint: 'playback',
      });
      var micStream =
        await navigator.mediaDevices.getUserMedia({
          audio: {
             echoCancellation: false,
             noiseSuppression: true,
             autoGainControl: false,
          }});
    

    Setting high latency values on getUserMedia doesn't seem to have an effect in either Chrome or Firefox, so I've left it off.

    One of the most awkward aspects of Bucket Brigade today, which also would apply to anything else that lets you record multitrack audio in the browser, is that you have to measure latency. There is not yet any way to ask a browser how much end-to-end latency there is. There's input latency, MediaTrackSettings.latency, only implemented in Chrome, but in my testing it always returns 0.01 (10ms) even when I switch to a higher latency microphone. There's output latency, AudioContext.outputLatency, only implemented in Firefox, but in my testing it returns implausibly low numbers, less than half of the end-to-end latency.

    There are some situations in which measurement is going to be the best you can do: Bluetooth headsets are notorious for reporting much lower latencies than they actually implement. That is beyond the control of the browser, however, and giving developers the number the devices claim would still be a big improvement. Even if you can't completely trust the number, if it changes then you know you need to re-run measurement.

    Comment via: facebook, lesswrong

    Recent posts on blogs I like:

    Fireside Friday, February 26, 2021

    Fireside this week, but next week we are diving into our long awaited series on pre-modern textile production, though we will be particularly focused on the most important clothing fibers in the Mediterranean world, wool and linen (rather than, say, silk …

    via A Collection of Unmitigated Pedantry February 26, 2021

    Cut-and-Cover is Underrated

    Subways can be built in two ways: cut-and-cover, and bored tunnel. Cut-and-cover means opening up the street top-down, building the system, and roofing it to restore surface traffic; bored tunnel means opening up one portal and digging horizontally, with …

    via Pedestrian Observations February 26, 2021

    The Troubling Ethics of Writing (A Speech from Ancient Sumer)

    (Translated from a transcript of an ancient Sumerian speech by Uruk's most well-respected Scriptological Ethicist) Writing is a profoundly dangerous technology: Access to writing was initially, and still remains, uneven. What's worse, the rich are m…

    via BLOG - Cullen O'Keefe February 15, 2021

    more     (via openring)


  • Posts
  • RSS
  • ◂◂RSS
  • Contact