• Posts
  • RSS
  • ◂◂RSS
  • Contact

  • Browser Audio Latency

    February 2nd, 2021
    audio, bucket_brigade, tech
    Any time you are working with digital audio, you have to handle latency: how long does it take from when the audio enters the system until it leaves again? Special purpose professional-level audio hardware generally has been optimized for extremely low latency, often adding as little as 2ms. Extreme low latency, however, has costs: scheduling must be very predictable, power consumption is higher, audio processing algorithms can look at fewer samples, and you have much less recovery time before causing an audible glitch.

    Web browsers are in an interesting position, where they have to work with whatever audio hardware the computer happens to have, and need to handle a very wide range of use cases.

    I've recently been working on two musical projects where I care a lot about latency, but in different ways:

    • Bass Whistle JS: a bass synthesizer that you control by whistling. To feel natural and responsive, it needs the lowest possible latency.

    • Bucket Brigade: a platform for making music with others over the internet. It doesn't need low latency, but it does need to know precisely how much there is so it can compensate.

    But how much latency is there? We can measure it: send a beep to the speaker, check how much time passes until we detect the beep in the samples coming from the microphone (example worklet). On my 2016 15" MacBook running OSX 11.1, if I do:

      var audioCtx = new window.AudioContext();
      var micStream =
        await navigator.mediaDevices.getUserMedia(
          {audio: true});
    
    I measure ~67ms end-to-end latency in Chrome, and ~55ms in Firefox. Instead of using default settings, though, we could tune for low latency:
      var audioCtx = new window.AudioContext({
        latencyHint: 0,
      });
      var micStream =
        await navigator.mediaDevices.getUserMedia({
          audio: {
             latency: 0,
             echoCancellation: false,
             noiseSuppression: false,
             autoGainControl: false,
          }});
    

    Note that we're asking AudioContext for an output latency of '0' and not interactive. While MDN documents interactive as "try to use the lowest possible and reliable latency it can", '0' is even lower. We are also turning off all of the smart processing options, which would add latency.

    This gives me ~19ms in Chrome and ~14ms in Firefox. Try it yourself.

    In the other direction, for Bucket Brigade, I tune for high latency with the hope that it will be completely consistent:

      var audioCtx = new window.AudioContext({
        latencyHint: 'playback',
      });
      var micStream =
        await navigator.mediaDevices.getUserMedia({
          audio: {
             echoCancellation: false,
             noiseSuppression: true,
             autoGainControl: false,
          }});
    

    Setting high latency values on getUserMedia doesn't seem to have an effect in either Chrome or Firefox, so I've left it off.

    One of the most awkward aspects of Bucket Brigade today, which also would apply to anything else that lets you record multitrack audio in the browser, is that you have to measure latency. There is not yet any way to ask a browser how much end-to-end latency there is. There's input latency, MediaTrackSettings.latency, only implemented in Chrome, but in my testing it always returns 0.01 (10ms) even when I switch to a higher latency microphone. There's output latency, AudioContext.outputLatency, only implemented in Firefox, but in my testing it returns implausibly low numbers, less than half of the end-to-end latency.

    There are some situations in which measurement is going to be the best you can do: Bluetooth headsets are notorious for reporting much lower latencies than they actually implement. That is beyond the control of the browser, however, and giving developers the number the devices claim would still be a big improvement. Even if you can't completely trust the number, if it changes then you know you need to re-run measurement.

    Comment via: facebook, lesswrong

    Recent posts on blogs I like:

    Moral aesthetics

    “Doing good” differs by subculture The post Moral aesthetics appeared first on Otherwise.

    via Otherwise September 29, 2022

    Futurist prediction methods and accuracy

    I've been reading a lot of predictions from people who are looking to understand what problems humanity will face 10-50 years out (and sometimes longer) in order to work in areas that will be instrumental for the future and wondering how accurate thes…

    via Posts on September 12, 2022

    On the Beach

    I really like going in the water and this beach is a great place for building sand castles and boogie boarding. I also like trying to float on top of big waves. I'm not very good at it. I only float on the flat waves.

    via Anna Wise's Blog Posts July 12, 2022

    more     (via openring)


  • Posts
  • RSS
  • ◂◂RSS
  • Contact