• Posts
  • RSS
  • ◂◂RSS
  • Contact

  • Browser Audio Latency

    February 2nd, 2021
    audio, bucket_brigade, tech  [html]
    Any time you are working with digital audio, you have to handle latency: how long does it take from when the audio enters the system until it leaves again? Special purpose professional-level audio hardware generally has been optimized for extremely low latency, often adding as little as 2ms. Extreme low latency, however, has costs: scheduling must be very predictable, power consumption is higher, audio processing algorithms can look at fewer samples, and you have much less recovery time before causing an audible glitch.

    Web browsers are in an interesting position, where they have to work with whatever audio hardware the computer happens to have, and need to handle a very wide range of use cases.

    I've recently been working on two musical projects where I care a lot about latency, but in different ways:

    • Bass Whistle JS: a bass synthesizer that you control by whistling. To feel natural and responsive, it needs the lowest possible latency.

    • Bucket Brigade: a platform for making music with others over the internet. It doesn't need low latency, but it does need to know precisely how much there is so it can compensate.

    But how much latency is there? We can measure it: send a beep to the speaker, check how much time passes until we detect the beep in the samples coming from the microphone (example worklet). On my 2016 15" MacBook running OSX 11.1, if I do:

      var audioCtx = new window.AudioContext();
      var micStream =
        await navigator.mediaDevices.getUserMedia(
          {audio: true});
    
    I measure ~67ms end-to-end latency in Chrome, and ~55ms in Firefox. Instead of using default settings, though, we could tune for low latency:
      var audioCtx = new window.AudioContext({
        latencyHint: 0,
      });
      var micStream =
        await navigator.mediaDevices.getUserMedia({
          audio: {
             latency: 0,
             echoCancellation: false,
             noiseSuppression: false,
             autoGainControl: false,
          }});
    

    Note that we're asking AudioContext for an output latency of '0' and not interactive. While MDN documents interactive as "try to use the lowest possible and reliable latency it can", '0' is even lower. We are also turning off all of the smart processing options, which would add latency.

    This gives me ~19ms in Chrome and ~14ms in Firefox. Try it yourself.

    In the other direction, for Bucket Brigade, I tune for high latency with the hope that it will be completely consistent:

      var audioCtx = new window.AudioContext({
        latencyHint: 'playback',
      });
      var micStream =
        await navigator.mediaDevices.getUserMedia({
          audio: {
             echoCancellation: false,
             noiseSuppression: true,
             autoGainControl: false,
          }});
    

    Setting high latency values on getUserMedia doesn't seem to have an effect in either Chrome or Firefox, so I've left it off.

    One of the most awkward aspects of Bucket Brigade today, which also would apply to anything else that lets you record multitrack audio in the browser, is that you have to measure latency. There is not yet any way to ask a browser how much end-to-end latency there is. There's input latency, MediaTrackSettings.latency, only implemented in Chrome, but in my testing it always returns 0.01 (10ms) even when I switch to a higher latency microphone. There's output latency, AudioContext.outputLatency, only implemented in Firefox, but in my testing it returns implausibly low numbers, less than half of the end-to-end latency.

    There are some situations in which measurement is going to be the best you can do: Bluetooth headsets are notorious for reporting much lower latencies than they actually implement. That is beyond the control of the browser, however, and giving developers the number the devices claim would still be a big improvement. Even if you can't completely trust the number, if it changes then you know you need to re-run measurement.

    Comment via: facebook, lesswrong

    Recent posts on blogs I like:

    How to extend pockets

    Make women's pants pockets big enough to hold a phone properly The post How to extend pockets appeared first on Otherwise.

    via Otherwise May 19, 2022

    Buckingham Palace

    I love England. Especially because of the big castle called Buckingham Palace. I got to see the outside there, but my mom showed me some pictures of the inside. I love it there. But the outside doesn't look very fancy to me. But I never knew why those …

    via Anna Wise's Blog Posts April 25, 2022

    What is causality to an evidential decision theorist?

    (Subsumed by: Timeless Decision Theory, EDT=CDT) People sometimes object to evidential decision theory by saying: “It seems like the distinction between correlation and causation is really important to making good decisions in practice. So how can a theor…

    via The sideways view April 17, 2022

    more     (via openring)


  • Posts
  • RSS
  • ◂◂RSS
  • Contact