In the bucket brigade singing project I talked about yesterday, the server is Python. We wrote it in a very straightforward "it won't be fast but at least it's correct" style. In simple stress testing, I found that it could handle about 10 clients, which is definitely not enough.

My first thought was to switch it to C with libmicrohttpd, and for a program which is just a web interface to a circular buffer C isn't terrible. But how far can we get if we do the hard stuff in Numpy?

In the initial version there were four parts that did work in proportion to either the input or the whole buffer:

• Clearing the buffer: this is a circular buffer, and as we wrap around we want to be following the lead singer. We should not hear things from the last time around. So we need to zero out the buffer ahead of the lead person.

• Incoming audio: we receive a large block of samples from the network, and need to sum them with what's already there at the right part of the circular buffer.

• Outgoing audio: we need to send a section of the buffer out to the client to play.

• Computing metadata: We draw a debugging chart at the bottom of the page, and it needs to know what parts of the queue have audio data. We send information about which frames (128 samples) are non-zero.

If this were a large project, I would start with profiling, but this was small enough that I was pretty sure we were just going to want to convert everything to an efficient implementation. David and I pair-programmed for a couple hours and turned this naive code into this numpy-using code.

The refactor brought a single request from 218ms to 74ms. This was ok, but still not great. The problem was that computing metadata was really very slow. The other three operations are in proportion to the size of a request, but the metadata one was in proportion to the size of the queue. And worse, it involved a Python loop over 128-sample frames.

Since the metadata was only used to populate a debugging chart, I measured the effect of removing it. In the pure Python version this brought us from 218ms to 23ms, while with Numpy it brought us from 74ms to 0.06ms. I won't credit Numpy with the effect of removing the chart, but 23 ms to 0.06 ms is still an excellent 383x speedup.

I doubt this is enough to move us from 10 clients to 3830 clients, but it helps enough that I expect encoding and decoding Opus to be the major factor, once we get that added.