themonkeysdidit

I finally decided to take the plunge and test whether or not XMediaStream  is dropping packets or not. Every so often when listening to the shoutcast stream, there’s a glitch in the music so I was pretty confident something wasn’t quite right.

First thing was to prove whether or not packets were getting dropped. This was quite simple, every SimpleMessageObject sent as a multicast packet now has a sequence number added to it. The receiver of these packets then checks the sequence number to make sure it is what it is expecting and spits a warning out to the log if it misses one.

I connected a listener to the feed over night and checked the log file, low and behold, a packet was getting dropped about once every two and a half minutes. At this point I was faced with 4 options:

1. Do nothing, one dropped packet every 2 minutes 30 seconds isn’t so bad and results in only a slight audio glitch.
2. Implement a recovery mechanism to allow the recovery of missed packets.
3. Find out where in the network the packet was being dropped and fix it.
4. Do something else.

After a while, the audio glitches are a bit annoying and it’s not a particularly intelligent solution so that was 1 out of the window. A recovery mechanism would mean the receiver would have to request a particular packet, that’s OK, we now have a sequence number we can use. But the sender has no packet history so implementing a recovery mechanism would involve storing history in some sort of data structure. This sounds like a fair amount of work so I won’t be doing 2. Both sender and receiver run on the same server so there’s no routers/switches etc in the way that could be dropping packets so no point doing anything for 3. Interestingly, this also means that we must be filling a Solaris TCP/IP buffer somewhere…

This leaves 4 (did you see that coming?). The way the MulticastShoutcastServerStreamer works (that’s the object that the squeezebox connects to and receives the shoutcast stream from) is that it grabs a SimpleMessageObject off the multicast stream (blocking), extracts the raw mp3 data and then sends it to the squeezebox using the OutputStream from the Socket. Once this write() completes, it then grabs the next SimpleMessageObject from the multicast stream. Here’s where the problem lies. I think the socket writes can block if the send buffer is full, meaning the receiving multicast buffer is filling up while we’re waiting to write to the squeezebox.

As a quick side note, there is some flow control here. The multicast sender sends data pretty much at the bit rate of the mp3, so it won’t swamp the network with packets. However, this simple method isn’t quite enough.

My current way of thinking is to split the MulticastShoutcastServerStreamer further. We’ll grab a load of the multicast SimpleMessageObject off the network and write them to an internal queue. The shoutcast streaming part will then read these from the queue and send the data to the squeeze box. So as long as our queue is big enough, we shouldn’t drop any more packets…