themonkeysdidit

Following on from the design issues in XMediaStream, the plan of implementing a queue to hold the SimpleMessageObjects received over multicast has been implemented.

This was done with three new classes, and a modification to MulticastShoutcastServer.java.

SimpleQueue.java

This class holds the queue implementation. In a bid for code re-use, the underlying data structure is a simple array of Objects. A read pointer and write pointer point to the next read and write position (wrapping back to zero when they reach array.length). When an object is read, that position in the array is set to null. Therefore whenever a write is taking place, the position in the array is checked to make sure it is null. If it isn’t null, the write pointer has caught up to the read pointer and we’re about to overwrite an object that hasn’t yet been read. If this happens, the queue will re-size and copy the unread objects into the new queue, setting the read and write pointers appropriately.

MulticastPacketListener.java

This is pretty simple, it continuously listens for new multicast packets and for each one receives, check’s it’s for this configured source and if so, add it to the SimpleQueue.

QueueShoutcastServerStreamer.java

This continuously reads from the queue and sends the mp3 bytes to the listening squeezebox.

The only problem encountered was if the squeezebox disconnected (or was started and stopped). Both MulticastPacketListener’s and QueueShoutcastServerStreamer’s are Threads, so if the squeezebox disconnected, the QueueShoutcastServerStreamer stopped reading from the queue but the MulticastPacketListener kept writing new packets to the queue. The queue therefore continuously re-sized until the JVM ran out of memory.

To get round this problem, the SimpleQueue keeps a timestamp every time an object is added to or read from the queue. The MulticastPacketListener then checks each time it writes a new object when the last read took place, if it was more than 30s ago, it deems the reading client is dead and ends.

This implementation has been running successfully for a couple of days now with no glitches in the music, no dropped multicast packets and only a few queue resizes.

I finally decided to take the plunge and test whether or not XMediaStream  is dropping packets or not. Every so often when listening to the shoutcast stream, there’s a glitch in the music so I was pretty confident something wasn’t quite right.

First thing was to prove whether or not packets were getting dropped. This was quite simple, every SimpleMessageObject sent as a multicast packet now has a sequence number added to it. The receiver of these packets then checks the sequence number to make sure it is what it is expecting and spits a warning out to the log if it misses one.

I connected a listener to the feed over night and checked the log file, low and behold, a packet was getting dropped about once every two and a half minutes. At this point I was faced with 4 options:

1. Do nothing, one dropped packet every 2 minutes 30 seconds isn’t so bad and results in only a slight audio glitch.
2. Implement a recovery mechanism to allow the recovery of missed packets.
3. Find out where in the network the packet was being dropped and fix it.
4. Do something else.

After a while, the audio glitches are a bit annoying and it’s not a particularly intelligent solution so that was 1 out of the window. A recovery mechanism would mean the receiver would have to request a particular packet, that’s OK, we now have a sequence number we can use. But the sender has no packet history so implementing a recovery mechanism would involve storing history in some sort of data structure. This sounds like a fair amount of work so I won’t be doing 2. Both sender and receiver run on the same server so there’s no routers/switches etc in the way that could be dropping packets so no point doing anything for 3. Interestingly, this also means that we must be filling a Solaris TCP/IP buffer somewhere…

This leaves 4 (did you see that coming?). The way the MulticastShoutcastServerStreamer works (that’s the object that the squeezebox connects to and receives the shoutcast stream from) is that it grabs a SimpleMessageObject off the multicast stream (blocking), extracts the raw mp3 data and then sends it to the squeezebox using the OutputStream from the Socket. Once this write() completes, it then grabs the next SimpleMessageObject from the multicast stream. Here’s where the problem lies. I think the socket writes can block if the send buffer is full, meaning the receiving multicast buffer is filling up while we’re waiting to write to the squeezebox.

As a quick side note, there is some flow control here. The multicast sender sends data pretty much at the bit rate of the mp3, so it won’t swamp the network with packets. However, this simple method isn’t quite enough.

My current way of thinking is to split the MulticastShoutcastServerStreamer further. We’ll grab a load of the multicast SimpleMessageObject off the network and write them to an internal queue. The shoutcast streaming part will then read these from the queue and send the data to the squeeze box. So as long as our queue is big enough, we shouldn’t drop any more packets…

An overview of some home grown software called XMediaStream has been added to the site. This is currently powering the shoutcast stream my squeezebox plays. It sounds simple but there’s some clever play list adjustment based on who is in the house at any one time in there that I’m actually quite proud of.

More technical details will appear over time…