« General life-like update | Main | Book reviews »
July 22, 2005
Loadbalanced media servers
One of my more interesting projects over the past few months was an "upgrade" of our media streaming servers.
The configuration is a shitty Dell 100mbit switch hooked via a gigE uplink to CIS's network. On that switch there are 2 media servers, 2 load balancers, a webserver, and a NetApp fileserver.
The media servers were originally RedHat 7.something boxes, with the Real Helix server on them. I wiped one box and moved it to Slackware (I also shot a man in Reno just to watch him die.) The loadbalancers were originally running RedHat and the shittiest virtual software in existence . I notice they ported that crap to FreeBSD, and I have no idea why anyone in their right might would cripple themselves that way.
Anyway, I moved the loadbalancers to FreeBSD. They run CARP between them, and the Helix servers are set to use the CARP virtual address as their default gateway (yes, this works. No, it isn't something you are normally taught to do.) The FreeBSD boxen also run pf, and have a round robin rdr rule to balance media traffic between the linux Helix servers. I wrote up a script to remove a server from the pool if it pings out automatically, it's available here for anyone interested. It's not particularly complicated, but the design is rather robust. I was able to test it by rebooting the loadbalancer boxes with minimal service disruption, something that was often problematic under Linux.
Posted by jeff at July 22, 2005 12:19 AM