I am looking for recommendations for hardware specs for a server that needs to be a web cache for a user population of about 2,000 concurrent connections. The clients are viewing segmented HTTP video in bitrates ranging from 150kbps to 2mbps. Most video is "live" meaning segments of 2-10 secs each, of which 100 or so are maintained at a time. There are also some pre-recorded fixed length videos. How would I go about doing the provisioning calculation for such a server: What kind of HDD (SSD?), how many NICs how much RAM etc? I am thinking of using Varnish on Linux, all the RAM I can get my hands on, 2 CPUs with 6-8 cores each.
Answer
Will Varnish be able to share objects across sessions? In other words, is your architecture such that the object being loaded by the video stream client is /somestream/1h42m0s-1h42m10s/, as opposed to /somestream/for/joeuser? In that case, based on what you're describing, I'd skip the SSDs and just go for a ton of RAM; Varnish doesn't really need disk storage for anything when configured with a malloc pool. Remember to turn down swappiness. If, on the other hand, Varnish is just passing every request along to the backend, then at best all you're getting out of it is some SYN flood protection.
I've seen Varnish happily push multiple gigabits out the door, but make sure everything above Varnish knows how to handle that kind of traffic, too. Some firewalls (names redacted to protect the guilty) can barely handle 100 Mbit/s, let alone the kind of traffic (4Gbit/s) you're talking about if every one of those connections is saturating its link at 2Mbit/s. For a NIC, even an onboard 10Gbit port will probably do you just fine; if you need to, you can also do silly channel-bonding stuff with GigE, but why bother? Just get a 10G port (or three) and call it a day.
For the server itself, you don't need very much CPU at all, but max out the RAM. Stick a couple fast disks in there if you'd like, but the only reason they're there is for bootup. Varnish is a very, very slim server. You could go for multi-level caching and try to offload videos onto the HDD once they rotate out of RAM, but why? They're already cached to disk on your content servers; it's not like the disks on the Varnish server are somehow magically better. Spend that money on yet more RAM. Seriously, you shouldn't bat an eye at 64 or 96 GB given the kinds of bandwidth you're talking about.
In unrelated advice, if you're using disk at any point on those streaming servers, you should absolutely go for SSDs (or even more exotic options like FusionIO). For streaming applications where you're jumping all over the disk as people are streaming different parts of the same file, SSDs are not just a little better; they are, effectively, your only rational choice.
Comments
Post a Comment