networking - Realistic Network Load Testing

I am trying to benchmark an ASA under various conditions but what is throwing me off is my baseline seems to be odd. I am trying to load an ASA to full capacity. See the attached topology diagram:

The players are:

• C1 a Linux client runs a continuous download of a 300 GB file and loops this from S1, a Linux server running HTTPD.


• C2 a Linux client also runs a continuous download of a 300 GB file and loops this from S2, a Linux server running HTTPD.


• C3 runs AB to try and generate more connections. ab -n100 -c99999999 http://10.0.0.57/ This is to S3, a Linux server running HTTPD.


• Cisco ASA 5520 running 8.4.

What I found odd was that even with all this going on the max I saw was just over 500 Mbps (observed via NLOAD on both VM box physical interfaces). Is this normal? Everything is Gig. Some questions:

• Is it likely that my crappy Linux desk switch is bottlenecking?


• Does NATing really kill performance that bad or is something else going on? The CPU on the Dispatch Process was 30% under load.


• Is this is likely a disk issue as the servers are simply reading the file as fast as they can?


• What I found odd was that C1 would not transfer at it's full speed until I had it download 3 copies of the file from S1 at once (about 250 Mbps at this point). Why are 3 parallel downloads from S1 faster than a single download? Shouldn't S1 send as quick as it possibly can?

Is there a better way to load test network equipment. Downloading a single large file does not seem realistic. I am trying to simulate a busy network doing web things and load the ASA to capacity.