When a distributed application is running in-house the IT has a lot of control over the environment. They know exactly about the hardware and resources available:
- network bandwith available
- network latency
Now move to EC2 and other than some vague figures there is very little documented.
The guys at RighScale did some tests for ‘EC2-EC2′ bandwidth and ‘EC2-S3′ bandwidth which is very informative. To test ‘EC2-EC2′ large instance bandwidth they used apache(non-SSL) and curl(one instance acting as a server the other as a client)
“Using 1 single curl file retrieval, we were able to get around 75MB/s consistently. And adding additional curls uncovered even more network bandwidth, reaching close to 100MB/s”
They also got quite good numbers between an EC2 instance and S3 using curl to download and upload from/to S3.
|1 Curl (MB/S)||Max(MB/S)|
|Download SSL||12.6||49.8 (8 curls)|
|Download non SSL||10.2||51.5 (8 curls)|
|Upload SSL||6.9||53.8 (12 curls)|
So in both EC2-EC2 and EC2-S3 the bandwidth is quite reasonable for a general purpose application and you can get 1 Gigabit between 2 EC2 instances.
Now what about network latency? A friend of mine sent me a link to an interesting paper which contains some interesing latency information. The paper is “Benchmarking Amazon EC2 for high-performance scientific computing” . To test the latency they used mpptest to measure both network performance and latency. The results compared to infinband are an order of magnitude inferior!