Each AMI instance on EC2 is assigned two IP addresses and corresponding DNS names. A public IP address that is accessable over the internet and an internal IP address only accessable inside internal EC2 regional network. You don’t have any control over the internal IP address and it is assigned randomly when you start the instance. For a public IP address you can assign an Elastic IP address to a running instance, elastic IP address is reserved and associated with your account and you pay for it when not in use. If you communicate between instances using public or elastic IP address even in the same region you pay regional data transfer rates(0.01$ per GB in/out).

There might be some scenarios where you might be tempted to use elastic IP address to communicate inside the same region e.g when your distributed system needs fixed ip addresses but you should carefully weigh the cons/pros. Not only are you paying for the traffic that would be free if you use internal IP address but also the performance will be lower. I ran some simple tests to find out more about this.

For the test I started two large instances in the same region using the same security group. And the results were quite interesting:

1) public or private dns name resolve to internal IP address inside EC2
2) There is a big hit in network latency between using internal and public IP address
3) Using traceroute shows that with public IP address network traffic goes through a lot more routers/hops

Here at Jana, we hope that Amazon will soon provide:

1) Internal static IP address so we don’t go through configuration hell and enjoy fast network communication
2) Machines without public IP/DNS address e.g for machines that will be used behind firewalls and will never be accessed outside EC2 network directly e.g Database or Application Servers

Test Details


- Machine A used to run ping and traceroute internal IP address: 10.250.79.223
- Machine B Machine associated to an elastic IP address:
- Internal dns name: ip-10-250-78-208.ec2.internal
- Public dns name: ec2-174-129-227-190.compute-1.amazonaws.com
- Internal ip: 10.250.78.208
- Elastic ip: 174.129.227.190

DNS Ping Tests

ip-10-250-79-223:~# ping ip-10-250-78-208.ec2.internal

PING ip-10-250-78-208.ec2.internal (10.250.78.208) 56(84) bytes of data.

64 bytes from ip-10-250-78-208.ec2.internal (10.250.78.208): icmp_seq=1 ttl=62 time=0.346 ms

64 bytes from ip-10-250-78-208.ec2.internal (10.250.78.208): icmp_seq=2 ttl=62 time=0.226 ms

64 bytes from ip-10-250-78-208.ec2.internal (10.250.78.208): icmp_seq=3 ttl=62 time=0.384 ms

64 bytes from ip-10-250-78-208.ec2.internal (10.250.78.208): icmp_seq=4 ttl=62 time=0.257 ms

64 bytes from ip-10-250-78-208.ec2.internal (10.250.78.208): icmp_seq=5 ttl=62 time=0.252 ms

— ip-10-250-78-208.ec2.internal ping statistics —

5 packets transmitted, 5 received, 0% packet loss, time 3999ms

rtt min/avg/max/mdev = 0.226/0.293/0.384/0.060 ms

ip-10-250-79-223:~# ping ec2-174-129-227-190.compute-1.amazonaws.com

PING ec2-174-129-227-190.compute-1.amazonaws.com (10.250.78.208) 56(84) bytes of data.

64 bytes from ip-10-250-78-208.ec2.internal (10.250.78.208): icmp_seq=1 ttl=62 time=6.52 ms
64 bytes from ip-10-250-78-208.ec2.internal (10.250.78.208): icmp_seq=2 ttl=62 time=0.262 ms
64 bytes from ip-10-250-78-208.ec2.internal (10.250.78.208): icmp_seq=3 ttl=62 time=0.329 ms
64 bytes from ip-10-250-78-208.ec2.internal (10.250.78.208): icmp_seq=4 ttl=62 time=0.359 ms
64 bytes from ip-10-250-78-208.ec2.internal (10.250.78.208): icmp_seq=5 ttl=62 time=0.327 ms
64 bytes from ip-10-250-78-208.ec2.internal (10.250.78.208): icmp_seq=6 ttl=62 time=0.367 ms
64 bytes from ip-10-250-78-208.ec2.internal (10.250.78.208): icmp_seq=7 ttl=62 time=1.63 ms

— ec2-174-129-227-190.compute-1.amazonaws.com ping statistics —

7 packets transmitted, 7 received, 0% packet loss, time 5999ms

rtt min/avg/max/mdev = 0.262/1.400/6.520/2.138 ms

The above two commands show the public and private dns resolve to internal ip address when pinging from another EC2 machine

Public/Private network Ping tests

ip-10-250-79-223:~# ping 10.250.78.208

PING 10.250.78.208 (10.250.78.208) 56(84) bytes of data.

64 bytes from 10.250.78.208: icmp_seq=1 ttl=62 time=7.93 ms

64 bytes from 10.250.78.208: icmp_seq=2 ttl=62 time=0.250 ms

64 bytes from 10.250.78.208: icmp_seq=3 ttl=62 time=0.244 ms

64 bytes from 10.250.78.208: icmp_seq=4 ttl=62 time=0.360 ms

64 bytes from 10.250.78.208: icmp_seq=5 ttl=62 time=0.311 ms
— 10.250.78.208 ping statistics —

5 packets transmitted, 5 received, 0% packet loss, time 4000ms

rtt min/avg/max/mdev = 0.244/1.820/7.938/3.059 ms

ip-10-250-79-223:~# ping 174.129.227.190

PING 174.129.227.190 (174.129.227.190) 56(84) bytes of data.

64 bytes from 174.129.227.190: icmp_seq=1 ttl=52 time=1.62 ms

64 bytes from 174.129.227.190: icmp_seq=2 ttl=52 time=1.50 ms

64 bytes from 174.129.227.190: icmp_seq=3 ttl=52 time=1.46 ms

64 bytes from 174.129.227.190: icmp_seq=4 ttl=52 time=1.52 ms

64 bytes from 174.129.227.190: icmp_seq=5 ttl=52 time=1.49 ms

64 bytes from 174.129.227.190: icmp_seq=6 ttl=52 time=1.37 ms

64 bytes from 174.129.227.190: icmp_seq=7 ttl=52 time=1.38 ms

— 174.129.227.190 ping statistics —

7 packets transmitted, 7 received, 0% packet loss, time 5997ms

rtt min/avg/max/mdev = 1.375/1.482/1.621/0.092 ms
  
The above two ping commands show the difference in ping performance to the same machine using public and private ip address.

TraceRoute Tests

ip-10-250-79-223:~# traceroute 10.250.78.208

traceroute to 10.250.78.208 (10.250.78.208), 30 hops max, 52 byte packets

1 ip-10-250-76-177 (10.250.76.177) 0.155 ms 0.070 ms 0.046 ms

2 ip-10-250-76-160 (10.250.76.160) 11.776 ms 0.092 ms 0.087 ms

3 ip-10-250-78-208 (10.250.78.208) 0.267 ms 0.160 ms 0.127 ms

ip-10-250-79-223:~# traceroute -m 100 174.129.227.190

traceroute to 174.129.227.190 (174.129.227.190), 100 hops max, 52 byte packets

1 ip-10-250-76-177 (10.250.76.177) 0.121 ms 0.208 ms 0.047 ms

2 ip-10-250-76-3 (10.250.76.3) 0.295 ms 0.208 ms 0.209 ms

3 ec2-75-101-160-114.compute-1.amazonaws.com (75.101.160.114) 0.243 ms 0.226 ms 0.221 ms

4 othr-216-182-224-19.usma1.compute.amazonaws.com (216.182.224.19) 0.677 ms 20.055 ms 0.631 ms

5 72.21.197.200 (72.21.197.200) 0.797 ms 0.673 ms 0.593 ms

6 othr-216-182-232-72.usma2.compute.amazonaws.com (216.182.232.72) 0.897 ms 0.860 ms 0.808 ms

7 72.21.197.201 (72.21.197.201) 0.679 ms 0.865 ms 0.850 ms

8 othr-216-182-232-102.usma2.compute.amazonaws.com (216.182.232.102) 1.084 ms 1.129 ms 0.988 ms

9 othr-216-182-224-18.usma1.compute.amazonaws.com (216.182.224.18) 1.353 ms 1.308 ms 1.472 ms

10 ec2-75-101-160-115.compute-1.amazonaws.com (75.101.160.115) 1.823 ms 1.455 ms 1.608 ms

11 198.19.63.211 (198.19.63.211) 1.299 ms 1.305 ms 1.241 ms

12 ec2-174-129-227-190.compute-1.amazonaws.com (174.129.227.190) 1.363 ms 1.519 ms 1.254 ms

ip-10-250-79-223:~#

Traceroute shows the traffic has to go through multiple hops when using public ip address, this also requires opening more ports.

Share and Enjoy:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Google
  • Live
  • MySpace
  • description
  • Technorati
  • TwitThis
  • Yahoo! Buzz

Tags: , , , , , ,