Hardening RedHat (CentOS) Linux for use on Cloud

If you next to deploy Linux on Cloud you should consider hardening the Linux instance prior to any deployment. Below are guidelines we have pulled together with regards to hardening a RedHat or CentOS instance.

Hardening Redhat linux guidelines

enable selinux

Ensure that /etc/selinux/config includes the following lines:
SELINUX=enforcing
SELINUXTYPE=targeted

Run the following on commandline to allow httpd to create outbound network connections
setsebool httpd_can_network_connect=1

check using
sestatus
To enable/disable
echo 1 >/selinux/enforce

disable the services

chkconfig anacron off
chkconfig autofs off
chkconfig avahi-daemon off
chkconfig gpm off
chkconfig haldaemon off
chkconfig mcstrans off
chkconfig mdmonitor off
chkconfig messagebus off
chkconfig readahead_early
chkconfig readahead_early off
chkconfig readahead_later off
chkconfig xfs off

Disable SUID and SGID Binaries

chmod -s /bin/ping6
chmod -s /usr/bin/chfn
chmod -s /usr/bin/chsh
chmod -s /usr/bin/chage
chmod -s /usr/bin/wall
chmod -s /usr/bin/rcp
chmod -s /usr/bin/rlogin
chmod -s /usr/bin/rsh
chmod -s /usr/bin/write

Set Kernel parameters

At boot, the system reads and applies a set of kernel parameters from /etc/sysctl.conf. Add the following lines to that file to prevent certain kinds of attacks:

net.ipv4.conf.all.rp_filter=1
net.ipv4.conf.all.accept_source_route=0
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.icmp_ignore_bogus_error_messages=1
kernel.exec-shield=1
kernel.randomize_va_space=1

Disable IPv6

Unless your policy or network configuration requires it, disable IPv6. To do so, prevent the kernel module from loading by adding the following line to /etc/modprobe.conf:
install ipv6 /bin/true
Next, add or change the following lines in /etc/sysconfig/network:
NETWORKING_IPV6=no
IPV6INIT=no

Nessus PCI Scan

Upgrade openssh to latest version

upgrade bash to latest version

http://www.thecpaneladmin.com/upgrading-openssh-on-centos-5/

Set HTTP headers off

In /etc/httpd/conf/httpd.conf set the following values
ServerTokens Prod
ServerSignature Off
TraceEnable off

In /etc/php.ini set
expose_php = Off

Change MySQL to listens on only localhost

Edit /etc/my.cnf and add following to mysqld section
bind-address = 127.0.0.1

Make sure only port 80 443 21 are open

vi /etc/sysconfig/iptables
and add
ACCEPT tcp anywhere anywhere state NEW tcp dpt:https
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:ftp

Gawker news sites cloud security breach

If you did not notice the Gawker set of news sites recently has it’s online security compromised. You may not have heard of Gawker but you will probably know of the set of news sites they encompass which includes Gizmodo, Lifehacker, Kotaku, io9 or Jezebel. Over 1.3 million passwords where stolen and uploaded as a 500MB torrent file. Also posted where Gawker’s source code and internal employee conversations. The disclosure of this authentication information led to a viral effect with increased spam attacks, for example, on Twitter being attributed to the breach. Many users use the same web password everywhere so such a breach could leave them exposed on every site where they use the same username and password.

Apparently the passwords where encrypted in the torrent but as Gawker used an outdated encryption scheme they are relatively straightforward to crack. If you have ever registered on any of these sites then and tend to use the same username and password then you should change your username and password anywhere else you have used it on the web. Some sites are already pro-actively forcing you to do this. I receive an email from LinkedIN today that made me go through the lost password security mechanism to reset my account.

So what does this mean for Cloud ? Can one site damage the concept of storing and accessing information on the Cloud ? I think for sure, yes. It will make companies who were reticent about going to Cloud because of security concerns even more reticent, and such a breach has an effect on other sites, and I am sure we have not seen the full fallout of this yet. As for Gawker’s brand, well I think it is hugely damaging, although the web can be a fickle place, it remains to be seen how badly affected the Gawker brand will be. I can imagine potential advertisers do not want to be associated with it.

What can you do to protect yourself ? Well first, for sure change any username/password combos that are the same as the one you registered on this site, and in future consider having a separate username/password combination for each site you register. I create email addresses specifically for a registration for such sites on the web and I file them in KeepPass to be able to remember them. Ulitmately, remember, as a user don’t rely that such sites will protect your data, and as a vendor, revisit your security mechanisms to ensure the next Gawker is not you !

Amazon S3, EC2 and VPC ISO 27001 certified

As well as being SAS 70 Type II-certified Amazon is now ISO 27001 certified. ISO/IEC 27001 formally outlines a management system that brings information security under management control, and mandates requirements that have to be met. Organisations that have adopted ISO/IEC 27001 may be formally audited to maintain compliance with the standard.

As stated on WikiPedia:

SO/IEC 27001 requires that management:

Systematically examine the organization’s information security risks, taking account of the threats, vulnerabilities and impacts;

Design and implement a coherent and comprehensive suite of information security controls and/or other forms of risk treatment (such as risk avoidance or risk transfer) to address those risks that are deemed unacceptable; and

Adopt an overarching management process to ensure that the information security controls continue to meet the organization’s information security needs on an ongoing basis.

“Amazon Web Services is continuing its commitment to provide further assurance of AWS security controls and practices through third-party audits and certifications such as SAS 70 Type II and ISO 27001,” said Stephen Schmidt, Chief Information Security Officer for Amazon Web Services.

“Via ISO 27001 and other certifications, we continue to provide our customers with confidence that our security controls and practices follow internationally-recognized security standards.”

You can learn more about Amazon and it’s compliance and security provisions here.

System hardening guidelines for Amazon EC2

One of the biggest questions we get from Clients is “Is Amazon EC2 secure” . That is like saying is my Vanilla network secure. Like anything you can take some steps to make the environment as secure as you can, such as:

– First read the Amazon Security Whitepaper and the Amazon discussion of Security processes

– Ensure the system key is encrypted at start-up

– Ensure you plan for load balancing in case an instance goes down. Ensure you understand all the security implications of this and harden any other instances.

– Test or emulate the performance of applications deployed to the cloud in all geographies where you plan to deploy them. The latency could vary greatly for each.

– Never ever allow password base authentication for shell access.

– Encrypt all network traffic always.

– Always encrypt everything stored on S3

– Encrypt file systems for Block devices

– Open only the minimum required ports

– Include no authentication information in any AMI images

– Think about how your system can be hardened and what is out there such as SELinux, PAX,  ExecShield etc

– Don’t allows any decryption keys into the cloud – understand the perils of keys and security

– Install host based intrusion detection system such as OSSEC

– Regularly backup Amazon instances and store them securely. 

– Use Security Groups. With EC2 security groups, you can completely isolate every tier, even internally to the EC2 cloud.

– Design in a way you can issue security patches to AMI instances

The nightmare scenario that you cannot cater for is is that Xen has unforeseen security issues which would allow inter-VM communication and which in essence would enable instance spying. Amazons doomsday scenario…..

Securing n-tier and distributed applications on EC2

In this post I will walk you through the  high level  of securing a normal tiered application running on EC2. First I will cover the basics of what EC2 provides and then briefly discuss how this can be used in a real life scenario.

Security Groups

For Network security EC2 provides a security groups, security groups are essentially inbound firewalls  suited to the dynamic nature of EC2.  Using security groups you can specify which incoming network traffic should be delivered to your instance.

  • The default mode is to deny access, you have to explicitly open ports to allow for inbound network traffic
  • If no security group is specified a special default group is assigned to the instance. This group allows all network traffic from other members of this group and discards traffic from other IP addresses and groups. You can change settings for this group
  • You can assign multiple security groups to an AMI instance.
  • The security groups for an instance are set at launch time and can not be changed. You can dynamically modify the rules in a security group and the new rules are automatically enforced for all running and future instance, there may be a small delay depending on the number of instances
  • You can control access either from  named security groups or source IP address range. You can specify the protocol(TCP, UDP, or ICMP) , individual ports or port range to open
  • You can allow access to other users security groups using user-group pair
  • The current API (Amazon EC2 on 2008-12-17) does not support port ranges for security group using command line tools or Query API, you will need to use SOAP API
  • An account can have a maximum of 100 security groups
  • Security groups are just access rules applied to a single or collection of instances, if two instances are part of the same security group this does not afford them any special access between them.
  • An instance running in promiscuous can not sniff any traffic intended for a different instance.
  • A running instance cannot change security group access rules. You need access keys or X 509 key to authorize change.
  • In the instance you can get the security group information from the instance meta-data (curl http://169.254.169.254/1.0/meta-data/security-groups)

Key Pair

Amazon discourages the use of passwords and the normal way to access an instance is using ssh and a private key. Amazon EC2 provides facilities to generate the key(2048 bit RSA key), at instance startup you can attach the key name to the instance and this will allow root access. Normally you will customize the AMI with your own less privileged user public keys and disable root login

Securing Your Application

Now that we have covered the basics how can we use these to secure a distributed application. Below is the normal deployment architecture for a typical tiered application.
ec2

In the above deployment we have created 4 security groups

Web-Security group: Allows http (80) and https(443) to everyone to access the application

App security group: Only allows access from instances running in web security group on required ports e.g. 8080

DB security group : Only allows access from instances running in app security group on required ports e.g. 3306

ssh-admin security group: Only allows access to ssh port 22 and as a matter of policy access is allowed from specific host address or organization network. This allows easy management of permissions.

As you can start an instance with multiple security groups the web tier instances will run with web and ssh-admin security groups, app server instances with app and ssh-admin and finally database instances with db and ssh-admin.

You will not need to change web, app or db security groups, The cloud administrator will allow or revoke admin access by  just adding or removing hosts from ssh-admin group with port 22 access. You can write scripts or use any GUI (Elasticfox, Amazon admin console) tool

Other Best practices

  • Make secure requests to Amazon Web Services see
  • Restrict ssh port(22) access to  host or organization network
  • You can and are encouraged by amazon to use an other firewall (e.g iptables) in conjunction with security groups  on an instance to restrict inbound/outbound traffic and have finer control
  • Dont open any port unnecessarily
  • Have separate application administrator (ssh access to instances) and cloud administrator(setting up security groups and key-pair generation with access to amazon EC2 certificate and access keys but no ssh access to running instances)
  • Disable password based login( set PasswordAuthentication no in /etc/ssh/sshd_config) see
  • Customize the AMI with your own user public keys and disable root login. If you need root login use sudo see
  • Keep your AMI up-to-date with security patches and fixes