22 December 2010

hitting the ground running

I've mentioned creating and injecting a root ssh key into a new instance. The images we run are deployed with ssh enabled and not TCP wrappered; we COULD wrapper them and use the 'add an exception for ssh from a single IP' tool which the PMman web control interface has, but we have found the support load fallout from people just getting started is too high. Once they have deployed and hardened a couple of boxes, they 'get it and can use the 'lock from all' web tool, and then add a single IP if their taste runs to web tools

The very next step I take as to each machine I administer, is to run a hardening script. While I have published an outline here, I use a script rather than reading and scrape and pasting from that outline. This step is done through a script, not because I think I WILL forget something, but because I know the script will NOT forget anything, and is written to perform the hardening process in an idempotent fashion -- that is, when done, finishing the same end result, time after time. One path to get to better host security is to have good processes, consciously designed, systematically applied, and continuously improved

herrold@centos-5 admin]$ ./hardening.sh hostname.pmman.net
The authenticity of host 'hostname.pmman.net (198.178.231.xyz)' can't be established.
RSA key fingerprint is 86:6e:84:e0:27:57:dd:4d:1f:88:82:fc:42:1d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hostname.pmman.net,198.178.231.xyz' (RSA) to the list of known hosts.
hosts.allow 100% 488 0.5KB/s 00:00
hosts.deny 100% 390 0.4KB/s 00:00
iptables 100% 1337 1.3KB/s 00:00
sshd_config 100% 3325 3.3KB/s 00:00
README 100% 897 0.9KB/s 00:00
rollup.pem 100% 0 0.0KB/s 00:00
openssl.cnf 100% 9682 9.5KB/s 00:00
arm-pmman.sh 100% 363 0.4KB/s 00:00
sa-update-local-NOTES 100% 877 0.9KB/s 00:00
sa-update-local 100% 117 0.1KB/s 00:00
logwatch.conf 100% 80 0.1KB/s 00:00
rollup.pem 100% 0 0.0KB/s 00:00
Package sendmail-8.13.8-8.el5.x86_64 already installed and latest version
Package 1:make-3.81-3.el5.x86_64 already installed and latest version
Package m4-1.4.15-2orc.x86_64 already installed and latest version
Package iputils-20020927-46.el5.x86_64 already installed and latest version
logwatch.conf 100% 80 0.1KB/s 00:00
Stopping crond: cannot stop crond: crond is not running.[FAILED]
Starting crond: [ OK ]
Shutting down sendmail: [FAILED]
Starting sendmail: [ OK ]
Starting sm-client: [ OK ]
Flushing firewall rules: [ OK ]
Setting chains to policy ACCEPT: filter [ OK ]
Unloading iptables modules: [ OK ]
Applying iptables firewall rules: [ OK ]
Loading additional iptables modules: ip_conntrack_netbios_ns [ OK ]
Stopping auditd: [FAILED]
Starting auditd: [ OK ]
/etc/aliases: 76 aliases, longest 10 bytes, 765 bytes total

info: inspecting /etc/aliases for a root email forwarder off the box
# Person who should get root's mail
#root: marc

info: 1. do you want fail2ban -- if so, run: ./fix-fail2ban.sh hostname.pmman.net

info: 2. updates are not run by this script: consider running:
ssh -l root hostname.pmman.net yum -y -q upgrade --enablerepo=pmman-mail

info: 3. verify that root's email is properly handled

info: 4. now: ssh -l root hostname.pmman.net
cd /root/hardening/

and do some patching and service restarting ...
[herrold@centos-5 admin]$

Note: the IP is obscured, and the host name and ssh host key altered. The edit to add an opff-box alias entry for root's email is to centralize all the miscellaneous cron and asynchronous notifications off the box, to centrally monitored point

Then as noted before, this is a stock CentOS 5 image, and so needs some further tightening done and updates run. I have long since scripted that process:

[herrold@centos-5 admin]$ ./fix-fail2ban.sh hostname.pmman.net
local-fb-fix.sh 100% 256 0.3KB/s 00:00
Stopping fail2ban: [FAILED]
Starting fail2ban: [ OK ]
[herrold@centos-5 admin]$ ssh -l root hostname.pmman.net yum -y -q upgrade --enablerepo=pmman-mail
[herrold@centos-5 admin]$

Then, as suggested, the edits on the remote machine

[root@vm175551137 hardening]# netstat -pant
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 3641/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 2627/sendmail: acce
tcp 0 240 198.178.231.xyz:22 76.242.0.abc:41936 ESTABLISHED 3593/0
[root@vm175551137 hardening]# history
1 cd /etc/
2 joe aliases
3 yum install ipsec-tools
4 rm *~
5 newaliases
6 cd /root/hardening/
7 ls
8 joe /etc/ssh/sshd_config
9 /sbin/service sshd restart
10 ls
11 cp hosts.allow hosts.deny /etc
12 joe iptables /etc/sysconfig/iptables
13 /sbin/service iptables restart
14 netstat -pant
15 history
[root@vm175551137 hardening]# reboot

Broadcast message from root (pts/0) (Wed Dec 22 14:47:10 2010):

The system is going down for reboot NOW!
[root@vm175551137 hardening]# Connection to hostname.pmman.net closed by remote host.
Connection to hostname.pmman.net closed.
[herrold@centos-5 admin]$

I added the ipsec-tools in support of an application this particular unit will be performing. The reboot at the end is for good measure to apply any new kernel and libraries through a clean boot. The 'true' hostname will be picked up from DNS PTR records, once that has been done already, after a reboot, as well