Every once in a while one of the LUG lists that I am on decides to have a big dick contest and everyone shows off who has the better uptime. Its actually a way of generating traffic on mailing lists that have greatly suffered any sort of reasonable traffic in a while. Its the same as starting a vi vs emacs flame war. No one really wins, but it always gets people contributing to the list.
The list started off fairly respectable with a few people putting up some pathetic numbers of 15 days or 30 days. Someone threw out a 100+ and then someone managed to get a 400+. I decided it was time to put these folks to shame. I threw down the following:
matt@goober:~> uptime
03:30:14 up 1041 days, 10:41, 2 users, load average: 0.00, 0.00, 0.00
This is not my record, I have had a DNS server here at work at 1500+.
As usual, these email threads will bring out the security conscious folks who believe that if you haven’t updated your kernel, you’re a bad admin. I typically argue that if you are relying on only security patches, you’re probably not a good admin. It should always be a layered approach! Some people use k-splice on their boxes so they can patch the kernel effectively without a reboot. I don’t in this instance.
So, as always, I had to describe some of the security features of this box that would convince these people that I’m not just an ignorant ass. Here is how I do it.
First off, when I build a box, I install the bare minimum. This server happens to be a file server. It doesn’t need to serve up web pages so apache is not installed. It does need to send mail out from its local queue for monit messages so the smtp server is locked down by a firewall running on the server as well as the SMTP configuration is set to only listen on localhost. As I stated, the server has a local iptables FW running on it. It is also protected by a hardware firewall. I keep the various services the server provides up to date and patched up fully. I load minimal modules in the kernel. It is the bare bones of what it needs. In the instance where I can unmount, patch and remount the module, I do it. Other than that, I can ignore this box.
Do I do this for every server that I admin, no. At my current job we have the resources where just about every service that I role out is built in some sort of a high availability cluster. If I need to take down a node to patch it, I can. This server that I haven’t taken down in 2.5 years doesn’t have that luxury. Its also in California. So if I had a kernel update go south on me, I have to walk a buddy through getting it back up…over the phone. My buddy has some skills on the computer, but not necessarily a lot of Linux skills. So imagine walking your mom through the Linux command line. Scared yet?
I’m not trying to promote not patching your kernel. But what I am trying to promote is in some instances, maybe even in a lot of instances, you can get by with _not_ updating your kernel every time a new one comes out. Its about cost/benefit analysis. Not many sheople realize that it is actually a part of their job. Sometimes you don’t have $20K to put in a solution that will never go down. EVER! Sometimes you have a couple hundred bucks and you need to provide something that is stable, secure, and hassle free. And that’s what I have provided, going on 1042 days =)
