security - What are main steps doing forensic analysis of linux box after it was hacked?

itemprop="text">

What are main steps doing forensic
analysis of linux box after it was hacked?

Lets
say it is a generic linux server mail/web/database/ftp/ssh/samba. And it started sending
spam, scanning other systems.. How to start searching for ways hack was done and who is
responsible?

itemprop="text">

class="normal">Answer

Here are some things to try before
rebooting:

First of all, if you
think you might be compromised unplug your network cable so
the machine can't do further damage.

Then, if
possible refrain from rebooting, as many traces of an
intruder can be removed by re-booting.

If you
thought ahead, and had remote logging in place, use your
remote logs, not the ones on the machine, as it's all too easy for someone to tamper
with the logs on the machine. But if you don't have remote logs, examine the local ones
thoroughly.

Check
dmesg, as this will be replaced upon reboot as
well.

In linux it is possible to have
running programs - even after the running file has been deleted. Check for these with
the command file /proc/[0-9]*/exe|grep "(deleted)". (these
disappear on reboot, of course). If you want to save a copy of the running program to
disk, use /bin/dd if=/proc/filename/exe
of=filename

If
you have known good copies of who/ps/ls/netstat, use these
tools to examine what is going on on the box. Note that if a
rootkit has been installed, these utilities are usually
replaced with copies that won't give accurate information.