security - How to find an unauthorized person's entry point to a server

itemprop="text">

We're running an in house
dedicated server with 100's of sites on it. Some of these sites were very insecure and
the box was not managed very well. Some one got in, messed up a bunch of stuff and is
pestering us by defacing some of the sites, asking for money
etc.

We made a lot of fixes but there is still
some sort of backdoor through which he can get in. We found and deleted a few files that
were a kind of hacker control panel but since the file was just called contactUs.php,
it's impossible to know that all instances of these kinds of files were
removed.

I realise this is a very broad subject
and a question that may be difficult to answer but what steps would one undertake to
find out how this person is getting in to the
system?

It's a Fedora machine running mainly PHP
sites.

Answer

Honestly, if you're running a business I
wouldn't try to study the attacker, I'd just lock them out and move on. It's just not
worth the time, hassle, and frustration of getting into a battle of wits with someone
like this. Some ideas for getting rid of
them:

• Reinstall
  the box, from scratch, using known-good media.


• Before you
  start putting any sites on it, make sure it's completely patched and up to date. If
  you're running an out-of-date Fedora release (one that isn't actively receiving security
  updates) then upgrade to the latest release (or use something with proper security
  support).


• Don't run PHP scripts via
  mod_php, because it's child's play to exploit the inevitable
  permissions problems that result from having to give the webserver write access to
  various parts of the system. Use suexec or suphp.


• Run PHP
  scripts as a separate user from the main "FTP account" (or equivalent), so if you have a
  user example (for the site
  example.com) then run their dynamic content in a user called
  example-cgi (or equivalent). Then only grant the
  example-cgi user write access to those parts of the filesystem
  tree that require them. (This requires modifying suexec's permissions checking, but it's
  worth it). This way the attacker can't run scripts that modify the site's PHP code, and
  they have to work much harder to inject an exploit.


• Seek
  and destroy all world-writable files on a proactive, on-going
  basis.


• Restrict FTP and SSH connections from known
  locations, rather than allowing the entire Internet to
  connect.


• Enforce strong passwords using something like
  PAM's cracklib module -- or, better still, get rid of FTP entirely and require the use
  of SSH keys to get into the
  machine.

Probably
a pile of other things I can't think of at the moment, but that'll keep you going for a
while.