[IMC-Tech] System Compromise Report

[Zachary C. Miller]

  Bottom line: We do not believe that your account has been
  compromised but it would be a good idea for you to change your
  password (and if you use SSH PKA, to change your keypair, if you
  don't know what SSH PKA is then don't worry about this) at this
  time. Remember that your password must not contain words that are in
  any dictionary in any language, attackers frequently use dictionary
  attacks to gain access to servers.

  Here's what CERT has to say about good passwords: 
  http://www.cert.org/tech_tips/unix_configuration_guidelines.html#A1

    A good [method] for choosing a password is to choose an
    easy-to-remember phrase, such as "By The Dawn's Early Light", and
    use the first letters to form a password. Add some punctuation or
    mix case letters as well. For the phrase above, one example
    password might be: bt}DeL{. (DO NOT use this sample phrase for
    your password.)

  Other Bottom Line: Please do not install phpmyadmin on your website
  as it compromises security for the whole system. phpmyadmin wasn't
  used in today's attack but today's attack made me aware of the
  security problem with users installing this software. There is an
  officially system supported secure install of phpmyadmin that you
  can use, please see the end of this email for details.

Here is the technical summary of today's compromise: 

Today the chambana.net server was compromised and the attacker briefly
acquired root level access. Arun Bhalla and I have been working on
this for the past 6 hours and think we have finished damage control,
reversed all changes made by the attacker, and locked out the holes
the attacker used to gain access. Big thanks to Arun for being the
first responder on this and ensuring that the attacker only had access
for a brief period of time today.

The attacker compromised the machine of another user (abhalla)
(probably via an SSH security hole, but possibly also via a sniffer on
another compromised host) and used information gained from that
compromise to log in to our server. Armed with the user password for
abhalla and the fact that abhalla is a sudoer, the attacker gained
root. With root the attacker installed a trojan version of the SSH
binary which logs passwords, a packet sniffer that steals POP
passwords, and a logger which monitors HTTP transactions for
passwords.

The only users whose passwords were logged by these loggers were
jeychaner and sascha. I contacted those users immediately after the
compromise and they changed their passwords.

I detected the trojan ssh binary by checking the MD5 checksums of all
system binaries against Debian's database of checksums. No other
debian system binaries were altered. The /usr/bin/ssh trojan was
modified with the chattr command so that it couldn't be deleted
without first clearing it's attributes. If you want to check your own
Linux system for a trojan like this you can run "lsattr /usr/bin/ssh"
and see if any attribute flags are set. This one had the s, i, and a
attributes set.

The trojan also put a bunch of log files and sniffer binaries files in
/usr/lib/named and /etc/cron.daily/dnsquery and /usr/lib/test and
/usr/lib/+c0d.init. These have all been cleaned up. 

The attacker came from benjamin-sisko.cs.upb.de. I have banned that IP
from the server. The attacker attempted to send sniffed passwords
(already changed) to ovi_magic at yahoo.com.au and soferel at yahoo.com. I
have reported this incident to abuse at yahoo.com (though I doubt they'll
do anything about it).

Besides the (now changed) passwords for user accounts jeychaner and
sascha (the only users to use POP during the compromise window), the
attacker also gained knowledge of a phpMyAdmin install on the
books2prisoners website and the auth information for the
books2prisoners MySQL database. I do not believe that knowledge was
actually used. I have disabled that phpMyAdmin install so the auth
information for the MySQL database (which can only be accessed via
localhost) should be useless to the attacker.

There is an officially supported install of phpmyadmin that is
accessible only from localhost. The way to access this remotely is to
open an ssh tunnel and assign it to a local port like this:

First run "ssh -L 8080:localhost:80 imsahp.chambana.net" on your local
workstation.

Then (while the SSH session is open) fire up your web browser and
access: "http://localhost:8080/phpmyadmin/"

As far as your webbrowser is concerned it is accessing port 8080 on
your local machine but ssh is quietly forwarding the requests over an
encrypted tunnel to localhost on imsahp.chambana.net.

Similarly you can run POP more securely by running "ssh -L
110:localhost:110 imsahp.chambana.net" on your local workstation and
configuring your POP client to access "localhost" (or "127.0.0.1")
instead of "mail.chambana.net". This will prevent your password from
going cleartext over the wire.

After reading through many log files and analyzing many files on the
system, looking for open ports that shouldn't be open, and looking for
running processes that shouldn't be running, I have concluded that
this incident is now fully resolved, the attacker locked out, the
damage fixed, and the holes plugged. I encourage everyone to change
their passwords by SSH'ing in to imsahp.chambana.net and running the
"passwd" command.

-- 
Zachary C. Miller - @= - http://zach.chambana.net/
IMSA 1995 - UIUC 2000 - Just Another Leftist Muppet - Ya Basta!
 Social Justice, Community, Nonviolence, Decentralization, Feminism,
 Sustainability, Responsibility, Diversity, Democracy, Ecology