Possible site breach

SpindleyQ's picture

I'm not certain, but I think the server running Glorious Trainwrecks may have been hacked again recently.

IMMEDIATE ACTIONS YOU SHOULD TAKE:
- You should NOT YET change your password on this site, as if it has been hacked, the attacker could still have access.
- You SHOULD change your password on any websites that you're using that share a password with Glorious Trainwrecks.

WHAT I AM DOING TO FIX THIS:
- I am going to create a new, clean Linode instance and migrate the site to use it.
- I am going to modify Drupal to use a more secure password hashing mechanism.
- I will tell you when this is done so that you can change your passwords here, if you wish.

WHAT, EXACTLY, IS GOING ON:
(Warning: gory technical details)
After the last time the site was hacked, I installed a program called rkhunter, which runs daily and searches for changes to the running system that have security implications.

On the morning of March 17th, rkhunter started to send emails that flagged something I had never seen before. This message has persisted since. There is no possible way I accidentally did anything to the server in the 24 hours prior, as on March 16th I was travelling all day.

[blockquote]Warning: Network TCP port 47018 is being used by /usr/lib/libice.log. Possible rootkit: Possible Universal Rootkit (URK) component
Use the 'lsof -i' or 'netstat -an' command to check this.[/blockquote]

According to ls, /usr/lib/libice.log does not exist. Neither 'lsof' nor 'netstat' show anything that has that port open. I can't connect to that port, either locally, or from an external machine. And rkhunter's mailing lists suggest that URK is very old and very unlikely to be seen in the wild.

It is possible that this is a weird false positive. If it doesn't go away after rebooting the server, then that seems very unlikely, since this is very dubious behaviour. I am being cautious.

One other thing happened that I find alarming: This morning, someone logged into my Facebook account from Taiwan. Facebook caught it, locked my account, and notified me. I had been under the belief that the password I was using was A) secure, and B) had yet to be leaked. Clearly it is now out there and associated with my email address, so I have a long slog of password changes ahead.

RE: password hashing, Drupal 5 performs by default an SHA-1 hash of your password with no salt. If you have a common password, this means that someone with access to our database could figure out your password literally by putting your password hash into Google. If you have an uncommon password, the work is slightly more complicated, but because there is no salt, it does not take an attacker much more time.

(I would love to do away with storing passwords entirely and switch to Mozilla Persona or something, but the chances of making that work with Drupal 5 are basically nil. I should be able to fairly easily switch to a bcrypted version of the SHA-1 hash of your password, which won't require a password reset and will bring us up to modern password hashing standards.)

Anyway, I'm sorry that this happened, and I'm hopeful that I can get us transferred over to a fresh, new, more secure server reasonably quickly.

Smedis2's picture

Oh, lovely. First the

Oh, lovely. First the millions of Spambots now this. :/

SpindleyQ's picture

AN UPDATE

I rebooted the machine last night. This morning, the mysterious rkhunter message is gone. Now, there is a new warning, that port 6667 is being used by /usr/bin/irssi, and that this is a possible rogue IRC bot. This is, in fact, my IRC client that I have been running 24 hours a day for years.

rkhunter is seeming a bit janky to me right now.

Still, a clean start is long overdue. I'm going to proceed with the server migration and security upgrade as planned. And personally, I'm going to start gradually switching to KeePass + DropBox + randomly generated passwords for everything, because even if nobody got hold of Glorious Trainwrecks' password database specifically, there is definitely some bot in Taiwan who has at least one of my passwords from SOMEwhere.

FlaviusMaximus's picture

...

I hope everything turns out ok