Update on Korean Denial of Service Attempt
At 17:51 today, UK Grid confirmed that both UK Grid and Tiscali have removed their blocks on traffic between the attacking host and bunny.shell.
Faelix extends their thanks to UK Grid for their swift response and support in dealing with the matter, and helping us isolate the problem swiftly and effectively.
Analysis of Incident
Following incident response we inspected the snapshot of the host taken immediately prior to its shutdown. Our initial suspicions of an unpatched libssl or sshd were unfounded: an attacker had actually compromised a customer account with an extremely weak password (username has been changed to protect the guilty):
Jan 15 16:10:13 bunny sshd[22467]: (pam_unix) authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=host-static-92-114-xxx-yyy.moldtelecom.md user=haX0r Jan 15 16:10:13 bunny sshd[22467]: Accepted password for haX0r from 92.114.xxx.yyy port 1715 ssh2
Earlier log entries suggest that this was a brute-force scanner that got very lucky -- most of our hosts are protected by fail2ban, but bunny.shell wasn't for "customer convenience". This policy has since been changed.
Having gained unprivileged access, the miscreant downloaded a copy of PsyBNC:
# ls var/tmp/.../psybnc/ CHANGES config.h motd/ src/ COPYING help/ psybnc.conf sshd FAQ lang/ psybnc.conf.old targets.mak Makefile log/ psybnc.pid tools/ README makefile.out psybncchk SCRIPTING makesalt salt.h TODO menuconf/ scripts/
They then proceeded to run a "fake" sshd process (username has been changed to protect the guilty):
haX0r 22476 0.0 0.5 2840 1320 ? S Jan15 0:01 ./sshd
This process listened on TCP port 31337 (presenting a PsyBNC header):
Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:31337 0.0.0.0:* LISTEN 22476/sshd tcp6 0 0 :::22 :::* LISTEN 1145/sshd
Inspection of checksums suggests that no other files were modified, confirming our suspicion that no elevation of privileges took place. The attacker's files were all stored in one of the few places the user account could write: /var/tmp.
Faelix suggests the theory that the PsyBNCer attracted a little too much attention; that retaliation was swift in the form of a UDP denial of service attack.
Actions Taken
- compromised host has been snapshotted and integrity tested
- insecure password has been fixed
- customer has been educated
- other customers have been notified of the problem
- fail2ban deployed on bunny.shell
- bunny.shell has been brought back online
- blocks at UK Grid and Tiscali have been lifted
Follow-on Actions
- password strength enforcement for bunny.shell
- appropriate abuse addresses to be notified