General

SSHD rootkit, cPanel affected

As a follow-up to the previous post, it has now been confirmed that there is an SSHD rootkit in the wild that spreads itself if you ssh from an infected server to another. The details are explained in this article as a summary of the webhostingtalk thread:
http://isc.sans.edu/diary/SSHD+rootkit+in+the+wild/15229
Unfortunately, cPanel is a victim of this malware and their workstation(s) have been infected leading to possible infection of client servers:

Salutations,
You are receiving this email because you have opened a ticket with our support staff in the last 6 months. cPanel, Inc. has discovered that one of the servers we utilize in the technical support department has been compromised. While we do not know if your machine is affected, you should change your root level password if you are not already using ssh keys. If you are using an unprivileged account with “sudo” or “su” for root logins, we recommend you change the account password. Even if you are using ssh keys we still recommend rotating keys on a regular basis.
As we do not know the exact nature of this compromise we are asking for customers to take immediate action on their own servers. cPanel's security team is continuing to investigate the nature of this security issue.
–cPanel Security Team

We would recommend (as we do on our helpdesk) that you change your root passwords immediately after any third-party has accessed your server. Additionally, to those affected, please remember that you cannot “clean” this infection, you must reinstall your OS and restore user accounts from backups.

New csf v5.79

Changes:
– Modified csf error routine to store failing error in csf.error and display an instructional message
– Check for libkeyutils-1.2.so.2 in LF_EXPLOIT option SSHDSPAM
– Modified the Server Report proxysubdomains check on cPanel servers
– Added new options CC_DENY_PORTS, CC_DENY_PORTS_TCP, CC_DENY_PORTS_UDP. This feature denies access from the countries listed in CC_DENY_PORTS to listed TCP/UDP ports. For example, using this FTP access port 21 could be blocked to only the specified countries

New csf v5.78

Changes:
– Due to issues that some are experiencing with the switch from the state to the conntrack module a new settings has been added USE_CONNTRACK which is disabled by default except on servers running kernel 3.7+ where on new installations it will be enabled

New csf v5.77

Changes:
– Add an exception for the useless Virtuozzo kernels iptables implementation so that csf uses the deprecated state module instead of conntrack

New csf v5.76

Changes:
– Only add the /128 IPv6 bound address per NIC instead of the whole /64 to the local IPv6 addresses
– Modify SSHD and SU regexes to allow for empty hostname field in log file
– Added new option UNBLOCK_REPORT. This option will run an external script when a temporary block is unblocked
– Additional entries in csf.logignore on new installations
– Switched from using the iptables state module to using the conntrack module in preparation of the formers obsolescence
– Removed LF_EXPLOIT_CHECK and replaced it with LF_EXPLOIT_IGNORE so that new tests can be easily added and then ignored desired
– Added new LF_EXPLOIT check SSHDSPAM to check for the existence of /lib64/libkeyutils.so.1.9 or /lib/libkeyutils.so.1.9, See:

Time to check if you have suffered a root compromise

There's a quickly spreading root compromise that everyone should check for that latches onto the sshd daemon. See the following threads for details on detecting the compromise:
http://forums.cpanel.net/f185/sshd-rootkit-323962.html
http://www.webhostingtalk.com/showthread.php?t=1235797
At the very least check for the existense of libkeyutils.so.1.9
As with all root compromises, simply deleting it and carrying on is not an option. If your server has been compromised you most likely cannot trust it and will need to perform an OS reinstall and restore from backups. However, unless you fix the original method of compromise, the server may simply be exploited again.
On a maybe related note, though not proven, it appears that there's a scary kernel exploit about which RedHat should have fixed soon (CentOS and CloudLinux are likely to follow quickly afterwards). So, make sure that your kernel is kept up to date at all times and look out for a new one soon:
https://access.redhat.com/security/cve/CVE-2013-0871

New csf v5.75

Changes:
– Fixed issue with single quotes appearing in CC lookup names leading to lfd IP blocks to fail

New csf v5.74

Changes:
– Additional entries in csf.pignore for the cPanel installation to cater for v11.36 processes on new installations
– Added workaround for cPanel /etc/cpupdate.conf check in Server Report for changes in v11.36
– Additional entries in csf.logignore on new installations
– Try harder to get a CPU temperature if lm_sensors is installed for System Statistics
– Enforce PORTFLOOD setting restrictions and issue warning if entry discarded
– Correct location of CC_ALLOWF in LOCALINPUT after update from lfd
– Make CC_[chain] actions more verbose in lfd.log
– Added new options CC_ALLOW_PORTS, CC_ALLOW_PORTS_TCP, CC_ALLOW_PORTS_UDP. This feature allows access from the countries listed in CC_ALLOW_PORTS to listed TCP/UDP ports. For example, using this FTP access port 21 could be restricted to only the specified countries
– Moved temporary and csf.allow/csf.deny rules from LOCALINPUT/LOCALOUTPUT chains to ALLOWIN/ALLOWOUT to allow for the new CC_ALLOW_PORTS feature
– Modified SMTP_PORTS to include ports 465 and 587 on new installations
– Added new option PT_FORKBOMB. Fork Bomb Protection. This option checks the number of processes with the same session id and if greater than the value set, the whole session tree is terminated and an alert sent

New cxs v2.86

Changes:
– Improvements to installer on initial fresh cPanel v11.36 installations
– Added a 20 second timeout for running –Wsymlink [script] and switched from using system call to open3
– Added a 20 second timeout for running –script [script] and improve output printing from [script]
– Modified –options [u] to include more suspicious locations
– Exploit fingerprint definitions database additions