 |
GENERAL PRINCIPLESGeneral principles are exceedingly simple. Rootkits rely on inserting a layer of indirection between system usage and the users and administrators. The layer mimics standard system behavior while implementing new functionality.The Root of the Problem. This brings up the topic of architecture in modern UNIX systems. We won't delve into a complete discussion of system security. Interestingly, UNIX has an effective method of separating dangerous and non-dangerous types of system behavior. The real problem that we face in computing is that the root user is exempt from all safeguards. The concept of a single system account that has no restrictions associated with it seems counter to the notion of security that was associated with UNIX system design in the beginning. There are many reasons why most systems are insecure upon initial install, but the actual design of the operating system is solid. The weak link in the chain is a lack of secure coding guidelines by programmers and the single system account that has no safeguards associated with it. In the course of buffer overflow attacks or merely misconfigured systems, an intruder can gain root access due to human error. It should be noted that there is more than enough blame to go around. Many vendors, commercial and free alike, release code with buffer overflows. Most operating system vendors ship their system with applications that are vulnerable to attack enabled by default. Most vendors do not enforce safe implementation guidelines at runtime. Plenty of blame. This paper, however, is not about blame. This paper is about dealing with the reality that any system can potentially be breeched, and then the intruder will need to be ferreted out of the system. To approach this issue with the right frame of mind, entertain the following hypothesis.
Imagine if there was not a single system account that had no restrictions associated with it. If certain permissions were required to open network sockets (required for outgoing attacks and setting up backdoor access) but that user did not have the ability to manipulate special files on the filesystem. If an entirely different set of permissions were required to edit files marked as special in someway, files created at system install time, for example, then the intruder would have an additional hurdle to overcome to hide his presence effectively. In this particular case, the intruder could establish a method of entry into the system, but it would not be a trivial matter to keep the entry method covert. In a layered structure of this type, intrusions and a complete backdoor solution would not be impossible, but would raise the level of effort required to gain complete access. Currently, the intruder is faced with the challenge of:
Fairly simple. No wonder there are so many security incidents. If the procedure to administer resources on a server were broken into different modules the intruder would then have to subvert additional levels of authorization or access. That in and of itself would raise the level of effort required for the intruder. However, that effort would become even more complicated due to the fact there is no one all-powerful system account that would enable to the intruder to burrow down into the logical layers of the server. Each service that the intruder would want to breech, would have to be breeched independently. In the current model, the services to breech are access to the host and access to a single system account. Once that single system account is compromised, little further effort is required in gaining access and keeping it. A more complete resource administration solution would require completely separate permissions to:
In order to grant the relevant permissions, seperate system accounts would need to be used. If there was a single system account capable of granting these permissions, then an intruder could break into that account and merely give himself all possible perimssions. However, a happy compromise may to allow a single system account to grant these perimssions, but log all activity associated with that user and those permissions to a facility that cannot be accessed by any account but one, a special system audit user. The facilities associated with the system audit user could not granted to any other user, without exception.
Back to the PresentThe brief discussion above is not necessarily meant as a treatise on security architecture or as a suggestion to future engineers, but to impress upon the reader the extent of the problem with root user and the associated lack of controls.The original idea was to have a single user that could fix problems that were associated with misconfigured or malfunctioning system controls. The idea is valid, but the approach has had unintended consequences as server class systems proliferate beyond the lab. Now that we have discussed how easy it is (philosophically) to insert a layer of indirection, once the intruder gets root access, we shall discuss what the intruder does with it. This is generally accomplished by creating utilities that look and act as standard system utilities, with additional code to achieve whatever aim the intruder wishes. Usually, this means providing a covert means of manipulating the server. To put this into practice, we need to discuss what tools are typically used which could potentially lead to the discovery of an intruder, and hence, what tools are likely to be subverted by a rootkit. These tools include:
The ps command includes fields for running time for a given process and the amount of CPU time taken by the process. Any processes taking a disproportionatly large amount of CPU time will become immediately suspect. Therefore, hiding attack nodes from ps, and related utilities is vital to prevent discovery when attack programs run for long periods of time. In the event that the ps or top commands do not reveal any leads for the system administrator, the next step may be to take a look at open network connections on the server to determine if a misbehaving service is causing a blockage of system resources. In this case, the netstat program is used to determine active network connections. However, these connections also include any backdoors the intruder has left in order to gain access quickly and easily in the future. Therefore, altering the output of netstat is necessary to safegaurd access to the compromised server. The actual backdoor into the compromised can take many shapes. The normal system access and authentication mechanisms may be subverted by merely replacing the normal mechanisms (such as /bin/login) with look-alikes which continue to provide normal system services, but also provide undocumented functionality when certain criteria are met. This criteria may include incoming connections from certain networks (again, we need to alter netstat to prevent detection of this particular connection) or merely a special keyword or syntax provided as an argument. For example, the /bin/login replacement in the lrk (Linux rootkit) package allows root to login merely by using the special account "rewt" and whatever password was defined during build time. The cancerserver rootkit allows root to login by specifying a particular terminal type in the user environment when connecting from a remote server. If a system suddenly becomes low on diskspace on a filesystems that is normally static in size, the inquisitiveness of a systems administrator may be aroused. In this case, the system administrator may navigate around the filesystem using the usual comamnds such as "ls" and using standard utilities to such as "du" to determine where diskspace has suddenly gone. In the case of missappropriation of data, the sensitive data may need to be collated or centralized before transmission. This is especially true when an intruder is intercepting passwords, credit card numbers or even social security numbers over the network from the compromised system. The logs containing this data will grow in accordance with network utilization. Potentially, the logs may grow fast enough to trigger disk space alarms or even fill up available disk space causing normal operations to fail. Therefore, the ls, du and associated utilities must also be replaced by a rootkit to avoid detection. For example, the t0rnkit, a widely used rootkit, typically creates a directory called .puta in the /usr/info directory. However, with the trojanized ls command, the system administrator will never see the .puta directory when he looks for it with ls. Apart from the evading detection, most rootkits also come tools to allow quick reentry, and surveillance. TRENDS IN ROOTKIT DEVELOPMENTThere is a new threat to the attentive system administrator.Of primary concern when looking for rootkits and trojans is a new ability called binary redirection. Using binary redirection, the rootkit has the ability to intercept calls to a valid binary on disk and redirect those calls, completely transparently, to another binary located elsewhere on disk, or theoritically, even on a remote server. The primary threat here to the system administrator is that neither a visual inspection (via strings [see below]) or a file integrity check will reveal the presense of the subversion, with certain exceptions. In the case of knark version 0.59, the key component is called ered and relies upon the knark kernel module being loaded. The ered utility is very easy to use. Merely call ered with two arguments, the first being the binary which you wish to subvert, the second being your replacement. For example:
./ered /bin/login /usr/lib/login.bogus At this point, any time /bin/login is called, the /usr/lib/login.bogus file will actually be executed. Any amount of inspection on the /bin/login file will reveal no signs of intrusion. However, ered and knark are not foolproof. In lab conditions, running fuser on the legitimate binary which is supposed to be executed, but has been subverted, does not show as a running process. In other words, if you run /bin/login manually, it will never show in the process table. This has been replicated using "ps" and "fuser." For example, redirecting the command /usr/bin/yes to a harmless shell script called redirect.sh which simply prints "Hello World" on standard out will will yield the following results:
# ered /usr/bin/yes /usr/src/redirect.sh # /usr/bin/yes Hello World # If you are playing along at home, you may notice that the binary which is running in place of the legitimate binary shows up in the process table. In the above example, while the user executed the yes command, the redirect.sh shell script will appear in the process table. Of course, this could be hidden by the intruder using standard practices, but it is yet another detail to keep track of on the part of the intruder; again raising the level of effort on the part of the rootkit author and the intruder. All of this means that without additional tools it is still possible to determine if redirection is being employed on a system without approval of the system administrator. If a system administrator becomes suspicious, all he has to do is run any given utility by hand, and watch for it in the process table. If the utility never shows up, or the running file does not reflect it's properties correctly via fuser or lsof, then the system administrator has a lead. This is not a perfect solution, and the system administrators are always recommended to run file integrity checkers from read-only media and on an alternate kernel on read-only media, without the ability to turn on loadable kernel modules. There are situations where manual checks are preferable for short periods. Many systems in a corporate or high use environment are considered mission critical, where any downtime is considered detrimental. Unfortunately, this means booting an alternate kernel and running a file integrity checker is problematic, especially since the process can take some time on a large server. In these cases, a balance has to be struck between periodic rebooting and checking the filesystem and manual checks. We discuss rootkit detection by use of data integrity checkers and manually in the section DETECTION. AVAILABLE ROOTKITSCurrently available rootkits include:NO EXPERIENCE REQUIREDFor the most part, the rootkits available on the net are easily compiled and installed with the traditional "make" and "make install" Coding skills become important only when the rootkit in question does not compile on a specific host or platform. Traditional rootkits are easily and come with adequate documentation. Inexperienced crackers will slip-up, however. They sometimes leave the tar package of the rootkit lying around not giving any thought to the fact that the rootkit will not make the tar package invisible. AVAILABITY OF ROOTKITSJust like any other piece of software, rootkits are merely a search engine engine away. For example, searching for "linux rootkit" turned up:
And the list goes on. Of course, many of the hits were merely references for people asking for information because they were infected by a rootkit, and other wishing to learn about them even though they have not infected any machines themselves. Still, even that information is a resource for the bad guys just as for good guys. Some of the hits included in those searches actually contained software archives of rootkits. Additionally, the cracker underground circulates rootkits and software archives among themselves via irc, email, and non-archived web sites. All that is required to get the more widespread rootkits is a little bit of time, and a little persistence is all that is required for more specialized or non-public rootkits. PREVENTIONWe will not discuss how to prevent a system from being compromised, but we will discuss steps a system administrator can take to mitigate damage during the break-in process and prevent the rootkit from being successfully deployed.
Kernel ModulesSome operating systems support kernel modules. The notion of a kernel module is quite useful. The kernel loads device drivers of various types only when they are needed and are unloaded when not in use. This results in a less complicated and more efficient kernel. However, as we have seen with binary redirection, kernel modules can also be dangerous.To avoid this particular vulnerability, disable loadable kernel modules on the system. This is usually done at kernel compile-time.
Immutable FilesSome operating systems support immutable files. By setting the immutable attribute on a file, system administrators can prevent unsophisticated intruders from subverting any given file. The chattr(8) man page on setting the immutable flag says:
A file with the `i' attribute cannot be modified: it can
not be deleted or renamed, no link can be created to this
file and no data can be written to the file. Only the
superuser can set or clear this attribute.
If an intruder gains root access, they can simply remove the attribute.
However, most attacks are scripted and many script attacks will
not check the immutable attribute, thus the script will fail.
Read-Only FilesystemsSome filesystems can be mounted read-only with significant trouble. The /bin, /sbin, /usr/bin, /usr/sbin, and /lib directories can be mounted read-only. The /etc filesystem does contain files that need to be modified at run-time such as /etc/mtab. An example /etc/fstab file may resemble: Device Mountpoint FStype Options Dump Pass# /dev/da0s1b none swap sw 0 0 /dev/da0s1a / ufs rw 1 1 /dev/da0s1f /usr ufs ro 2 2 /dev/da0s1c /sbin ufs ro 2 2 /dev/da0s1d /lib ufs ro 2 2 /dev/da0s1e /var ufs rw 2 2 /dev/da0s2e /usr/ports ufs rw 2 2 /dev/da0s3e /usr/local/ ufs rw DETECTIONAs we have discussed, most contemporary rootkits merely replace the standard utilities with special versions. Rootkit installations often falsify timestamp and filesize information to defeat the system administrators from visually confirming the integrity of the utilities via the ls command.However, systems administrators have several additional utilities at their disposal. These are: MANUAL INSPECTIONTypically, a manual inspection is carried out using the strings command. The strings utility is standard on all modern UNIX platforms. Its purpose is merely to display the human readable (ASCII) portions of a binary file. Fortunately for the experienced systems administrator, this human readable data includes the names of files where intruder passwords are kept, library versions the trojan was compiled with, and other information which does not normally correlate with the original data in the target file.
DATA INTEGRITYIn addition to visual inspection, there are various approaches to taking snapshots of key files on the system and calculating speical checksums to come up with a specific signature for a file that cannot be falsified without great effort. It is outside of the scope of this paper to discuss each file integrity tool in detail. Rather we will discuss the general approach. There are several factors to consider: Most data integrity checkers keep their information in a database (possibly little more than a text file). If the database is stored on the same machine that is compromised, the signatures which are used to verify the integrity of your data is also compromised. While there are no known rootkits which automatically seek out and alter the signatures in one of these databases, it is not difficult to conceptualize. Therefore, it is important to keep the signatures where they will not be tampered with. Generally, it is best to take the signatures and burn them onto a CD-ROM or on other read-only media for storage. Simply the load CD-ROM whenever the system needs to be checked. With the advent of binary redirection, it is also important that the integrity checker be run from a clean kernel. This means creating bootdisks or a bootable CD-ROM and running the entire check from the read-only media. This also means shutting down the system for general use, which can introduce disruption into high availability environments. While booting from a clean kernel is strongly recommended, a site may be able to reach an adequate compromise by running the integrity checker from a read-only on the running kernel periodically, perhaps once an evening. Then the server can be checked using a clean kernel (and a re-boot) once a month. In the end, downtime from periodic security sweeps will be less damaging than downtime from a successful compromise. The single most well known implementation is tripwire. While tripwire is effective, there are also other approaches would should be used in conjunction with each other. Of particular interest are: AIDE is similar to tripwire. It does not differ radically from tripwire in function. More information on AIDE can be found at: AIDE.
RPM is the Redhat Package Manager. File integrity is small part of what RPM is meant for and is not particularly flexible in the same manner as tripwire or AIDE. RPM should not be relied upon as the sole file integrity checker for a system, but in a pinch it can be quite useful. In order to use RPM to verify the integrity of an RPM package, the syntax is: rpm -V [package name] If a file within the package fails the integrity test, rpm
will report thusly: More information on RPM can be found at: RPM.org spfDB is the Solaris Fingerprint database. Most files that ship in Solaris media are cataloged in the SPF. For files which should not change on demand (/bin/login as opposed to /etc/motd) this includes all versions of /bin/login that shipped on Solaris media, including all patched versions. This methodology will alert the system administrator when the given files are updated by patches that did not originate from a Sun certified source. The downside to the SPF is the less than elegant interface via an HTTP connection. More information on spfDB can be found at: spfDB.
NETWORK DETECTIONIt is also possible to detect rootkits remotely. There are two approaches: In active detection, a port mapper or security scanner is run against your own hosts to look for anything abnormal. The ability of a rootkit to guard against this type of detection is minimal, hence this is an important approach for security organizations to consider. The criteria to look for in scanning a network for rootkit installations would be:
Some freely available tools to conduct sweeps are:
Every organization should have a security policy which stipulates that security scans be performed periodicially. Only by checking the system from the inside, on the console, and from the outside, via a scan from the network, can an organization be reasonably sure that a server is still clean. An example of a network scan detecting unauthorized backdoors would include the following output of nmap: Starting nmap V. 2.54BETA4 ( www.insecure.org/nmap/ ) Interesting ports on testbox (xx.xx.xx.xx): (The 65535 ports scanned but not shown below are in state: closed) Port State Service Protocol Version 21/tcp open ftp FTP 6.00LS 22/tcp open ssh SSH 1.99-2.4.0 SSH Secure Shell (non-commercial) 23/tcp open telnet 25/tcp filtered smtp 79/tcp open finger 80/tcp open http HTTP Apache/1.3.14 (Unix) 111/tcp open sunrpc RPC 513/tcp open login 514/tcp open shell 515/tcp open printer 1241/tcp open msg 3001/tcp open nessusd 3306/tcp open mysql 5432/tcp open postgres PostgreSQL 6000/tcp open X11 6112/tcp open dtspc 30299/tcp open ssh SSH 1.99-2.4.0 SSH Secure Shell (non-commercial)It is unusual, to say the least, to have a daemon like SSH running on a high port number Passive detection involves running a network monitor to listen for network traffic which commonly indicates a rootkit install. These items would include:
IRC (Internet Relay Chat) is a favorite means of communication for crackers, and also a favorite toy for some crackers to create bots (autonomous agents) which harass valid IRC channels with seemingly random IRC traffic (spam or inflammatory language, for example). UDP (User Datagram Protocol) is a popular method of remotely controlling attack agents. UDP traffic is more easily spoofed than TCP (Transmission Control Protocol) and does not involve the same level of accountability as TCP connections. It is easier to hide UDP connections in the noise of an active network. ICMP (Internet Control Message Protocol) is often used to disguise remote control channels to rootkits and attack agents as normal network diagnostic traffic. Fortunately, it is fairly easy to pick out unusual ICMP traffic on a network compared to UDP or TCP traffic. CAUTIONOnce a rootkit is detected a proactive security organization may wish to track down the culprit. Once the rootkit is detected, the gig is up as they say. That does not mean the intruder is going to give-up, however. Some intruders include logic bombs with their rootkits. Upon detection, a rootkit may go into self-destruct mode. It may actually be as innocent as deleting the rootkit itself and not affecting the rest of the server, or it may be as serious as destroying the entire filesystem and the boot block. The trigger to self-destruct can be anything. The self-destruct code could potentially be activated by:
Not all intruders are experts. It is just as feasible that a cracker damages the system during the intrusion, hence causing unintended damage. Make sure you have clean backups handy.
CONCLUSIONWe have discussed why you may find rootkits on your servers, how easy it is to find rootkits, and the varying levels of risk across different rootkits.
BIBLIOGRAPHY |