Securing Java

Previous Page
Previous Page
The Base Java Security Model: The Original Applet Sandbox
CHAPTER SECTIONS: 1 / 2 / 3 / 4 / 5 / 6 / 7 / 8 / 9 / 10 / 11 / 12 / 13

Section 1 -- Potential Threats

Next Page
Next Page

Java applets are far more powerful than the usual HTML code served up on the Web. When not restricted by applet-security measures, Java is a complete and powerful programming language capable of sending information over the network; reading, altering, or deleting files; using system resources; and so on. This is powerful stuff, and in the hands of a malicious programmer (or even just a sloppy or incompetent programmer), Java code could do some damage to a user's system. People surfing the Web should not be burdened with such worries (although developers and system administrators unfortunately must be). So from a user's perspective, Java should put these worries to rest by providing an automatic security solution. Java should restrict itself such that the full power and potential of the Java language is not misused. After all, who wants to run a Java applet that erases your hard disk? And who wants to develop or distribute an applet that accidentally erases someone else's hard disk?

The design problem lies in the fact that programs running on a personal computer usually have unlimited access to all of the machine's resources. (That's what we mean when we say that ActiveX controls are omnipotent.) Most PC applications are total system tyrants while they run. But if the Java applets you retrieve from the Web have been written by someone else, you should not trust them to perform with integrity. Unless you grant Java code special privileges, Java downloaded from the Net is automatically considered untrusted code. In order to ensure that untrusted code does nothing mischievous, it is important to limit what that untrusted code can do. Of course, completely limiting access to a system defeats the purpose of having executable content in the first place. After all, who wants to run a program that is not allowed to do anything? And who wants to develop or distribute such a program?

Somehow these two extremes need to be balanced. Java applets need enough power to do some things and sufficient restrictions so that they can't do others. The solution is controlling access to system resources carefully. This is what the Java security model aims to do.

Before we talk about the internals of the Java security model, it is important to discuss the potential problems raised by mobile code. There are four basic categories of potential attacks Java applets could facilitate:

  • Attacks that modify the system
  • Attacks that invade a user's privacy
  • Attacks that deny legitimate use of the machine by hogging resources
  • Attacks that antagonize a user
Table 2.1 lists the four classes in order of severity. There will be a brief discussion of each in turn. Keep in mind that this list of attacks is meant only to give a flavor of the kinds of things possible; it is by no means a complete list. Also note that the four attack classes overlap. As we shall see, hostile applets often fall into more than one category.

System Modification The most severe class of attacks. Applets that implement such attacks are attack applets. Consequences of these attacks: severe. Strong
Invasion of Privacy If you value your privacy, this attack class may be particularly odious. They are implemented by malicious applets. Include mail forging. Consequences of these attacks: moderate. Strong
Denial of Service Also serious but not severely so, these attacks can bring a machine to a standstill. Also implemented by malicious applets. May require reboot. Consequences of these attacks: moderate. Weak
Antagonism Merely annoying, this attack class is the most commonly encountered. Implemented by malicious applets. May require restart of browser. Consequences of these attacks: light to moderate. Weak
Figure 2.1

The relative severity of the four attack classes varies depending on the situation. For a lone user browsing on a personal computer, system modification attacks and privacy attacks are serious, but the other two classes are only a mild inconvenience. For big enterprise servers, however, attacks that antagonize employees and customers or deny them service altogether are serious issues.

System Modification

Java is a very powerful programming language, and with this power comes the potential for abuse. Most programming languages give programs the ability to read and modify data on the system where they are running. Java includes predefined classes with methods that can delete and otherwise modify files, modify memory, and even kill processes and threads. System modification attacks comprise the most critical risks. Java's designers have given much thought to preventing this class of attack.

In the most serious cases, system modification involves intrusion into the system itself. Like many parts of today's complex systems, Java can be misused as an avenue of attack. Given that crackers will use any tool available to compromise the security of a machine, special care must be taken to ensure that Java does not provide new ports of entry to a machine. That Java is designed to work on many different platforms makes this task that much more important.1

The good news is that using Java to break into a machine is not easy. The bad news is that such break-ins are certainly possible. This book refers to applets that implement system modification attacks as attack applets. Attack applets are a serious concern. In Chapter 5, "Attack Applets: Exploiting Holes in the Security Model," you will see how several such attacks have been successfully discovered and applied in the laboratory. Patches have been developed that make these attacks impossible, but the threat of other sophisticated attacks remains.

A good example of a system modification attack involves a security hole discovered in JDK 1.1.1 by the Princeton team in February 1997. The hole has since been fixed. The particular hole was a problem in the way code signing was implemented. As a result of a simple error in the JDK 1.1.1 code signing system (returning a mutable array on a Class.getsigners() method call instead of a copy of the array), an attack applet signed with a fake signature could escape the sandbox completely and acquire all the privileges available to completely trusted code. Once outside the sandbox, the attack applet could do anything at all, including installing a virus or a Trojan Horse. Put in simple terms, any machine attacked through the code signing hole could be completely compromised. More detail on the code signing hole and an attack applet called the Magic Coat is found in Chapter 5.

With so many machines running mission-critical applications, system modification attacks could lead to problems such as modified financial records in a database, which could in turn lead to financial loss and corporate bankruptcy. Modified medical records could result in fatally incorrect treatment. Care must be taken not to expose critical systems to new lines of attack. Crackers flock to the latest and greatest vulnerabilities. It is important that Java not become a cracking vehicle.

It is important to emphasize that no system modification attacks using Java have been seen outside the lab. That's lucky because if one of these attacks were invented by crackers, Java viruses could become a reality. Attack applets designed to carry out system modification attacks are the subject of Chapter 5.

Invasion of Privacy

A second general type of attack involves disclosing information about a user or host machine that should not be publicized. Some files are meant to be kept confidential. For example, on Unix machines, if someone gains access to the /etc/passwd file (which contains usernames and encrypted passwords) he or she could mount a password-cracking attack. A successful password-cracking attack is a complete invasion of a machine and can be escalated into a system-modification attack.

Sensitive information of other sorts can also be leaked from a system. Consider the implications of an unscrupulous company being able to steal the secret business plans of a competitor through corporate espionage. Or, if you are not a corporate user, consider your private email correspondence or your financial records being made public. If such confidential information is mailed or otherwise transferred off a system, the act can be called an invasion of privacy.

A particularly egregious risk, especially in today's e-commerce systems based on the use of cryptography, is loss of a private key. Consider that in Utah, legislation can be enacted into law through the application of a digital signature. If the governor's private key falls into the wrong hands, legal ramifications would be interesting at best (and disastrous at worst). Private keys should be among the most closely guarded secrets in existence.

Most modern workstations include sound capability. This opens up users to a new kind of eavesdropping. If an attacker can gain control of the microphone, then it is possible to listen in on the area immediately surrounding the workstation. More subtle eavesdropping includes monitoring process tables and file access. A Web-based version of eavesdropping might include keeping track of which links a user follows.

Forging mail could also be construed as a kind of invasion of privacy attack. If an outsider can gain enough information to forge mail that appears to be from you, then you are exposed to a large number of serious risks. As is discussed in Chapter 4, "Malicious Applets: Avoiding a Common Nuisance," Java makes the standard mail-forging attack a much more serious threat. Java successfully defends against some of these attacks. For example, file I/O is very closely guarded. However, this good feature is countered by the fact that applets always have a channel open back to their original host server. Should the applet somehow dig up some information, it would be very easy to send that information back to the original host server. Non-file-related members of the invasion of privacy class (such as mail forging) are harder to defend against. Short of defenses such as disabling a system's network ports, the mail forging attack is likely to remain a threat.

Denial of Service

Denial of service attacks make system resources unavailable. They occur, for example, when a process eats up more than the standard allotment of resources, essentially hogging the machine. There are many subcategories of denial of service attacks. Some examples include:

  • Completely filling a file system
  • Using up all available file pointers
  • Allocating all of a system's memory
  • Creating thousands of windows, effectively denying access to the output screen or window event queue
  • Using all of the machine's cycles (CPU time) by creating many high-priority threads
There are also some types of denial of service attacks that do not hog the systems resources, but lock up the system in some other way; for example, by acquiring a lock on some critical system resource, or by causing the system to wait for an input or output operation that can never complete. Although denial of service attacks are a real concern, Java's designers were not able to protect users from this class of attacks.

There is some debate over the relative importance of stopping denial of service attacks. In most cases, denial of service is more closely related to the class of annoyance attacks than to anything else. This is because recovering from a denial of service attack is usually not difficult (simply reboot the machine). But some computer systems perform very important, even mission-critical tasks. Denial of service to such a machine could be very serious. Consider the mayhem that would erupt should a malicious program lock up the machines running the stock market. Losses could be staggering.

Denial of service attacks are by far the most commonly encountered Java security concern. Implementing such an attack is not hard, but stopping one is. Malicious applets, the subject of Chapter 4, often make use of denial of service attacks. Unfortunately, the current security model does not offer a good solution to the denial of service problem. Planned enhancements to the Java security model promise to lessen the threat posed by denial of service attacks. Security experts know, however, that denial of service attacks are very tough to prevent, so we expect Java will not have strong defenses against denial of service any time soon.

If you are running a Java-enabled server application, you should be worried about denial of service attacks. If your server is configured to allow untrusted clients to upload "servlets" into your server, then any Internet user, anywhere in the world, can upload a denial of service servlet and crash your server. Your best defense is to make sure your server is configured to accept servlets only if they are written by people you trust. Remember, the Java sandbox does not protect you against denial of service attacks. Figure 2.1 shows how servlets differ from applets.

Fig 2.1

Figure 2.1 Servlets in Java are the server-side equivalent of applets for the client.

The idea is to provide a VM inside a Web server on which servlets can be run to carry out tasks once delegated to CGI scripts. When mobile code runs on a Web server, the risk of denial-of-service attack carries more importance. Servlets often access back-end databases on behalf of a Web server.


Less odious, but still of some concern, are attacks that merely antagonize or annoy a user. Playing unwanted sound files through a speaker or displaying obscene pictures on a monitor are two examples. Sometimes seemingly antagonistic attacks may be the result of simple programming errors. Chapter 4 contains some examples of antagonistic malicious applets.

Some denial of service attacks could be classified as merely antagonistic. A denial of service attack that opens many windows simultaneously, for example, can be reduced to an annoyance depending on the window manager in use. Many window managers require user intervention to place a window and, thus, open only one window at a time. Judging the severity and category of a particular attack is always a subjective and context-sensitive problem. This is no reason to pretend that such categories do not exist. In order to more thoroughly understand the risks associated with mobile code, these attacks need to be considered.

Java Risks in Perspective

It is important to emphasize that using Java is not the only way to carry out the four classes of attack discussed previously (system modification, invasion of privacy, denial of service, and antagonism). In fact, the risk of being hit with a non-Java-based system modification attack is much greater than the risk of an attack applet exploiting a hole in the Java security model to modify your system. There are countless non-Java examples of all four of attack classes sprinkled throughout the computer security literature. If you are interested in these topics, one of the most encyclopedic sources of information is the book Computer Related Risks by Peter G. Neumann [Neumann, 1995]. In any case, the book you're reading now is about Java security, so we will emphasize Java's vulnerability to attacks and protections against these attacks.

Java's designers are primarily concerned with stopping the worst potential attacks that hostile applets might carry out; that is, system modification and invasion of privacy attacks. One kind of mobile code that everyone wants to avoid is a computer virus. Though viruses traditionally target personal computers and have yet to make much of a dent in the networked Unix world, Java might well have changed all that. Fortunately, Java's security model seems to have been successful so far in thwarting the possibility of cross-platform viruses. Simply put, there have been no Java applet viruses (yet).2

That is not to say, however, that a Java virus is impossible. Despite claims to the contrary, any hole in the security sandbox would allow a Java virus to propagate. A system modification attack on a user's browser could choose to modify that user's homepage (and any other Web pages it could find), thereby propagating itself virus-like onto more Web pages.

The remaining attacks, denial of service and antagonism, are much harder to stop. Denial of service can be as simple as using all system resources, and it is sometimes hard to decide what constitutes a legitimate but taxing use of, say, your CPU, and what is an attack. The same thing goes for annoyance. A thread that never dies in one applet may be useful (if the applet is doing something like perpetually updating a stock quote in a window), while a thread that never dies in a hostile applet may be doing something evil such as monitoring your Web usage.

For a number of reasons, the market appears to consider Java security risks important. It all started when Sun included "secure" in the Java buzzwords list. The research community took this as a challenge, and the holes described in this book began to be discovered. These days, perceived Java security risks may, in fact, outstrip actual risks, though any such judgment needs to be made on a case-by-case basis.

One side effect of the large amount of attention paid to Java security is the proliferation of Java security vendors who market products that attempt to block all Java applets, scan applets for what might be hostile behavior before they run, or banish Java applet code to a central "sacrificial" server. If you are considering any of the many Java security enhancement products on the market, one good exercise is to ask how successful such products are in practice against each of the four attack classes introduced here. We will revisit this topic in Chapter 6, "Securing Java: Improvements, Solutions, and Snake Oil."

Previous Page
Previous Page

The Web

Next Page
Next Page

Menu Map -- Text links below

Chapter... Preface -- 1 -- 2 -- 3 -- 4 -- 5 -- 6 -- 7 -- 8 -- 9 -- A -- B -- C -- Refs
Front -- Contents -- Help

Copyright ©1999 Gary McGraw and Edward Felten.
All rights reserved.
Published by John Wiley & Sons, Inc.