Jar files and indexing

Recently at work, I ran into an interesting "feature" of how jar files and indexing work with Java and ant.

For background on indexing see:

http://java.sun.com/j2se/1.3/docs/guide/jar/jar.html#JAR%20Index

http://closingbraces.net/2007/05/13/jarclasspathandindex/

On some other developer's machines, some java applications were failing with NoClassDefFoundError exceptions. However, I personally never saw this error. The jar's with the classes that were not being found were listed in the Class-Path entry of the manifest of the jar that used them. I was extremely puzzled by this. I could fix the problem by explicitly adding the jar's to the classpath, but I didn't understand why I needed to do this when everything worked fine on my machine.

From the title of this post you can probably guess that the problem involved indexing. Our build process (using ant), creates a jar (let's call in myjar.jar) with the <jar> task, then it runs "jar -i" on it to index it (using an ). If all this happens everything works fine. However, there seems to be a race condition between when ant closes the jar file that it created with the <jar> task and when the "jar -i" command is run. When this happens, the "jar -i" command fails because the file is still locked and the index is not created.

I will go into exactly why this caused the error in a minute, but first I want to look at how I first tried to fix this because it also brings up an important point on the problem. The <jar> task has an optional attribute named "index" which can be used to have ant index the file. I tried using this instead of calling "jar -i" on it, but it did not fix the problem. Why didn't it? Because setting index="true" in the <jar> task does not do the same thing as running "jar -i" on the jar! This is very uninituitive! So what is the difference? Using the <jar> task, on the classes in the jar itself are added to the index; however, using "jar -i" the classes in the jar and all the classes used by the jar are added to the index.

Why does this make a difference? The key is in the second link above. Quoting:

The executable jar must either not have a META-INF/INDEX.LST file, or if it does have such an INDEX.LST file this needs to list the contents of both the executable jar and all of the other jars as well. Anything not in this list will not be found on the class path, regardless of the “Class-Path” entry in the manifest...

Two things to note, the filename I see is META-INF/INDEX.LIST (not LST) and the link goes on to say:

and regardless of any command-line “-classpath” (which is ignored anyway).

This did not apply in my case. We are using java -cp file.jar file.MainClass to run our apps, not java -jar file.jar. In a nutshell, if the jar has an index, it's Class-Path entry is ignored.

In my case, our application uses jacorb.jar (from JacORB). It uses two other jars: logkit-1.2.jar and avalon-framework-4.1.5.jar. These are the two jars I had to add to the classpath to fix the problem. My jar has jacorb.jar listed in its Class-Path entry.

If everything builds properly and myjar.jar is indexed using "jar -i", everything works because entries for all the classes used by myjar.jar are included in the index (including the ones in logkit-1.2.jar and avalon-framework-4.1.5.jar. The Class-Path entry is ignored since the index is present.

However, if the "jar -i" command fails, then the Class-Path entry is in effect and classes in jacorb.jar can be resolved, but not the classes that jacorb.jar uses. Why not? Because jacorb.jar was indexed using ant. So, it has an index, but it only contains its own classes, not the ones in the other two jars.

If you want to see this problem yourself. You can download the code I used to verify this from here. It has 3 jars: myjar.jar, dependjar.jar and depend2jar.jar. myjar uses dependjar which uses depend2jar. This is equivalent to myjar.jar which uses jacorb.jar which uses logkit-1.2.jar.

The ant script by default will build the jars without indexing and use jar -i to create indexed versions of the jars named imyjar.jar, idependjar.jar and idepend2jar.jar.

If you run ant with -Dindex="yes" it will index the plain jars (myjar.jar, dependjar.jar and depend2jar.jar) using ant's index method.

To recreate the problem I saw, type:

ant

ant build.dependjar -Dindex="yes"

java -cp myjar.jar myjar.MyClass

The first line builds all the plain jars with no indexing. The second line replaces dependjar.jar with one using ant's index funciton (just like jacorb.jar). The last line demonstrates the error that occurs. You can also verify that adding dependjar.jar to the classpath doesn't fix the error either.

In general I see several ways to fix this problem.

1. Avoid using indexing
2. Put all the jars on the classpath
3. Fully index all jars with "jar -i"
   

Hopefully this post will help anyone else running into this problem.

Time to start using this blog?

I guess it might be time to start using this blog.

Here's some background. I'm a software developer and ex-CS professor. When I was teaching, I wrote a book on 32-bit x86 assembly language that I provide free on the web.

In my current job, I mostly program in C++ with some Java. I'm also a big fan of Python. I expect that most of my posts with be dealing with these languages (and assembly of course).