compilerMonitor: a CAUSE tool for javac.exe

Goals

• How long does it take a novice software developer to decipher the cryptic error messages generated by their favorite compiler?
• Do more experienced developers process compiler error messages more quickly?
• How long does it take to transition from being a novice to an experienced compiler user?

The compilerMonitor is a research tool that helps to answer these questions. One hypothesis is that it takes too long time (perhaps months) to transition from being a novice to an experienced compiler user. Better error messages from the compiler could provide significant productivity gains.

• Results that support the hypothesis will be data that suggest that the transition time between a novice and an experienced compiler user is greater than 40 hours of compiler usage. In other words, the transition period is so long that improving the tool to minimize the transition time would be well worth the effort.
• Results that negate the hypothesis will be data that suggest that the transition time is less than 40 hours, ie, the transition time is negligible.

What is the compilerMonitor?

Specifically, the compilerMonitor persists a record of all javac error messages and the sources that caused/fixed the errors.

Data Generated by compilerMonitor

Don't monkey with the archive/compilationDb directory. The compilerMonitor stores zero-byte text files in this directory to indicate whether a particular file compiled with errors on the previous invocation. It uses this information to determine if a successfully compiled file is a fix (and thus stored in the archive) or not.

Strategy for collecting compilerMonitor Data

1. A sufficiently clear picture of the main patterns of developer-compiler interacation
2. Time trials for how long it takes for developers of different experience levels to work through each pattern.

The second thing we'll need is the data for the time-trials. Unlike the pattern data, this will require a more controlled setting. Real-world interruptions (like trips to the john and conversations with one's neighbor) would cloud any 'timed' data taken directly from the field. So, one tact would be to create sample source code that generates particular compiler errors that emulate the discovered patterns. Then, developers of different levels of experience would be asked to work through the compiler errors in the sample source code, uninterrupted.

What are CAUSE tools?

• Jacob Neilson popularized the relatively new concept of CAUSE tools. Take a look at chapter 10 of his book, "Usability Engineering", as well as this fascinating survey of usability laboratories.
• This is a great document about 'CAUE' tools. This is a great paper, but I think they need to add the 'S' to 'CAUE' (to end up with 'cauSe') like Jacob did. :-)
• This thorough document appears to describe a CAUSE tool called 'EMBASSI'. My German ist a bit rusty, so I'm not quite sure. Does anyone know of an English translation?
• This paper came out of Daimler-Chrysler, as did the previous one. It defines CAUSE as "Computer-Aided Usability Engineering Environment".
• I could spend hours browsing through usability books. Here is one interesting group of books on usability.
• This is a nice little introduction to usability testing.

1. Perhaps a shrinking average amount of time between a compiler error and a corresponding fix suggests increasing productivity.
2. Perhaps few errors for a given time period suggests increasing productivity.

1. Unzip files from compilerMonitor-bin-???.zip into a blank directory. Perhaps c:\compilerMonitor-0.04
2. Tell the compilerMonitor which JDK (1.3 and 1.4 are supported) you want to compile with by setting the JDK_HOME environment variable, like this:

            set JDK_HOME=c:\j2sdk1.4.1
            

3. Instead of using javac.exe, use javacM.exe
4. Optionally, add the new directory to your PATH environment variable -- this is for your convenience in using javacM.exe.
You can configure your favorite IDE to use javacM.exe instead of plain old javac.exe.

Use the compilerMonitor to monitor ANT <javac> activity.

1. Tell ANT about the directory where you unzipped compilerMonitor-bin-???.zip, by setting this environment variable:

              set ANT_OPTS=-Dcompilermonitor.home=c:\myDir

2. Put compilerMonitor-???.jar, regexp-1.2.jar and junit3.7.jar in your computer's CLASSPATH. Once I figure out how to dynamically 'update' the classpath, this last step will no longer be a requirement.
3. Drop the jar file compilermonitor-ant-???.jar into the lib directory under %ANT_HOME%.

   NOTE: this jar must precede the ant.jar in the same directory, because I hijacked org.apache.tools.ant.taskdefs.compilers.CompilerAdapterFacory. One way to do this is to make sure the compilermonitor-ant-???.jar has a newer time stamp than ant.jar.

License

Platform

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.