View source for JDT Core Programmer Guide/ECJ/Investigating

<css>
/* box for bash listings */
.source-bash { border:#2F6FAB 1px dashed; background-color:#F9F9F9; padding:4px; }
</css>

==Strategies for investigating compiler bugs==

===Comparing compiler versions===
For initial triage it is useful to test how different compiler versions respond to a given source file. This can be done quickly using a shell and a few shell scripts. The following assumes <code>bash</code>.

'''<code>ecj-HEAD.sh</code>: Run the compiler from class files of an existing JDT/Core workspace / git clone''':
<syntaxhighlight lang="bash">
#!/bin/sh

# Adjust path to the root of a git clone of JDT/Core:
JDT_GIT=/home/git/eclipse.jdt.core
# Adjust path to a suitable Java installation:
JAVA_HOME=${JAVA_HOME:-/home/java/jdk1.8.0}

ECJ_BINS=${JDT_GIT}/org.eclipse.jdt.core/bin:${JDT_GIT}/org.eclipse.jdt.compiler.apt/bin
PATH=${JAVA_HOME}/bin:${PATH}

java -classpath $ECJ_BINS org.eclipse.jdt.internal.compiler.batch.Main $*
</syntaxhighlight>


'''<code>ecj.sh</code> Run the compiler from one out of a set of stored ecj jar files''':

Assume you have a directory (<code>$BASE</code> in the script below) holding different versions of ecj, with file names like <code>ecj-4.16.jar</code>, then the following script can be used to pick and use any of the stored versions:
<syntaxhighlight lang="bash">#!/bin/sh

BASE=${HOME}/jdt/ecj

ECJ=${BASE}/ecj-${1}.jar
if [ -f ${ECJ} ]
then
  echo "Compiling with $ECJ"
else
  echo "Not found: ${ECJ}"
  echo "Available:"
  cd ${BASE}
  ls ecj-*.jar
  exit 2
fi
EXTRA="-proc:none"
case ${1} in
    4.[4-7]*)
        JAVA_HOME=/home/java/jdk1.8.0
        PATH=${JAVA_HOME}/bin:${PATH}
        ;;
    4.[0-3].*)
	JAVA_HOME=/home/java/jdk1.6.0
        PATH=${JAVA_HOME}/bin:${PATH}
	;;
    3.[0-9]*)
	JAVA_HOME=/home/java/jdk1.5.0
        PATH=${JAVA_HOME}/bin:${PATH}
	EXTRA=""
	;;
esac

shift
echo java -jar ${ECJ} ${EXTRA} $*
java -jar ${ECJ} ${EXTRA} $*
</syntaxhighlight>


Example invocation:
<code>$ ecj.sh 4.16 -1.8 Hello.java</code>

The extra argument <code>-proc:none</code> is useful because some historic ecj versions lack the annotation processing classes and thus crash when trying to perform annotation processing.

Also note, that different version ranges of ecj require different Java versions to run. Adjust paths in the script to your local installations.

====Speedy bisecting====
If the compiler behavior changed significantly at a yet unknown point in time, the following appears to be the fasted approach to find the exact commit causing the change:

# Use the parametric script <code>ecj.sh</code> to find the release or even milestone that first introduced the change.
# Identify git tags of last version with old behavior and first version with new behavior
# In a shell inside a JDT/Core git clone perform the <code>git bisect</code> subcommands
# In an Eclipse workspace that has org.eclipse.jdt.core from that git imported, refresh the git repo and wait for auto build to complete
# In another shell where you have your test source invoke <code>ecj-HEAD.sh</code>
# Check the result and go back to 3. until done


It is useful to have a git clone and eclipse workspace dedicated only to bisecting, so that you never mess up your regular workspace. This is reflected by a copy of <code>ecj-HEAD.sh</code> where <code>${JDT_GIT}</code> points to that instance of org.eclipse.jdt.core.

===Comparing compiler output===

In most cases, it is sufficient to check which compiler version '''accepts''' / '''rejects''' the given test program, possibly checking also for warnings.

In other cases it is also necessary to compare '''executions''' of the compiled test program.

In rare case it may even be necessary to also check the compiler '''output files''' (e.g., to check class file attributes like annotations etc). When different compiler versions generate different byte code, the following sequence would identify that difference:

<source lang="bash">
$ ecj.sh 4.10 Hello.java
$ javap -p -v -c Hello.class > Hello-4.10.list
$ ecj.sh 4.11 Hello.java
$ javap -p -v -c Hello.class > Hello-4.11.list
$ diff Hello-4.10.list Hello-4.11.list
</source>

Obviously, a graphical diff tool (like <code>kompare</code>) comes in handy for this task.

===Typical breakpoints===

'''Error reporting'''
* Inside <code>ProblemHandler.handle(int,String[],int,String[],int,int,int,ReferenceContext,CompilationResult)</code>
** Breakpoints after <code>case ProblemSeverities.Error</code> and <code>case ProblemSeverities.Warning</code> will be triggered by every detected problem of the given severity (although some may be filtered out during <code>CompilationUnitDeclaration.finalizeProblems</code>).
* Breakpoints in constructors of <code>Problem*Binding</code>: this bindings are created when resolving found an error, sometimes the creation of the problem binding is closer to the cause then the above handle() method.
* To find the locations responsible for a particular known error message proceed as follows:
** Inside <code>org/eclipse/jdt/internal/compiler/problem/messages.properties</code> locate the error message and identify the error number at the start of that text line
** Find the same number in <code>org.eclipse.jdt.core.compiler.IProblem</code> and search for references of the corresponding constant
** You will have found one or more methods in org.eclipse.jdt.internal.compiler.problem.ProblemReporter, call hierarchy of its callers are what you are looking for
*** Shortcut: call hierarchy on the constant in IProblem (make sure the '''Field access''' option from the view menu is ''not'' set to '''Write access''').


'''Type lookup from the environment'''
* Methods <code>LookupEnvironment.askForType</code> (both overloads!): every additional type pulled into compilation must be found by one of these methods.

'''Parsing'''
* A breakpoint on the first line of method <code>Parser.consumeRule(int)</code> approximates single stepping through the parse process.
** Note, that Parser & Scanner have useful toString() methods, so the progress of parsing can be watched in the details of the Variables view.

'''TypeInference'''
* ''I'll write a section dedicated only to debugging type inference''

'''Checking compiler output during tests'''
* <code>AbstractRegressionTest.runTest()</code> (the core method, longest parameter list): a breakpoint immediately after <code>batchCompiler.compile()</code> let's you inspect the generated class files, to be found in your tmp directory at <code>comptest/run&lt;timestamp&gt;/regression</code>.

'''Completion'''
* An exception breakpoint on <code>org.eclipse.jdt.internal.codeassist.complete.CompletionNodeFound</code> will trigger when the completion node is resolved.
** Use this breakpoint if the AST looks OK, but problems exist downstream

'''Other exceptions used in the compiler'''

Exceptions where throw and catch are near to each other are not listed here.
* <code>AbortCompilation</code> and subtypes: thrown when severe problems suggest that nothing good will come afterwards
* <code>CopyFailureException</code>: to enable multiple resolution attempts, lambda expressions are copied (AST). In case of syntax errors, this may not be possible.
* <code>ClassFormatException</code>: when a .class file cannot be interpreted.
* <code>InferenceFailureException</code>: initially used to detect gaps in the implementation of type inference, mostly obsolete by now.



[[Category:JDT]]