Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
How to test TPTP with line level code coverage enabled
Background
EMMA is an open-source toolkit written in Java for measuring and reporting line level code coverage for Java applications.
TPTP is currently conducting a Proof of Concept (PoC) to evaluate EMMA for generating and reporting line level code coverage statistics when testing TPTP.
EMMA Instrumentation
To generate line level code coverage statistics, the TPTP Software Development Kit (SDK) is statically instrumented using EMMA.
The Release Engineering Team has integrated the instrumentation with the TPTP build process to generate statically instrumented TPTP SDK ZIP for testing. The instrumented TPTP SDK ZIP are available for all candidate builds from the TPTP download site (see the 'EMMA Instrumented' link).
To manually generate a statically instrumented TPTP SDK ZIP:
- JARed plug-ins:
-
java.exe -cp emma.jar emma instr -verbose -instrpath org.eclipse.tptp.*.jar -outfile EMMA_Coverage.metadata -merge y -outmode overwrite -filter +org.eclipse.hyades.*,+org.eclipse.tptp.*
-
- JARs in plug-ins:
-
java.exe -cp %EMMA_JAR% emma instr -verbose -instrpath org.eclipse.tptp.*\*.jar -outfile EMMA_Coverage.metadata -merge y -outmode overwrite -filter +org.eclipse.hyades.*,+org.eclipse.tptp.*
-
where
- org.eclipse.tptp.*.jar: TPTP JARed plug-in.
- org.eclipse.tptp.*\*.jar: TPTP JARs in plug-ins.
- EMMA_Coverage.metadata: One or more EMMA metadata file(s).
- EMMA_Coverage.data: One or more EMMA coverage file(s).
- EMMA_Coverage_Report.html: HTML EMMA coverage report.
Note: Directory paths are omitted.
For more information, see the EMMA instructions.
Reporting
To view the line level code coverage statistics, the line level code coverage metadata and data files created when instrumenting and running TPTP are merged and reported using EMMA.
Note: The line level code coverage statistics in this report are generated from executing only a fraction of the test cases for the TPTP 4.4.0.3 smoke test pass.
To generate a EMMA coverage report:
java.exe -cp emma.jar emma report -verbose -input EMMA_Coverage.metadata,EMMA_Coverage.data -report html -Dreport.out.file=EMMA_Coverage_Report.html
where
- EMMA_Coverage.metadata: One or more EMMA metadata file(s).
- EMMA_Coverage.data: One or more EMMA coverage file(s).
- EMMA_Coverage_Report.html: HTML EMMA coverage report.
For more information, see the EMMA instructions.