EclipseLink/Development/Testing/Frameworks

EclipseLink has a significant number of tests, and supports many different test configurations, platforms, databases and parsers that need to be tested.

This article attempts to detail each component's testing requirements, framework and strategy.

Components

Each EclipseLink component defines its own tests under <component>/eclipselink.<component>.test. Each component has different testing requirements so uses slightly different methodologies.

The goals for test consistency across the components include the following:

• Each component defines an Eclipse test project in the root of the test directory.
• The test project should include pre-configured launch targets for running/debugging the tests, ideally through the Eclipse JUnit runner.
• All tests should be runnable through ant, and scripted in the <test>/build.xml.
• The ant build file should define any test configuration in a <test>/test.properties file.
• The test.properties should be able to be overwritten in {user.home}/test.properties.
• Ideally all tests should be written in JUnit 3.
• The ant test run should output results in html format in a <test>/reports directory.
• The ant build file should define targets for test, test.srg and test.lrg.
• A root test.srg and test.lrg will be defined to run each srg/lrg in each component.

Current testing frameworks

Core

• framework: org.eclipse.testing.framework.**
• legacy framework that has been migrated to be an extension of JUnit3
• provides UI runner with useful debugging functionality (SQL, Java, session properties, cache, logging, individual test running, storage of results in result database)
• UI can also run any JUnit3 test, and allows JPA JUnit3 tests to be run and be debuggable.
• provides advanced performance and concurrency tests which rely on result database storage
• tests are fully JUnit3 compatible and runnable in any JUnit3 runner including Eclipse
• provides command line runner and ant scripts
• ant scripts for cross database testing
• provides extended functionality for accessing POJO Session, table creation/population, object comparison, transactional tests, etc.
• 1000's of existing tests, POJO, cache-coordination, remote, POJO performance, JPA performance, POJO concurrent, MW integration, sessions XML, EIS, XDB, etc.

Issues

• not 'normal' JUnit3 tests
• too many tests to port to anything new
• contain functionality required for performance regression testing
• provide UI many developers are dependent on for debugging
• should not be adding any new tests for POJO API

Core : server

• (not currently ported to EclipseLink)
• framework: oracle.toplink.testing.ejb.testframework
• extension of POJO framework for running in JEE servers
• not currently compatible with JUnit3
• command line runner, ant scripts for building, deployment, and running
• provides extended functionality for JTA transactions, JEE platform independence
• supports running of existing POJO tests inside the JEE server
• ant scripts for cross server testing, clustering
• 1000's of existing tests, CMP (not part of EclipseLink), SessionBeans, BMP (not part of EclipseLink), CMP performance (not part of EclipseLink), JTA, resource, EIS/JCA, class-loader, clustering, stress

Issues

• not compatible with JUnit3
• too many tests to port to anything new, but most tests are CMP and obsolete
• contain functionality required for performance regression testing in server
• should not be adding any new tests for POJO API or CMP

JPA : legacy

• framework: org.eclipse.testing.jpa.EntityContainerTestBase/CMP3TestModel
• extension of POJO framework for JPA tests
• compatible with JUnit3
• 100's of tests

Issues

• not too many tests, should be port to new JPA JUnit3 framework

JPA

• Framework: org.eclipse.testing.framework.*/junit.JUnitTestCase
• Extension of JUnit3 for running JPA tests.
• Ant scripts for running tests with multiple weaving options.
• 1000's of existing tests, JPA, weaving, static weaving, orm XML, annotations, EclipseLink annotations.

Issues

• Tests need to verify results, currently most test write objects, but do not read them back and ensure they were persisted correctly.
• Require an SRG suite.

JPA : server

• Framework: org.eclipse.testing.framework.junit.JUnitTestCase/framework.*/framework.server.*
• Extension of JUnit3 for running JPA tests on a JEE server using a EJB 3 SessionBean.
• Generic Ant scripts for running tests on any server, including scripts for OC4J.
• Can run any JPA test, also includes tests for SessionBeans.

Issues

• JPA tests need to be migrated to allow running in JEE server (redirect transactions, entity manager access)
• Scripts required for other supported JEE servers (WLS, WAS, JBoss, Glassfish).
• Also need to support running tests with Spring, (Tomcat?).
• Ant test targets for all JPA tests.

JPA : Orajtst

• not currently ported to eclipselink
• framework: oracle.toplink.testing.tests.internal.ejb, oraj.jar
• extension of JUnit3 and oraj framework
• ant scripts for running tests
• cannot current run outside of ADE environment nor from IDE
• 1000's of existing tests, clone of JPA JUnit tests ported to JEE, advanced JPA, spring

Issues

• clone of existing tests, causes major maintenance effort keeping in synch
• clone prevent news tests from being run in server
• oraj and ADE dependency
• once JPA JUnit tests are migrate to support running in server these can be removed

MOXy : OXM

• framework: org.eclipse.testing.ox.XMLTestCase/*
• extension of JUnit3 for running OXM tests

• appears to be stateless; could be migrated to almost any other framework

• ant scripts for running tests (not yet ported to eclipselink)
• 1000's of existing tests, SAX, DOM

• EISMappingTestCases(6b) extends OXTestCase

• sub framework: org.eclipse.testing.oxm.OXTestCase (same tests are run multiple times)
  • Tests are run once using a SAX parser
  • Tests are run once using a DOM parser
  • Tests are run once using document preservation
  • Tests are run once using deployment XML as the source of the metadata

MOXy : JAXB

• framework: org.eclipse.testing.ox.jaxb
• extension of OXM tests for running JAXB tests
• ant scripts for running tests (not yet ported to eclipselink)
• 1000's of existing tests

SDO

• framework: org.eclipse.sdo.testing
• extension of JUnit3 for running SDO tests
• ant scripts for running tests (not yet ported to eclipselink)
• 1000's of existing tests
• Sub Framework: Loading metadata from XML Schema
  • Tests are run using generic DataObjects
  • Tests are run generating static classes from the XML Schema

DBWS

Used by DBWS components, some misc. core tests

• extension of JUnit4: use @RunWith annotation

Issues

• newest framework, fewest tests (~200)
• more 'future-proof' than JUnit3 which is now end-of-life