Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "EclipseLink/Development/Testing/Frameworks"
< EclipseLink | Development | Testing
| (39 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
| − | + | EclipseLink has a significant number of tests, and supports many different test configurations, platforms, databases and parsers that need to be tested. | |
| − | + | This article attempts to detail each component's testing requirements, framework and strategy. | |
| − | * | + | |
| − | * | + | == Components == |
| − | * | + | Each EclipseLink component defines its own tests under <component>/eclipselink.<component>.test. Each component has different testing requirements so uses slightly different methodologies. |
| − | * | + | |
| − | * | + | The goals for test consistency across the components include the following: |
| − | * | + | * Each component defines an Eclipse test project in the root of the test directory. |
| − | * | + | * The test project should include pre-configured launch targets for running/debugging the tests, ideally through the Eclipse JUnit runner. |
| − | * | + | * All tests should be runnable through ant, and scripted in the <test>/build.xml. |
| − | + | * The ant build file should define any test configuration in a <test>/test.properties file. | |
| − | * | + | * The test.properties should be able to be overwritten in {user.home}/test.properties. |
| + | * Ideally all tests should be written in JUnit 3. | ||
| + | * The ant test run should output results in html format in a <test>/report directory. | ||
| + | * The ant build file should define targets for test, test.srg and test.lrg. | ||
| + | * A root test.srg and test.lrg will be defined to run each srg/lrg in each component. | ||
== Current testing frameworks == | == Current testing frameworks == | ||
| − | === | + | === Core === |
| − | * framework: org.eclipse.testing.framework. | + | * framework: org.eclipse.persistence.testing.framework.** |
* legacy framework that has been migrated to be an extension of JUnit3 | * legacy framework that has been migrated to be an extension of JUnit3 | ||
* provides UI runner with useful debugging functionality (SQL, Java, session properties, cache, logging, individual test running, storage of results in result database) | * provides UI runner with useful debugging functionality (SQL, Java, session properties, cache, logging, individual test running, storage of results in result database) | ||
| − | * UI can also run any | + | * UI can also run any JUnit3 test, and allows JPA JUnit3 tests to be run and be debuggable. |
* provides advanced performance and concurrency tests which rely on result database storage | * provides advanced performance and concurrency tests which rely on result database storage | ||
| − | * tests are fully | + | * tests are fully JUnit3 compatible and runnable in any JUnit3 runner including Eclipse |
| − | * provides command line runner and ant scripts | + | * provides command line runner and ant scripts |
| − | * ant scripts for cross database testing | + | * ant scripts for cross database testing |
* provides extended functionality for accessing POJO Session, table creation/population, object comparison, transactional tests, etc. | * provides extended functionality for accessing POJO Session, table creation/population, object comparison, transactional tests, etc. | ||
* 1000's of existing tests, POJO, cache-coordination, remote, POJO performance, JPA performance, POJO concurrent, MW integration, sessions XML, EIS, XDB, etc. | * 1000's of existing tests, POJO, cache-coordination, remote, POJO performance, JPA performance, POJO concurrent, MW integration, sessions XML, EIS, XDB, etc. | ||
| − | + | : '''Issues''' | |
| − | * not normal JUnit3 tests | + | :* not 'normal' JUnit3 tests |
| − | * too many tests to port to anything new | + | :* too many tests to port to anything new |
| − | * contain functionality required for performance regression testing | + | :* contain functionality required for performance regression testing |
| − | * provide UI many developers are dependent on for debugging | + | :* provide UI many developers are dependent on for debugging |
| − | * should not be adding any new tests for POJO API | + | :* should not be adding any new tests for POJO API |
| + | :* ant test targets required for non LRG tests | ||
| + | :* need to be able to run tests in the server | ||
| − | === | + | === Core : server === |
| − | * (not currently ported to | + | * (not currently ported to EclipseLink) |
* framework: oracle.toplink.testing.ejb.testframework | * framework: oracle.toplink.testing.ejb.testframework | ||
* extension of POJO framework for running in JEE servers | * extension of POJO framework for running in JEE servers | ||
| Line 43: | Line 49: | ||
* supports running of existing POJO tests inside the JEE server | * supports running of existing POJO tests inside the JEE server | ||
* ant scripts for cross server testing, clustering | * ant scripts for cross server testing, clustering | ||
| − | * 1000's of existing tests, CMP (not part of | + | * 1000's of existing tests, CMP (not part of EclipseLink), SessionBeans, BMP (not part of EclipseLink), CMP performance (not part of EclipseLink), JTA, resource, EIS/JCA, class-loader, clustering, stress |
| − | + | : '''Issues''' | |
| − | * not compatible with | + | :* not compatible with JUnit3 |
| − | * too many tests to port to anything new, but most tests are CMP and obsolete | + | :* too many tests to port to anything new, but most tests are CMP and obsolete |
| − | * contain functionality required for performance regression testing in server | + | :* contain functionality required for performance regression testing in server |
| − | * should not be adding any new tests for POJO API or CMP | + | :* should not be adding any new tests for POJO API or CMP |
| − | === | + | ---- |
| − | * framework: org.eclipse.testing.jpa.EntityContainerTestBase/CMP3TestModel | + | |
| + | === JPA : legacy === | ||
| + | * framework: org.eclipse.persistence.testing.jpa.EntityContainerTestBase/CMP3TestModel | ||
* extension of POJO framework for JPA tests | * extension of POJO framework for JPA tests | ||
* compatible with JUnit3 | * compatible with JUnit3 | ||
* 100's of tests | * 100's of tests | ||
| − | + | : '''Issues''' | |
| − | * not too many tests, should be port to new JPA | + | :* not too many tests, should be port to new JPA JUnit3 framework |
| − | ===JPA | + | === JPA === |
| − | * | + | * Framework: org.eclipse.persistence.testing.framework.*/junit.JUnitTestCase |
| − | * | + | * Extension of JUnit3 for running JPA tests. |
| − | * | + | * Ant scripts for running tests with multiple weaving options. |
| − | + | * 1000's of existing tests, JPA, weaving, static weaving, orm XML, annotations, EclipseLink annotations. | |
| − | * 1000's of existing tests, JPA, weaving, orm XML, | + | |
| − | Issues: | + | : '''Issues''' |
| − | * | + | :* Tests need to verify results, currently most test write objects, but do not read them back and ensure they were persisted correctly. |
| − | * | + | :* Require an SRG suite. |
| − | === | + | === JPA : server === |
| − | * (not currently ported to eclipselink | + | * Framework: org.eclipse.persistence.testing.framework.junit.JUnitTestCase/framework.*/framework.server.* |
| + | * Extension of JUnit3 for running JPA tests on a JEE server using a EJB 3 SessionBean. | ||
| + | * Generic Ant scripts for running tests on any server, including scripts for OC4J. | ||
| + | * Can run any JPA test, also includes tests for SessionBeans. | ||
| + | |||
| + | : '''Issues''' | ||
| + | :* JPA tests need to be migrated to allow running in JEE server (redirect transactions, entity manager access) | ||
| + | :* Scripts required for other supported JEE servers (WLS, WAS, JBoss, Glassfish). | ||
| + | :* Also need to support running tests with Spring, (Tomcat?). | ||
| + | :* Ant test targets for all JPA tests. | ||
| + | :* Require clustering and stress tests. | ||
| + | |||
| + | === JPA : Orajtst === | ||
| + | |||
| + | * not currently ported to eclipselink | ||
* framework: oracle.toplink.testing.tests.internal.ejb, oraj.jar | * framework: oracle.toplink.testing.tests.internal.ejb, oraj.jar | ||
* extension of JUnit3 and oraj framework | * extension of JUnit3 and oraj framework | ||
| Line 79: | Line 100: | ||
* 1000's of existing tests, clone of JPA JUnit tests ported to JEE, advanced JPA, spring | * 1000's of existing tests, clone of JPA JUnit tests ported to JEE, advanced JPA, spring | ||
| − | + | : '''Issues''' | |
| − | * clone of existing tests, causes major maintenance effort keeping in synch | + | :* clone of existing tests, causes major maintenance effort keeping in synch |
| − | * clone prevent news tests from being run in server | + | :* clone prevent news tests from being run in server |
| − | * oraj and ADE dependency | + | :* oraj and ADE dependency |
| − | * once JPA JUnit tests are migrate to support running in server these can be removed | + | :* once JPA JUnit tests are migrate to support running in server these can be removed |
| − | ===OXM | + | ---- |
| − | * framework: org.eclipse.testing.ox. | + | |
| + | === MOXy : OXM === | ||
| + | * framework: org.eclipse.persistence.testing.ox.XMLTestCase/* | ||
* extension of JUnit3 for running OXM tests | * extension of JUnit3 for running OXM tests | ||
| − | * ant scripts for running tests | + | :* appears to be stateless; could be migrated to almost any other framework |
| + | * ant scripts for running tests | ||
* 1000's of existing tests, SAX, DOM | * 1000's of existing tests, SAX, DOM | ||
| + | :* EISMappingTestCases(6b) extends OXTestCase | ||
| + | * sub framework: org.eclipse.persistence.testing.oxm.OXTestCase (same tests are run multiple times) | ||
| + | ** Tests are run once using a SAX parser | ||
| + | ** Tests are run once using a DOM parser | ||
| + | ** Tests are run once using document preservation | ||
| + | ** Tests are run once using deployment XML as the source of the metadata | ||
| − | ===JAXB | + | === MOXy : JAXB === |
| − | * framework: org.eclipse.testing.ox.jaxb | + | * framework: org.eclipse.persistence.testing.ox.jaxb |
* extension of OXM tests for running JAXB tests | * extension of OXM tests for running JAXB tests | ||
| − | * ant scripts for running tests | + | * ant scripts for running tests |
* 1000's of existing tests | * 1000's of existing tests | ||
| − | ===SDO | + | === SDO === |
| − | * framework: org.eclipse.sdo.testing | + | * framework: org.eclipse.persistence.sdo.testing |
* extension of JUnit3 for running SDO tests | * extension of JUnit3 for running SDO tests | ||
* ant scripts for running tests (not yet ported to eclipselink) | * ant scripts for running tests (not yet ported to eclipselink) | ||
* 1000's of existing tests | * 1000's of existing tests | ||
| + | * Sub Framework: Loading metadata from XML Schema | ||
| + | ** Tests are run using generic DataObjects | ||
| + | ** Tests are run generating static classes from the XML Schema | ||
| + | |||
| + | ---- | ||
| + | |||
| + | === DBWS === | ||
| + | Used by DBWS components, some misc. core tests | ||
| + | * extension of JUnit4: use @RunWith annotation | ||
| + | |||
| + | : '''Issues''' | ||
| + | :* newest framework, fewest tests (~200) | ||
| + | :* more 'future-proof' than JUnit3 which is now end-of-life | ||
Latest revision as of 10:30, 20 December 2007
EclipseLink has a significant number of tests, and supports many different test configurations, platforms, databases and parsers that need to be tested.
This article attempts to detail each component's testing requirements, framework and strategy.
Contents
Components
Each EclipseLink component defines its own tests under <component>/eclipselink.<component>.test. Each component has different testing requirements so uses slightly different methodologies.
The goals for test consistency across the components include the following:
- Each component defines an Eclipse test project in the root of the test directory.
- The test project should include pre-configured launch targets for running/debugging the tests, ideally through the Eclipse JUnit runner.
- All tests should be runnable through ant, and scripted in the <test>/build.xml.
- The ant build file should define any test configuration in a <test>/test.properties file.
- The test.properties should be able to be overwritten in {user.home}/test.properties.
- Ideally all tests should be written in JUnit 3.
- The ant test run should output results in html format in a <test>/report directory.
- The ant build file should define targets for test, test.srg and test.lrg.
- A root test.srg and test.lrg will be defined to run each srg/lrg in each component.
Current testing frameworks
Core
- framework: org.eclipse.persistence.testing.framework.**
- legacy framework that has been migrated to be an extension of JUnit3
- provides UI runner with useful debugging functionality (SQL, Java, session properties, cache, logging, individual test running, storage of results in result database)
- UI can also run any JUnit3 test, and allows JPA JUnit3 tests to be run and be debuggable.
- provides advanced performance and concurrency tests which rely on result database storage
- tests are fully JUnit3 compatible and runnable in any JUnit3 runner including Eclipse
- provides command line runner and ant scripts
- ant scripts for cross database testing
- provides extended functionality for accessing POJO Session, table creation/population, object comparison, transactional tests, etc.
- 1000's of existing tests, POJO, cache-coordination, remote, POJO performance, JPA performance, POJO concurrent, MW integration, sessions XML, EIS, XDB, etc.
- Issues
- not 'normal' JUnit3 tests
- too many tests to port to anything new
- contain functionality required for performance regression testing
- provide UI many developers are dependent on for debugging
- should not be adding any new tests for POJO API
- ant test targets required for non LRG tests
- need to be able to run tests in the server
Core : server
- (not currently ported to EclipseLink)
- framework: oracle.toplink.testing.ejb.testframework
- extension of POJO framework for running in JEE servers
- not currently compatible with JUnit3
- command line runner, ant scripts for building, deployment, and running
- provides extended functionality for JTA transactions, JEE platform independence
- supports running of existing POJO tests inside the JEE server
- ant scripts for cross server testing, clustering
- 1000's of existing tests, CMP (not part of EclipseLink), SessionBeans, BMP (not part of EclipseLink), CMP performance (not part of EclipseLink), JTA, resource, EIS/JCA, class-loader, clustering, stress
- Issues
- not compatible with JUnit3
- too many tests to port to anything new, but most tests are CMP and obsolete
- contain functionality required for performance regression testing in server
- should not be adding any new tests for POJO API or CMP
JPA : legacy
- framework: org.eclipse.persistence.testing.jpa.EntityContainerTestBase/CMP3TestModel
- extension of POJO framework for JPA tests
- compatible with JUnit3
- 100's of tests
- Issues
- not too many tests, should be port to new JPA JUnit3 framework
JPA
- Framework: org.eclipse.persistence.testing.framework.*/junit.JUnitTestCase
- Extension of JUnit3 for running JPA tests.
- Ant scripts for running tests with multiple weaving options.
- 1000's of existing tests, JPA, weaving, static weaving, orm XML, annotations, EclipseLink annotations.
- Issues
- Tests need to verify results, currently most test write objects, but do not read them back and ensure they were persisted correctly.
- Require an SRG suite.
JPA : server
- Framework: org.eclipse.persistence.testing.framework.junit.JUnitTestCase/framework.*/framework.server.*
- Extension of JUnit3 for running JPA tests on a JEE server using a EJB 3 SessionBean.
- Generic Ant scripts for running tests on any server, including scripts for OC4J.
- Can run any JPA test, also includes tests for SessionBeans.
- Issues
- JPA tests need to be migrated to allow running in JEE server (redirect transactions, entity manager access)
- Scripts required for other supported JEE servers (WLS, WAS, JBoss, Glassfish).
- Also need to support running tests with Spring, (Tomcat?).
- Ant test targets for all JPA tests.
- Require clustering and stress tests.
JPA : Orajtst
- not currently ported to eclipselink
- framework: oracle.toplink.testing.tests.internal.ejb, oraj.jar
- extension of JUnit3 and oraj framework
- ant scripts for running tests
- cannot current run outside of ADE environment nor from IDE
- 1000's of existing tests, clone of JPA JUnit tests ported to JEE, advanced JPA, spring
- Issues
- clone of existing tests, causes major maintenance effort keeping in synch
- clone prevent news tests from being run in server
- oraj and ADE dependency
- once JPA JUnit tests are migrate to support running in server these can be removed
MOXy : OXM
- framework: org.eclipse.persistence.testing.ox.XMLTestCase/*
- extension of JUnit3 for running OXM tests
- appears to be stateless; could be migrated to almost any other framework
- ant scripts for running tests
- 1000's of existing tests, SAX, DOM
- EISMappingTestCases(6b) extends OXTestCase
- sub framework: org.eclipse.persistence.testing.oxm.OXTestCase (same tests are run multiple times)
- Tests are run once using a SAX parser
- Tests are run once using a DOM parser
- Tests are run once using document preservation
- Tests are run once using deployment XML as the source of the metadata
MOXy : JAXB
- framework: org.eclipse.persistence.testing.ox.jaxb
- extension of OXM tests for running JAXB tests
- ant scripts for running tests
- 1000's of existing tests
SDO
- framework: org.eclipse.persistence.sdo.testing
- extension of JUnit3 for running SDO tests
- ant scripts for running tests (not yet ported to eclipselink)
- 1000's of existing tests
- Sub Framework: Loading metadata from XML Schema
- Tests are run using generic DataObjects
- Tests are run generating static classes from the XML Schema
DBWS
Used by DBWS components, some misc. core tests
- extension of JUnit4: use @RunWith annotation
- Issues
- newest framework, fewest tests (~200)
- more 'future-proof' than JUnit3 which is now end-of-life