Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "EclipseLink/Development/Testing/Frameworks"

(→‎Current testing frameworks)
Line 1: Line 1:
−
__NOTOC__
+
EclipseLink has a significant number of tests, and supports many different test configurations, platforms, databases and parsers that need to be tested.
  
−
[[EclipseLink/Development/Testing/StrawManProposal| StrawMan Proposal]]
+
== Components ==
 +
Each EclipseLink component defines its own tests under <component>/eclipselink.<component>.test.  Each component has different testing requirements so uses slightly different methodologies.
  
−
== Requirements ==
+
The goals for test consistency across the components include the following:
−
 
+
* Each component defines an Eclipse test project in the root of the test directory.
−
* simple unit-testing inside Eclipse IDE environment
+
* The test project should include pre-configured launch targets for running the tests, ideally through the Eclipse JUnit runner.
−
* support nightly test runs
+
* All tests should be runnable through ant, and scripted in the <test>/build.xml.
−
* support running tests against eclipselink.jar
+
* The ant build file should define any test configuration in a <test>/test.properties file.
−
** report pass/fail/ignore
+
* The test.properties should be able to be overwritten in {user.home}/test.properties.
−
** post results to EclipseLink dashboard
+
* Ideally all tests should be written in JUnit 3.
−
** compare results to 'known' good results
+
* The ant test run should output results in html format in a <test>/reports directory.
−
* must be able to run tests within multiple servers
+
* The ant build file should define targets for test, test.srg and test.lrg.
−
* object-to-relational tests
+
* A root test.srg and test.lrg will be defined to run each srg/lrg in each component.
−
** must be able to run tests against different databases
+
−
* object-to-XML tests
+
−
** must be able to run tests against different XML parsers
+
−
* <s>must be able to run CTS nightly and publish CTS results</s>
+
−
** CTS cannot be run outside of Oracle; we can publish high-level 'pass/fail' results, but cannot say how many tests passed or failed, nor how many tests there are in the CTS
+
−
* each component has unique testing requirements so will need its own framework extensions (JPA, POJO, OX, JAXB, web-services)
+
−
* need to be able to run regression performance and concurrency tests
+
−
* need to be able to runs tests with spring
+
  
 
== Current testing frameworks ==
 
== Current testing frameworks ==
  
−
=== core/POJO: TopLink Testing Framework (1) ===
+
=== Core ===
 
* ported to EclipseLink: org.eclipse.testing.framework.**
 
* ported to EclipseLink: org.eclipse.testing.framework.**
 
* legacy framework that has been migrated to be an extension of JUnit3
 
* legacy framework that has been migrated to be an extension of JUnit3
Line 43: Line 36:
 
:* should not be adding any new tests for POJO API
 
:* should not be adding any new tests for POJO API
  
−
=== core/server: Server Testing Framework (2) ===
+
=== Core : server ===
 
* (not currently ported to EclipseLink)
 
* (not currently ported to EclipseLink)
 
* framework: oracle.toplink.testing.ejb.testframework
 
* framework: oracle.toplink.testing.ejb.testframework
Line 60: Line 53:
 
:* should not be adding any new tests for POJO API or CMP
 
:* should not be adding any new tests for POJO API or CMP
  
−
=== legacy CMP3/JPA (3) ===
+
=== JPA : legacy ===
 
* framework: org.eclipse.testing.jpa.EntityContainerTestBase/CMP3TestModel
 
* framework: org.eclipse.testing.jpa.EntityContainerTestBase/CMP3TestModel
 
* extension of POJO framework for JPA tests
 
* extension of POJO framework for JPA tests
Line 69: Line 62:
 
:* not too many tests, should be port to new JPA JUnit3 framework
 
:* not too many tests, should be port to new JPA JUnit3 framework
  
−
=== JUnit3 ===
+
==== JPA ====  
−
'''Used by many components, each with their own extensions to locate resources, setup sessions, etc.'''
+
−
 
+
−
==== JPA (4) ====  
+
 
* framework: org.eclipse.testing.framework.junit.JUnitTestCase/framework.*/framework.server.*
 
* framework: org.eclipse.testing.framework.junit.JUnitTestCase/framework.*/framework.server.*
 
* extension of JUnit3 for running JPA tests
 
* extension of JUnit3 for running JPA tests
Line 83: Line 73:
 
:* tests need to verify result
 
:* tests need to verify result
  
−
==== Orajtst (5) ====
+
==== JPA : Orajtst ====
  
 
* not currently ported to eclipselink
 
* not currently ported to eclipselink
Line 98: Line 88:
 
:* once JPA JUnit tests are migrate to support running in server these can be removed
 
:* once JPA JUnit tests are migrate to support running in server these can be removed
  
−
==== OXM (6) ====
+
==== MOXy : OXM ====
 
* framework: org.eclipse.testing.ox.XMLTestCase/*
 
* framework: org.eclipse.testing.ox.XMLTestCase/*
 
* extension of JUnit3 for running OXM tests
 
* extension of JUnit3 for running OXM tests
Line 111: Line 101:
 
** Tests are run once using deployment XML as the source of the metadata
 
** Tests are run once using deployment XML as the source of the metadata
  
−
==== JAXB (7) ====
+
==== MOXy : JAXB ====
 
* framework: org.eclipse.testing.ox.jaxb
 
* framework: org.eclipse.testing.ox.jaxb
 
* extension of OXM tests for running JAXB tests
 
* extension of OXM tests for running JAXB tests
Line 117: Line 107:
 
* 1000's of existing tests
 
* 1000's of existing tests
  
−
==== SDO (8) ====
+
==== SDO ====
 
* framework: org.eclipse.sdo.testing
 
* framework: org.eclipse.sdo.testing
 
* extension of JUnit3 for running SDO tests
 
* extension of JUnit3 for running SDO tests
Line 126: Line 116:
 
** Tests are run generating static classes from the XML Schema
 
** Tests are run generating static classes from the XML Schema
  
−
=== JUnit4 PB4 (9) ===
+
=== DBWS ===
 
Used by DBWS components, some misc. core tests
 
Used by DBWS components, some misc. core tests
 
* extension of JUnit4: use @RunWith annotation
 
* extension of JUnit4: use @RunWith annotation

Revision as of 12:08, 19 December 2007

EclipseLink has a significant number of tests, and supports many different test configurations, platforms, databases and parsers that need to be tested.

Components

Each EclipseLink component defines its own tests under <component>/eclipselink.<component>.test. Each component has different testing requirements so uses slightly different methodologies.

The goals for test consistency across the components include the following:

  • Each component defines an Eclipse test project in the root of the test directory.
  • The test project should include pre-configured launch targets for running the tests, ideally through the Eclipse JUnit runner.
  • All tests should be runnable through ant, and scripted in the <test>/build.xml.
  • The ant build file should define any test configuration in a <test>/test.properties file.
  • The test.properties should be able to be overwritten in {user.home}/test.properties.
  • Ideally all tests should be written in JUnit 3.
  • The ant test run should output results in html format in a <test>/reports directory.
  • The ant build file should define targets for test, test.srg and test.lrg.
  • A root test.srg and test.lrg will be defined to run each srg/lrg in each component.

Current testing frameworks

Core

  • ported to EclipseLink: org.eclipse.testing.framework.**
  • legacy framework that has been migrated to be an extension of JUnit3
  • provides UI runner with useful debugging functionality (SQL, Java, session properties, cache, logging, individual test running, storage of results in result database)
  • UI can also run any JUnit3 test, and allows JPA JUnit3 tests to be run and be debuggable.
  • provides advanced performance and concurrency tests which rely on result database storage
  • tests are fully JUnit3 compatible and runnable in any JUnit3 runner including Eclipse
  • provides command line runner and ant scripts (ant scripts not currently ported to EclipseLink)
  • ant scripts for cross database testing (ant scripts not currently ported to EclipseLink)
  • provides extended functionality for accessing POJO Session, table creation/population, object comparison, transactional tests, etc.
  • 1000's of existing tests, POJO, cache-coordination, remote, POJO performance, JPA performance, POJO concurrent, MW integration, sessions XML, EIS, XDB, etc.
Issues
  • not 'normal' JUnit3 tests
  • too many tests to port to anything new
  • contain functionality required for performance regression testing
  • provide UI many developers are dependent on for debugging
  • should not be adding any new tests for POJO API

Core : server

  • (not currently ported to EclipseLink)
  • framework: oracle.toplink.testing.ejb.testframework
  • extension of POJO framework for running in JEE servers
  • not currently compatible with JUnit3
  • command line runner, ant scripts for building, deployment, and running
  • provides extended functionality for JTA transactions, JEE platform independence
  • supports running of existing POJO tests inside the JEE server
  • ant scripts for cross server testing, clustering
  • 1000's of existing tests, CMP (not part of EclipseLink), SessionBeans, BMP (not part of EclipseLink), CMP performance (not part of EclipseLink), JTA, resource, EIS/JCA, class-loader, clustering, stress
Issues
  • not compatible with JUnit3
  • too many tests to port to anything new, but most tests are CMP and obsolete
  • contain functionality required for performance regression testing in server
  • should not be adding any new tests for POJO API or CMP

JPA : legacy

  • framework: org.eclipse.testing.jpa.EntityContainerTestBase/CMP3TestModel
  • extension of POJO framework for JPA tests
  • compatible with JUnit3
  • 100's of tests
Issues
  • not too many tests, should be port to new JPA JUnit3 framework

JPA

  • framework: org.eclipse.testing.framework.junit.JUnitTestCase/framework.*/framework.server.*
  • extension of JUnit3 for running JPA tests
  • ant scripts for running tests with multiple weaving options (ant scripts not currently ported to eclipselink)
  • recently extended to allow tests to run in JEE servers (not checked in yet)
  • 1000's of existing tests, JPA, weaving, orm XML, SessionBean (not checked in yet)
Issues
  • tests need to be migrated to allow running in JEE server
  • tests need to verify result

JPA : Orajtst

  • not currently ported to eclipselink
  • framework: oracle.toplink.testing.tests.internal.ejb, oraj.jar
  • extension of JUnit3 and oraj framework
  • ant scripts for running tests
  • cannot current run outside of ADE environment nor from IDE
  • 1000's of existing tests, clone of JPA JUnit tests ported to JEE, advanced JPA, spring
Issues
  • clone of existing tests, causes major maintenance effort keeping in synch
  • clone prevent news tests from being run in server
  • oraj and ADE dependency
  • once JPA JUnit tests are migrate to support running in server these can be removed

MOXy : OXM

  • framework: org.eclipse.testing.ox.XMLTestCase/*
  • extension of JUnit3 for running OXM tests
  • appears to be stateless; could be migrated to almost any other framework
  • ant scripts for running tests (not yet ported to eclipselink)
  • 1000's of existing tests, SAX, DOM
  • EISMappingTestCases(6b) extends OXTestCase
  • sub framework: org.eclipse.testing.oxm.OXTestCase (same tests are run multiple times)
    • Tests are run once using a SAX parser
    • Tests are run once using a DOM parser
    • Tests are run once using document preservation
    • Tests are run once using deployment XML as the source of the metadata

MOXy : JAXB

  • framework: org.eclipse.testing.ox.jaxb
  • extension of OXM tests for running JAXB tests
  • ant scripts for running tests (not yet ported to eclipselink)
  • 1000's of existing tests

SDO

  • framework: org.eclipse.sdo.testing
  • extension of JUnit3 for running SDO tests
  • ant scripts for running tests (not yet ported to eclipselink)
  • 1000's of existing tests
  • Sub Framework: Loading metadata from XML Schema
    • Tests are run using generic DataObjects
    • Tests are run generating static classes from the XML Schema

DBWS

Used by DBWS components, some misc. core tests

  • extension of JUnit4: use @RunWith annotation
Issues
  • newest framework, fewest tests (~200)
  • more 'future-proof' than JUnit3 which is now end-of-life
Retrieved from "https://wiki.eclipse.org/index.php?title=EclipseLink/Development/Testing/Frameworks&oldid=69852"

Back to the top