Difference between revisions of "EclipseLink/Development/Testing/Frameworks"

Revision as of 11:09, 10 October 2007

Requirements

• simple unit-testing inside Eclipse IDE environment
• support nightly test runs
  • report pass/fail/ignore
  • post results to EclipseLink dashboard
  • compare results to 'known' good results
• must be able to run tests within multiple servers
• must be able to run tests against different databases
• must be able to run CTS nightly and publish CTS results
• each component has unique testing requirements so will need its own framework extensions (JPA, POJO, OX, JAXB, web-services)
• need to be able to run regression performance and concurrency tests
• need to be able to runs tests with spring

Current testing frameworks

Yikes! Nine different testing frameworks

core/POJO: TopLink Testing Framework (1)

• ported to EclipseLink: org.eclipse.testing.framework.**
• legacy framework that has been migrated to be an extension of JUnit3
• provides UI runner with useful debugging functionality (SQL, Java, session properties, cache, logging, individual test running, storage of results in result database)
• UI can also run any JUnit3 test, and allows JPA JUnit3 tests to be run and be debuggable.
• provides advanced performance and concurrency tests which rely on result database storage
• tests are fully JUnit3 compatible and runnable in any JUnit3 runner including Eclipse
• provides command line runner and ant scripts (ant scripts not currently ported to EclipseLink)
• ant scripts for cross database testing (ant scripts not currently ported to EclipseLink)
• provides extended functionality for accessing POJO Session, table creation/population, object comparison, transactional tests, etc.
• 1000's of existing tests, POJO, cache-coordination, remote, POJO performance, JPA performance, POJO concurrent, MW integration, sessions XML, EIS, XDB, etc.

Issues

• not 'normal' JUnit3 tests
• too many tests to port to anything new
• contain functionality required for performance regression testing
• provide UI many developers are dependent on for debugging
• should not be adding any new tests for POJO API

core/server: Server Testing Framework (2)

• (not currently ported to EclipseLink)
• framework: oracle.toplink.testing.ejb.testframework
• extension of POJO framework for running in JEE servers
• not currently compatible with JUnit3
• command line runner, ant scripts for building, deployment, and running
• provides extended functionality for JTA transactions, JEE platform independence
• supports running of existing POJO tests inside the JEE server
• ant scripts for cross server testing, clustering
• 1000's of existing tests, CMP (not part of EclipseLink), SessionBeans, BMP (not part of EclipseLink), CMP performance (not part of EclipseLink), JTA, resource, EIS/JCA, class-loader, clustering, stress

Issues

• not compatible with JUnit3
• too many tests to port to anything new, but most tests are CMP and obsolete
• contain functionality required for performance regression testing in server
• should not be adding any new tests for POJO API or CMP

legacy CMP3/JPA (3)

• framework: org.eclipse.testing.jpa.EntityContainerTestBase/CMP3TestModel
• extension of POJO framework for JPA tests
• compatible with JUnit3
• 100's of tests

Issues

• not too many tests, should be port to new JPA JUnit3 framework

JUnit3

Used by many components, each with their own extensions to locate resources, setup sessions, etc.

JPA (4)

• framework: org.eclipse.testing.framework.junit.JUnitTestCase/framework.*/framework.server.*
• extension of JUnit3 for running JPA tests
• ant scripts for running tests with multiple weaving options (ant scripts not currently ported to eclipselink)
• recently extended to allow tests to run in JEE servers (not checked in yet)
• 1000's of existing tests, JPA, weaving, orm XML, SessionBean (not checked in yet)

Issues

• tests need to be migrated to allow running in JEE server
• tests need to verify result

Orajtst (5)

• not currently ported to eclipselink
• framework: oracle.toplink.testing.tests.internal.ejb, oraj.jar
• extension of JUnit3 and oraj framework
• ant scripts for running tests
• cannot current run outside of ADE environment nor from IDE
• 1000's of existing tests, clone of JPA JUnit tests ported to JEE, advanced JPA, spring

Issues

• clone of existing tests, causes major maintenance effort keeping in synch
• clone prevent news tests from being run in server
• oraj and ADE dependency
• once JPA JUnit tests are migrate to support running in server these can be removed

OXM (6)

• framework: org.eclipse.testing.ox.OXTestCase/*
• extension of JUnit3 for running OXM tests
• ant scripts for running tests (not yet ported to eclipselink)
• 1000's of existing tests, SAX, DOM

JAXB (7)

• framework: org.eclipse.testing.ox.jaxb
• extension of OXM tests for running JAXB tests
• ant scripts for running tests (not yet ported to eclipselink)
• 1000's of existing tests

SDO (8)

• framework: org.eclipse.sdo.testing
• extension of JUnit3 for running SDO tests
• ant scripts for running tests (not yet ported to eclipselink)
• 1000's of existing tests

JUnit4 PB4 (9)

Used by DBWS components, some misc. core tests

• extension of JUnit4: use @RunWith annotation

Issues

• newest framework, fewest tests (~200)
• more 'future-proof' than JUnit3 which is now end-of-life