Difference between revisions of "EclipseLink/Development/Testing/Frameworks"

Revision as of 09:33, 15 October 2007

Requirements

• simple unit-testing inside Eclipse IDE environment
• support nightly test runs
  • report pass/fail/ignore
  • post results to EclipseLink dashboard
  • compare results to 'known' good results
• must be able to run tests within multiple servers
• must be able to run tests against different databases
• must be able to run CTS nightly and publish CTS results
  • CTS cannot be run outside of Oracle; we can publish high-level 'pass/fail' results, but cannot say how many tests passed or failed, nor how many tests there are in the CTS
• each component has unique testing requirements so will need its own framework extensions (JPA, POJO, OX, JAXB, web-services)
• need to be able to run regression performance and concurrency tests
• need to be able to runs tests with spring

Current testing frameworks

Yikes! Nine Ten different testing frameworks and their extensions

Yikes what. Each component of EclipseLink has very different usages, supports different APIs and absolutely requires their own testing extensions, it would be incredible illogical not to reuse common testing functionality across the test cases. Most are based on JUnit3 so it is not as if they have nothing in common.

core/POJO: TopLink Testing Framework (1)

• ported to EclipseLink: org.eclipse.testing.framework.**
• legacy framework that has been migrated to be an extension of JUnit3
• provides UI runner with useful debugging functionality (SQL, Java, session properties, cache, logging, individual test running, storage of results in result database)
• UI can also run any JUnit3 test, and allows JPA JUnit3 tests to be run and be debuggable.
• provides advanced performance and concurrency tests which rely on result database storage
• tests are fully JUnit3 compatible and runnable in any JUnit3 runner including Eclipse
• provides command line runner and ant scripts (ant scripts not currently ported to EclipseLink)
• ant scripts for cross database testing (ant scripts not currently ported to EclipseLink)
• provides extended functionality for accessing POJO Session, table creation/population, object comparison, transactional tests, etc.
• 1000's of existing tests, POJO, cache-coordination, remote, POJO performance, JPA performance, POJO concurrent, MW integration, sessions XML, EIS, XDB, etc.

Issues

• not 'normal' JUnit3 tests
• too many tests to port to anything new
• contain functionality required for performance regression testing
• provide UI many developers are dependent on for debugging
• should not be adding any new tests for POJO API

core/server: Server Testing Framework (2)

• (not currently ported to EclipseLink)
• framework: oracle.toplink.testing.ejb.testframework
• extension of POJO framework for running in JEE servers
• not currently compatible with JUnit3
• command line runner, ant scripts for building, deployment, and running
• provides extended functionality for JTA transactions, JEE platform independence
• supports running of existing POJO tests inside the JEE server
• ant scripts for cross server testing, clustering
• 1000's of existing tests, CMP (not part of EclipseLink), SessionBeans, BMP (not part of EclipseLink), CMP performance (not part of EclipseLink), JTA, resource, EIS/JCA, class-loader, clustering, stress

Issues

• not compatible with JUnit3
• too many tests to port to anything new, but most tests are CMP and obsolete
• contain functionality required for performance regression testing in server
• should not be adding any new tests for POJO API or CMP

legacy CMP3/JPA (3)

• framework: org.eclipse.testing.jpa.EntityContainerTestBase/CMP3TestModel
• extension of POJO framework for JPA tests
• compatible with JUnit3
• 100's of tests

Issues

• not too many tests, should be port to new JPA JUnit3 framework

JUnit3

Used by many components, each with their own extensions to locate resources, setup sessions, etc.

JPA (4)

• framework: org.eclipse.testing.framework.junit.JUnitTestCase/framework.*/framework.server.*
• extension of JUnit3 for running JPA tests
• ant scripts for running tests with multiple weaving options (ant scripts not currently ported to eclipselink)
• recently extended to allow tests to run in JEE servers (not checked in yet)
• 1000's of existing tests, JPA, weaving, orm XML, SessionBean (not checked in yet)

Issues

• tests need to be migrated to allow running in JEE server
• tests need to verify result

Orajtst (5)

• not currently ported to eclipselink
• framework: oracle.toplink.testing.tests.internal.ejb, oraj.jar
• extension of JUnit3 and oraj framework
• ant scripts for running tests
• cannot current run outside of ADE environment nor from IDE
• 1000's of existing tests, clone of JPA JUnit tests ported to JEE, advanced JPA, spring

Issues

• clone of existing tests, causes major maintenance effort keeping in synch
• clone prevent news tests from being run in server
• oraj and ADE dependency
• once JPA JUnit tests are migrate to support running in server these can be removed

OXM (6)

• framework: org.eclipse.testing.ox.OXTestCase/*
• extension of JUnit3 for running OXM tests

• appears to be stateless; could be migrated to almost any other framework

• ant scripts for running tests (not yet ported to eclipselink)
• 1000's of existing tests, SAX, DOM

• EISMappingTestCases(6b) extends OXTestCase

JAXB (7)

• framework: org.eclipse.testing.ox.jaxb
• extension of OXM tests for running JAXB tests
• ant scripts for running tests (not yet ported to eclipselink)
• 1000's of existing tests

SDO (8)

• framework: org.eclipse.sdo.testing
• extension of JUnit3 for running SDO tests
• ant scripts for running tests (not yet ported to eclipselink)
• 1000's of existing tests

JUnit4 PB4 (9)

Used by DBWS components, some misc. core tests

• extension of JUnit4: use @RunWith annotation

Issues

• newest framework, fewest tests (~200)
• more 'future-proof' than JUnit3 which is now end-of-life