Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "EclipseLink/Development/Testing/DatabaseCertification"

(Concerns: Added two ideas regarding TCK compliance certification)
(Ideas: added suggestion to use TestNG)
 
Line 31: Line 31:
  
 
* In foundation, create a test suite in a similar way to the way we create the SRG test suite.
 
* In foundation, create a test suite in a similar way to the way we create the SRG test suite.
 +
* [http://testng.org TestNG] can run JUnit tests and has two functions that can help running subsets of tests:
 +
# TestNG's group feature allows test cases to be grouped (using annotations on test methods or classes). By specifying a particular group at test execution time, test cases not in that group are skipped. Any number of groups can be defined: ''platform-compliance'', ''LRG'', ''SRG'', ''JPA2.0'', ''Oracle10i-only'', etc.
 +
# TestNG's method selector allows you to implement a class or script that decides for each test method in the suite whether it should be run or skipped. The method selector could be implemented to accept a (platform specific) text file with test class/method names that need to be skipped for this platform. This solution prevents cluttering of test method implementations with selection code (as is the case now with ''if (getSession().getPlatform().isXxx()) return;'' statements in test code).
 
* ''Your idea here''
 
* ''Your idea here''
  

Latest revision as of 08:43, 8 September 2009

Under Construction

This page is a work in progress. Please feel free to add any ideas you think are helpful. The end goal is to provide a well-defined set of tests that can easily be run by anyone that wants to contribute a subclass of DatabasePlatform in order to determine if that DatabasePlatform can be certified as part of the EclipseLink product. Note: Actual certification of the DatabasePlatform should be handled through the incubation process: EclipseLink/Development/Incubator


Current State

There are two frameworks of interest

  1. JPA testing
  2. Foundation testing

JPA testing

The current set of JPA tests includes both basic tests and advanced tests. The way we certify database platforms is to run the whole suite and individually identify tests that should log warnings instead of failing.

Foundation testing

There is no simple way to certify a DatabasePlatform using the foundation testing framework. The SRG is too-small a test set and the LRG is too large a test-set.

End Goal

The end goal is to have a single build target that runs a test suite that is composed of the following:

  1. A set of JPA tests that is expected to run on every database we certify
  2. A set of foundation tests that is expected to run on every database we certify

Additionally, we will need a way to identify advanced tests that should run on a database-by-database basis

Ideas

  • In foundation, create a test suite in a similar way to the way we create the SRG test suite.
  • TestNG can run JUnit tests and has two functions that can help running subsets of tests:
  1. TestNG's group feature allows test cases to be grouped (using annotations on test methods or classes). By specifying a particular group at test execution time, test cases not in that group are skipped. Any number of groups can be defined: platform-compliance, LRG, SRG, JPA2.0, Oracle10i-only, etc.
  2. TestNG's method selector allows you to implement a class or script that decides for each test method in the suite whether it should be run or skipped. The method selector could be implemented to accept a (platform specific) text file with test class/method names that need to be skipped for this platform. This solution prevents cluttering of test method implementations with selection code (as is the case now with if (getSession().getPlatform().isXxx()) return; statements in test code).
  • Your idea here

Concerns

  • How do we certify JPA TCK compliance for database we cannot test on-site
    • Ask DB vendor to make a copy available for this purpose. As certification is only applied for once for a release, the overhead of running the TCK on extra DBs can be overseen. (The DB vendor might agree to having the database used for a longer period for continuous regression testing purposes.)
    • Ask DB vendor to make the database available off-site (remotely available). As certification is only applied for once for a release, the inconvenience of not having direct access to the database might be acceptable.

Back to the top