Difference between revisions of "EclipseLink/Development/Incubator/Extensions/DatabasePlatformPromotion"

Latest revision as of 09:58, 11 March 2010

The following is a proposal and has not been officially adopted by the EclipseLink community yet. Feedback is welcome either through the discussion page related to this wiki page or through the EclipseLink dev mailing list.

Database Platform Promotion

For a contributed DatabasePlatform to be included in EclipseLink it should pass some basic tests and provide detailed documentation about what can be expected to work on the contributed database platform.

Basic Test Suites required for inclusion

In order to be included in the product it must be possible to run our Core SRG tests without failures. In the case of the Core SRG if it is not possible to pass a test in the platform you are creating, a discussion should be started on the developer mailing list about excluding the test from the SRG. If the test cannot be excluded, the platform cannot be promoted into the product.

It is strongly recommended that you also run the JPA SRG without failures. In the case of the JPA SRG, failures can be avoided by either making tests pass, or assessing failures and determining that a test cannot be made to pass because of limitations on the platform and altering the tests to be excluded for that particular platform.

Test Execution

In general, tests will be executed by the contributer. The community trusts the contributer to provide accurate feedback about the success rates of tests and any failures. See "Documentation" below for information about how results are documented.

Currently the automated tests run in two places

1. Nightly tests run on MySQL at the Eclipse foundation
2. Nightly platform testing for different databases and servers run at Oracle

Restrictions on what software can run on Eclipse servers limits what can be run at Eclipse, and resourcing limits the types of tests Oracle is able to add to their suite. As a result, it is unlikely either of these test suites can have additional platforms added to them even if the software is made available. The nightly builds are made available on a download site and via maven and the committer team would be happy to help any one who wanted to leverage that to run automated tests at another location site.

Functionality Check List

In addition to passing SRG testing, a platform must be accompanied by documentation of what is expected to work. A check list of features must be included in the class comment that indicates that testing status of the following EclipseLink functionality. Testing Status could be one of the following - Succeeds, Succeeds with Limitations, Fails, Untested

In the case of bugs that either Succeed with Limitations or Fail, a bug should be entered in bugzilla documenting the failure. The goal of this bug is to document the issue for the EclipseLink community. The status of the bug should be set appropriately. For instance for driver issues that cannot be fixed within EclipseLink, but bug should be set to WONTFIX and an explanation should be provided. For issues that should be fixed in the future, the bug should be left open. Additionally, the EclipseLink team uses the bugzilla status whiteboard field to indicate which database a database-specific failure is against. That field should be updated. For any fields that require update that are not updatable by a non-committer, the commiter who is working on the database platform can make the updates.

The following is the list of features and the tests that should be run to test the functionality. To say the feature works, the accompanying test should run successfully.

• JPA Features
  • JPA SRG - JPA SRG test - this confirms the JPA SRG has been run.
  • Outer Join - test list under construction
  • Subquery - test list under construction
  • Native Sequences/Identitier fields - test list under construction
    • At least one of Sequnce/Identity should be supported for JPA compliance
  • JPA Bulk Update/Delete - test list under construction
  • Pessimistic Locking - test list under construction
  • Delimiters - test list under construction
  • JPA Queries - JPQL/JPA Criteria API - test list under construction

• JPA Optional Features
  • DDL Generation
    • The Core SRG covers basic DDL generation. Without, at least, this level of functionality, it will be difficult to run any tests successfully
    • JPA - DDLGenerationJUnitTestSuite to test DDL generation in JPA

• Extended EclipseLink functionality
  • Stored Procedure Calls - test list under construction
  • Stored Procedure Generation - test list under construction
  • Batch Reading - test list under construction
  • Batch Writing - test list under construction
  • First Result/Limit - test list under construction
    • First Result and Limit are required by JPA, but EclipseLink can manage these in memory. Some databases allow SQL-based First result and Limit and this testing is for the use of that feature
  • Auto Detection of database platform- test list under construction
  • Expression Framework - test list under construction
    • The expression framework provides a more-complete set of querying functionality than is currently available in JPA, it is a good idea to at least run these tests to determine what is missing from the platform

• Extended Database functionality
  • This is a list of any additional features added specifically for the database. (An example is if the database supports Spatial types and special support has been written)
    • Any tests in this list will be new tests written to exercise that functionality

Note: If a contributed DatabasePlatform replaces an existing DatabasePlatform, it must have a better success rate in our testing than the existing DatabasePlatform and pass all the tests the previous DataasePlatform passed

The following is a template for the class comment for a contributed platform:

/**
 *  FooPlatform
 *  <URL of wiki for databasePlatform>
 *  Test results: URL of wiki for test results
 *  Contributed by: FooCompany
 *  Contributed under bug: xxxxxxxx
 *  Initial tests run by: testerName
 *  @author authorName
 *  @since EclipseLink 1.0
 *
 *  Developed on FooDB version 1.0
 *  Initial SRG Passes on FoodDB version 1.1, 1.2, 2.0
 *
 *  Feature Testing
 *  ---------------
 * - DDL Generation - Succeeds
 * - Outer Join - Succeeds
 * - Subquery - Succeeds with Limitations (bug xxxxxx)
 * - Stored Procedure Calls - Untested
 * - Stored Procedure Generation - Untested
 * - Native Sequences/Identitier fields - Succeeds with Limitations (bug xxxxxy)
 * - JPA Bulk Update/Delete - Succeeds
 * - Batch Reading - Succeeds
 * - Batch Writing - Fails (bug xxxxxz)
 * - Pessimistic Locking - Succeeds with Limitations (bug xxxxxz)
 * - First Result/Limit - Succeeds
 * - Expression Framework - Succeeds with Limitations (bug xxxxxx)
 * - Delimiters - Succeeds
 * - Auto Detection - Succeeds
 *
 * Additional Functionality
 * ------------------------
 * - Additional Expression Operators: operator1, operator2
 * 
 * Limitations
 * -----------
 * - Subqueries with joins to the outer query are not supported
 * - Identity fields are not supported
 * - Pessimistic Locking with WAIT is not supported
 * - JPQL Simple Case is not supported
 * - Date Arithmetic is not supported in expressions
 */

Maintenance and Documentation

Test Documentation

Ideally, tests will be run at least once per major release and any new issues addressed. It will not always be possible for contributers to provide the resource to run the test at the times when EclipseLink is releasing. As a result, the web page describing the Platform with contain a full list of the tests run including the following:

• Date of the test run
• Version of EclipseLink
• Version of the Database
• JDBC driver information
• Special Database Configuration (e.g.non-default installed character sets, settings of system parameters, client side parameters etc. )
• Number of JPA SRG tests run and passed
• Number of Core SRG tests run and passed
• Information about both JPA and Core LRG runs including:
  • Number of tests run
  • Number of tests passed
  • Number and types of failures
• Any additional tests run and their results

Here is a sample set of documentation.

Maintenance

The EclipseLink committers endeavor to make DatabasePlatform changes in such a way as to avoid issues with DatabasePlatform. Generally, any new features added have implementation added to classes appropriate to the Databases where the new features exist and stubbed out methods added to DatabasePlatform. For contributed platforms, it will not be possible to run tests. The testing coverage is handled in two ways.

1. The platform testing that is run nightly covers a large number of database platforms and will likely turn up any inconsistencies and trigger bugs being filed for these issue
2. Any inconsistencies missed by those tests particular to a contributed database platform should have bugs filed for them when the tests are run against that database platform and the dev mailing list should be notified

End of Life

EclipseLink tries to remain backward compatible as long as possible. As a result, we very rarely remove a class. Instead of removing a class, each major release, we will review the test documentation for contributed platforms.

1. If the tests have not been run and the documentation updated either on the current major release, or the previous major release, the platform will be deprecated
2. If the tests have not been run and the documentation updated on the current major release, or the two previous major releases, the platform will be marked @obsolete
3. Once a platform is marked obsolete, it is a candidate for removal at the committer-group's discretion. Committers that intend to remove a platform should first inform the dev mailing list

Test Suite

The end goal of this exercise is to build up the list of tests that is expected to pass on a particular database and create a test suite that includes all those tests so the suite for a particular platform can be easily run. This goal will likely be deferred until we have added some contributed database platforms and have accumulated some knowledge about what works for best for the community.