Skip to main content
Jump to: navigation, search

Difference between revisions of "EclipseLink/DesignDocs/362899"

(New page: <div style="margin:5px;float:right;border:1px solid #000000;padding:5px">__TOC__</div> = Design Specification: Support NoSQL Databases = [https://bugs.eclipse.org/bugs/show_bug.cgi?id=362...)
 
Line 26: Line 26:
 
* Hadoop
 
* Hadoop
 
* Google BigTable
 
* Google BigTable
 +
* Cassandra
  
 
These databases offer alternative APIs to JDBC and either do not support querying, or support alternative query languages to SQL.
 
These databases offer alternative APIs to JDBC and either do not support querying, or support alternative query languages to SQL.
  
 
You could also include legacy hierarchical and non-relational databases, object databases and XML databases in the "NoSQL" category.
 
You could also include legacy hierarchical and non-relational databases, object databases and XML databases in the "NoSQL" category.
 +
 +
It is desirable for EclipseLink to support persistence and specifically JPA access to NoSQL databases.
  
 
EclipseLink currently provides persistence support for relational databases through JDBC, and non-relational datasources through JCA.
 
EclipseLink currently provides persistence support for relational databases through JDBC, and non-relational datasources through JCA.
 +
This support requires a JCA resource adapter/driver for the database.
 +
Most NoSQL databases do not provide a standard driver, although some may, and some do provide even JDBC drivers.
 +
There are also third parties that provide JDBC and JCA driver to a wide range of datasources, including some NoSQL databases.
  
It is desirable for EclipseLink to support persistence and specifically JPA access to NoSQL databases.
+
See:
 +
* [https://github.com/erh/mongo-jdbc Mongo JDBC Driver]
 +
* [http://www.datadirect.com/supported_data_sources/index.html DataDirect]
 +
* [http://www.attunity.com/attunity_connect_application_adapters Attunity JCA] [http://www.attunity.com/attunity_connect_jdbc_drivers JDBC]
  
= Concepts =
+
Users connecting EclipseLink to NoSQL databases have several options:
 +
* If they have a JDBC driver available they can connect using EclipseLink's relational support, creating their own DatabasePlatform in EclipseLink will provide enhanced functionality and ease of use.
 +
* If they have a JCA driver available they can connect using EclipseLink's EIS support, creating their own EISPlatform in EclipseLink will provide enhanced functionality and ease of use.
 +
* They can implement their own JCA (or JDBC) adapter for their database to enable usage of EclipseLink's EIS support.
  
 +
EclipseLink currently provides several JCA adapters to non-relational datasources.
 +
These include:
 +
* JMS
 +
* Oracle AQ
 +
* XML file adapter
  
= Requirements =
+
EclipseLink should provide a platform and adapter to popular NoSQL databases, similar to the platforms provided for popular relational databases.
  
 +
EclipseLink's EIS support currently has several limitations that must be resolved to support NoSQL databases.
 +
* No JPA annotation or XML support is currently available for EIS descriptors, mappings, and queries.
 +
* Although hooks exists, no current EIS adapters provide Expression (JPQL/Criteria) translation into non-SQL query languages or APIs.
 +
* The JPA API has not been tested with EIS descriptors and datasources.
  
= Design Constraints =
+
= Concepts =
 +
* NoSQL - see [http://en.wikipedia.org/wiki/NoSQL NoSQL - Wikipedia]
 +
* Hierarchical data - data that is nested like XML, not flat like relational data, most NoSQL data is hierarchical.
 +
* EIS - Enterprise Information System, includes non-relational databases, gateways, transaction monitors, and applications. (VSAM, ADA, CICS, IMS, SAP, Salesforce, MongoDB, BigTable).
  
 +
= Requirements =
 +
* Support JPA access to EIS and NoSQL databases.
 +
* Support EIS JPA annotations and XML.
 +
* Support JPQL subset (superset?) for NoSQL databases that support querying.
 +
* Support mapping XML/JSON data from NoSQL databases.
 +
* Support EISPlatforms and adapters for key NoSQL databases:
 +
** Oracle NoSQL
 +
** MongoDB
 +
** others?
 +
 +
= Design Constraints =
 +
* Some NoSQL databases do not support querying.
 +
* Transaction semantics may be different, or even not exist in some NoSQL databases.
 +
* Data and its access is non-standard and may require different types of mappings, queries or data access.
 +
* NoSQL database may be schema free, and may support storing of dynamic data.
 +
* NoSQL database provide non standard Java APIs, our adapters will require compile dependencies with these APIs.
  
 
= Functionality =
 
= Functionality =
 +
* Support EIS annotations and XML in JPA.
 +
* Support JPA auto-mapping and defaulting.
 +
* Support JCA adapter for Oracle NoSQL.
 +
* Support JCA adapter for MongoDB.
 +
  
 
= Testing =
 
= Testing =
Line 72: Line 117:
 
|-
 
|-
 
| 1  
 
| 1  
|
+
| Which NoSQL databases should be supported?  Oracle NoSQL, MongoDB, Hadoop, BigTable, ...?
|  
+
|-
 +
| 2
 +
| Where should JCA adapters be developed?
 +
|-
 +
| 3
 +
| How to resolve compile dependencies with third party APIs and drivers?
 
|-
 
|-
 
|}
 
|}
Line 84: Line 134:
 
! Description / Notes  
 
! Description / Notes  
 
! Decision
 
! Decision
|-
+
| Oracle NoSQL and MongoDB to start?
 +
|-  
 
|}
 
|}
  
 
= Future Considerations =
 
= Future Considerations =
 +
* Hadoop
 +
* Google BigTable
 +
* Cassandra
 +
* Oracle Coherence (as a datasource instead of a cache)

Revision as of 15:04, 8 December 2011

Design Specification: Support NoSQL Databases

ER 362899

Feedback

Document History

Date Author Version Description & Notes
2011-12-08 James 0.1 Draft

Project overview

There are several "NoSQL" databases that are non-relational. These databases include:

  • Oracle NoSQL
  • MonogoDB
  • Hadoop
  • Google BigTable
  • Cassandra

These databases offer alternative APIs to JDBC and either do not support querying, or support alternative query languages to SQL.

You could also include legacy hierarchical and non-relational databases, object databases and XML databases in the "NoSQL" category.

It is desirable for EclipseLink to support persistence and specifically JPA access to NoSQL databases.

EclipseLink currently provides persistence support for relational databases through JDBC, and non-relational datasources through JCA. This support requires a JCA resource adapter/driver for the database. Most NoSQL databases do not provide a standard driver, although some may, and some do provide even JDBC drivers. There are also third parties that provide JDBC and JCA driver to a wide range of datasources, including some NoSQL databases.

See:

Users connecting EclipseLink to NoSQL databases have several options:

  • If they have a JDBC driver available they can connect using EclipseLink's relational support, creating their own DatabasePlatform in EclipseLink will provide enhanced functionality and ease of use.
  • If they have a JCA driver available they can connect using EclipseLink's EIS support, creating their own EISPlatform in EclipseLink will provide enhanced functionality and ease of use.
  • They can implement their own JCA (or JDBC) adapter for their database to enable usage of EclipseLink's EIS support.

EclipseLink currently provides several JCA adapters to non-relational datasources. These include:

  • JMS
  • Oracle AQ
  • XML file adapter

EclipseLink should provide a platform and adapter to popular NoSQL databases, similar to the platforms provided for popular relational databases.

EclipseLink's EIS support currently has several limitations that must be resolved to support NoSQL databases.

  • No JPA annotation or XML support is currently available for EIS descriptors, mappings, and queries.
  • Although hooks exists, no current EIS adapters provide Expression (JPQL/Criteria) translation into non-SQL query languages or APIs.
  • The JPA API has not been tested with EIS descriptors and datasources.

Concepts

  • NoSQL - see NoSQL - Wikipedia
  • Hierarchical data - data that is nested like XML, not flat like relational data, most NoSQL data is hierarchical.
  • EIS - Enterprise Information System, includes non-relational databases, gateways, transaction monitors, and applications. (VSAM, ADA, CICS, IMS, SAP, Salesforce, MongoDB, BigTable).

Requirements

  • Support JPA access to EIS and NoSQL databases.
  • Support EIS JPA annotations and XML.
  • Support JPQL subset (superset?) for NoSQL databases that support querying.
  • Support mapping XML/JSON data from NoSQL databases.
  • Support EISPlatforms and adapters for key NoSQL databases:
    • Oracle NoSQL
    • MongoDB
    • others?

Design Constraints

  • Some NoSQL databases do not support querying.
  • Transaction semantics may be different, or even not exist in some NoSQL databases.
  • Data and its access is non-standard and may require different types of mappings, queries or data access.
  • NoSQL database may be schema free, and may support storing of dynamic data.
  • NoSQL database provide non standard Java APIs, our adapters will require compile dependencies with these APIs.

Functionality

  • Support EIS annotations and XML in JPA.
  • Support JPA auto-mapping and defaulting.
  • Support JCA adapter for Oracle NoSQL.
  • Support JCA adapter for MongoDB.


Testing

API

Native API

Examples:

 

Config files

Documentation

Need to document our BNF, example queries and support beyond JPA 2.0/2.1 in query section.

Open Issues

Issue # Owner Description / Notes
1 Which NoSQL databases should be supported? Oracle NoSQL, MongoDB, Hadoop, BigTable, ...?
2 Where should JCA adapters be developed?
3 How to resolve compile dependencies with third party APIs and drivers?

Decisions

Issue Description / Notes Decision Oracle NoSQL and MongoDB to start?

Future Considerations

  • Hadoop
  • Google BigTable
  • Cassandra
  • Oracle Coherence (as a datasource instead of a cache)

Copyright © Eclipse Foundation, Inc. All Rights Reserved.