Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "EclipseLink/Development/DBWS/Overview"
(→Metadata Generation) |
(→JPA Metadata Generation) |
||
(58 intermediate revisions by the same user not shown) | |||
Line 3: | Line 3: | ||
The purpose of this document is to provide a high-level overview of the design and runtime portions of the DBWS component. In addition, an outline of relevant changes from version to version will be provided. The EclipseLink DBWS documentation can be found here: http://www.eclipse.org/eclipselink/documentation/2.5/dbws/toc.htm. | The purpose of this document is to provide a high-level overview of the design and runtime portions of the DBWS component. In addition, an outline of relevant changes from version to version will be provided. The EclipseLink DBWS documentation can be found here: http://www.eclipse.org/eclipselink/documentation/2.5/dbws/toc.htm. | ||
− | === Design time Component === | + | === Design-time Component === |
DBWS builder is responsible for processing input from the user (or tooling, such as JDeveloper), then scraping the database DDL based on this input. The information retrieved from the database is used to construct a meta-model, which is in turn used to generate a number of artifacts required by the DBWS runtime in order to service SOAP message requests. | DBWS builder is responsible for processing input from the user (or tooling, such as JDeveloper), then scraping the database DDL based on this input. The information retrieved from the database is used to construct a meta-model, which is in turn used to generate a number of artifacts required by the DBWS runtime in order to service SOAP message requests. | ||
Line 37: | Line 37: | ||
eclipselink-dbws-schema.xsd | eclipselink-dbws-schema.xsd | ||
eclipselink-dbws.wsdl | eclipselink-dbws.wsdl | ||
− | swaref.xsd -- optionally generated | + | swaref.xsd -- optionally generated if swaRef is being utilized |
− | ==== Basic Flow ==== | + | ==== Basic Builder Flow ==== |
The basic flow of DBWS builder is as follows: | The basic flow of DBWS builder is as follows: | ||
− | # | + | # Scrape the database DDL based on user input and build meta-model (<code>org.eclipse.persistence.tools.oracleddl.metadata</code>) |
# Generate OR/OX projects based on meta-model | # Generate OR/OX projects based on meta-model | ||
# Generate XML schema based on OX project | # Generate XML schema based on OX project | ||
# Generate XR Service Model based on OR project | # Generate XR Service Model based on OR project | ||
− | # Generate WSDL based on | + | # Generate WSDL based on Service Model |
# Generate Web app and WebService implmentation | # Generate Web app and WebService implmentation | ||
# Generate JAXB and JPA metadata files based on OR/OX projects | # Generate JAXB and JPA metadata files based on OR/OX projects | ||
Line 121: | Line 121: | ||
procOpModel.setProcedurePattern("EMP%"); | procOpModel.setProcedurePattern("EMP%"); | ||
</source> | </source> | ||
+ | |||
+ | ==== Command-line Utility ==== | ||
+ | The EclipseLink install comes with a command-line DBWS builder utility (.cmd and .sh). This allows the DBWS builder to be configured and executed easily. The utility is in <code><eclipselink_home>\utils\dbws</code>. The JDBC driver must be on the classpath. The utility looks for a <code>DRIVER_CLASSPATH</code> variable for the .jar location; this variable can be set or simply edited in the .cmd/.sh file to point to the correct location, i.e. <code>set CLASSPATH=%CLASSPATH%;<work_home>\extension.oracle.lib.external\ojdbc6.jar</code>. | ||
+ | |||
+ | A builder XML configuration file is required, and the staging directory must be set. Following is an example of how to execute the builder (.war packaging with WebLogic server target) using the Windows command script: | ||
+ | <code>dbwsbuilder.cmd -builderFile employee.xml -stageDir stage -packageAs wls employee.war</code> | ||
==== Simple XML Format ==== | ==== Simple XML Format ==== | ||
Line 143: | Line 149: | ||
==== Shadow Types & Anonymous PL/SQL Generation ==== | ==== Shadow Types & Anonymous PL/SQL Generation ==== | ||
− | JDBC does not support transporting Oracle PL/SQL types. In order for DBWS to handle PL/SQL input and output arguments, two tasks are required. First, for each PL/SQL type | + | JDBC does not support transporting Oracle PL/SQL types. In order for DBWS to handle PL/SQL input and output arguments, two tasks are required. First, for each PL/SQL type there must be an equivalent JDBC type. DBWS will generate 'shadow' create/drop DDL for each PL/SQL type; this DDL can be executed on the DB to add and remove these required types. The second thing that is required are PL/SQL functions that covert PL/SQL types to equivalent JDBC types and vice-versa. To support this, anonymous PL/SQL is generated and executed on the database at runtime. |
− | + | ===== Shadow Type Example ===== | |
+ | If a given package <code>SOMEPKG</code> has a PL/SQL record <code>ORECORD</code> defined as follows: | ||
+ | <css> | ||
+ | .source-xml {padding:4px;border:1px solid black;} | ||
+ | </css> | ||
+ | <source lang="xml"> | ||
+ | CREATE OR REPLACE PACKAGE SOMEPKG AS | ||
+ | TYPE ORECORD IS RECORD ( | ||
+ | O1 VARCHAR2(10), | ||
+ | O2 DECIMAL(7,2) | ||
+ | ); | ||
+ | END PACKAGE SOMEPKG; | ||
+ | </source> | ||
+ | |||
+ | DBWS requires that an equivalant JDBC shadow type also exists at the TOPLEVEL (not in a package), i.e.: | ||
+ | <css> | ||
+ | .source-xml {padding:4px;border:1px solid black;} | ||
+ | </css> | ||
+ | <source lang="xml"> | ||
+ | CREATE OR REPLACE TYPE SOMEPKG_ORECORD AS OBJECT ( | ||
+ | O1 VARCHAR2(10), | ||
+ | O2 DECIMAL(7,2) | ||
+ | ) | ||
+ | </source> | ||
+ | Note that the expected naming convention for the shadow JDBC type is <code>packagename_plsqlname</code>. | ||
+ | |||
+ | ===== ShadowDDLGenerator ===== | ||
+ | DBWS builder generates required shadow create and drop DDL such that the user doesn't have to create the required DDL manually. Following is an example of using the builder-generated shadow DDL: | ||
+ | <css> | ||
+ | .source-java {padding:4px;border:1px solid black;} | ||
+ | </css> | ||
+ | <source lang="java"> | ||
+ | Connection conn = buildSQLConnection(); | ||
+ | PreparedStatement pStmt; | ||
+ | |||
+ | // execute shadow type ddl to generate JDBC equivalents of PL/SQL types | ||
+ | for (String ddl : builder.getTypeDDL()) { | ||
+ | try { | ||
+ | pStmt = conn.prepareStatement(ddl); | ||
+ | pStmt.execute(); | ||
+ | } catch (SQLException e) { | ||
+ | // handle exception | ||
+ | } | ||
+ | } | ||
+ | </source> | ||
+ | Note that similar code is used to execute the drop DDL. | ||
+ | |||
+ | ===== Anonymous PL/SQL Code Generation ===== | ||
+ | Considering a PL/SQL stored procedure that returns a new <code>ORECORD</code>: | ||
+ | <source lang="xml"> | ||
+ | CREATE OR REPLACE PACKAGE SOMEPKG AS | ||
+ | PROCEDURE GETNEWREC(NEWREC OUT ORECORD); | ||
+ | END PACKAGE SOMEPKG; | ||
+ | </source> | ||
+ | |||
+ | The following anonymous PL/SQL will be generated and executed against the database at runtime: | ||
+ | <source lang="xml"> | ||
+ | DECLARE | ||
+ | NEWRECTARGET SOMEPKG.ORECORD; <!-- PL/SQL record declaration --> | ||
+ | FUNCTION EL_PL2SQL_0(aPlsqlItem SOMEPKG.ORECORD) <!-- Converts the new PL/SQL record to it's shadow JDBC type --> | ||
+ | RETURN SOMEPKG_ORECORD IS | ||
+ | aSqlItem SOMEPKG_ORECORD; | ||
+ | BEGIN | ||
+ | aSqlItem := SOMEPKG_ORECORD(NULL, NULL); | ||
+ | aSqlItem.N1 := aPlsqlItem.N1; | ||
+ | aSqlItem.N2 := aPlsqlItem.N2; | ||
+ | RETURN aSqlItem; | ||
+ | END EL_PL2SQL_0; | ||
+ | BEGIN | ||
+ | SOMEPKG.GETNEWREC(NEWREC=>NEWRECTARGET); <!-- The OUT argument 'NEWREC' from the PL/SQL procedure is passed to --> | ||
+ | :1 := EL_PL2SQL_0(NEWRECTARGET); <!-- the anonymous function to be converted to the shadow JDBC type --> | ||
+ | END; | ||
+ | </source> | ||
+ | |||
+ | Note that this generation occurs in the <code>prepareInternal</code> method of <code>PLSQLStoredProcedureCall</code>. | ||
==== DDL Parser ==== | ==== DDL Parser ==== | ||
Line 160: | Line 240: | ||
* http://eclipse-javacc.sourceforge.net/ | * http://eclipse-javacc.sourceforge.net/ | ||
− | ==== | + | ===== Eclipse Configuration ===== |
− | + | For running in Eclipse, the DDL parser project requires the Maven (m2e) plugin and the Tycho Configurator. Once the m2e plugin is installed, Tycho can be setup by selecting Preferences > Maven > Discovery > Open Catalog, then selecting the Tycho Configurator. | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ==== XML Schema Generation ==== | |
− | + | DBWS builder utilizes the <code>org.eclipse.persistence.internal.oxm.schema.SchemaModelGenerator</code> class to generate the XML schema. The types generated are referenced in JAXB metadata and the DBWS service file. | |
− | + | ||
− | + | ||
− | + | ||
− | + | If DBWS builder is configured as follows for the <code>GETNEWREC</code> PL/SQL stored procedure above: | |
− | + | <source lang="java"> | |
− | + | procOpModel = new PLSQLProcedureOperationModel(); | |
− | + | procOpModel.setName("GetNewRecordTest"); | |
− | + | procOpModel.setCatalogPattern("SOMEPKG"); | |
+ | procOpModel.setProcedurePattern("GETNEWREC"); | ||
+ | </source> | ||
− | + | The following XML schema will be generated: | |
− | + | <source lang="xml"> | |
− | + | <?xml version="1.0" encoding="UTF-8"?> | |
− | + | <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" | |
+ | targetNamespace="urn:PLSQLRecord" | ||
+ | xmlns="urn:PLSQLRecord" | ||
+ | elementFormDefault="qualified"> | ||
+ | <xsd:complexType name="somepkg_orecordType"> | ||
+ | <xsd:sequence> | ||
+ | <xsd:element name="o1" type="xsd:string" minOccurs="0" nillable="true"/> | ||
+ | <xsd:element name="o2" type="xsd:decimal" minOccurs="0" nillable="true"/> | ||
+ | </xsd:sequence> | ||
+ | </xsd:complexType> | ||
+ | <xsd:element name="somepkg_orecordType" type="somepkg_orecordType"/> | ||
+ | </xsd:schema> | ||
+ | </source> | ||
− | + | ==== XR Service File Generation ==== | |
− | + | The service file contains query information required by the XR component at runtime. Database queries are looked up on the OR session based on the named-query value. | |
+ | |||
+ | <source lang="xml"> | ||
+ | <?xml version="1.0" encoding="UTF-8"?> | ||
+ | <dbws xmlns:ns1="urn:PLSQLRecord" xmlns:xsd="http://www.w3.org/2001/XMLSchema" | ||
+ | xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> | ||
+ | <name>PLSQLRecord</name> | ||
+ | <sessions-file>eclipselink-dbws-sessions.xml</sessions-file> | ||
+ | <query> | ||
+ | <name>GetNewRecordTest</name> | ||
+ | <result> | ||
+ | <type>ns1:somepkg_orecordType</type> | ||
+ | </result> | ||
+ | <named-query> | ||
+ | <name>GetNewRecordTest</name> | ||
+ | </named-query> | ||
+ | </query> | ||
+ | </dbws> | ||
+ | </source> | ||
+ | |||
+ | ==== JPA Metadata Generation ==== | ||
+ | The <code>org.eclipse.persistence.tools.dbws.XmlEntityMappingsGenerator</code> class is responsible for generating an <code>XMLEntityMappings</code> instance based on a given OR project's queries and descriptors. In addition to the OR project, a list of composite database types (DDL parser meta-model objects) is used to generate metadata for advanced Oracle and PL/SQL types, and a map of CRUD operations (including the SQL statements for each) is used to create named native queries - in addition to findAll and findByPK queries on each OR descriptor. | ||
+ | |||
+ | If DBWS builder is configured as above for the <code>GETNEWREC</code> PL/SQL stored procedure, the following JPA metadata would be generated: | ||
+ | <source lang="xml"> | ||
+ | <?xml version="1.0" encoding="UTF-8"?> | ||
+ | <orm:entity-mappings | ||
+ | xsi:schemaLocation="http://www.eclipse.org/eclipselink/xsds/persistence/orm org/eclipse/persistence/jpa/eclipselink_orm_2_5.xsd" | ||
+ | xmlns:orm="http://www.eclipse.org/eclipselink/xsds/persistence/orm" | ||
+ | xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> | ||
+ | <orm:named-plsql-stored-procedure-query name="GetNewRecordTest" procedure-name="SOMEPKG.GETNEWREC"> | ||
+ | <orm:parameter direction="OUT" name="NEWREC" database-type="SOMEPKG.ORECORD"/> | ||
+ | </orm:named-plsql-stored-procedure-query> | ||
+ | <orm:plsql-record name="SOMEPKG.ORECORD" compatible-type="SOMEPKG_ORECORD" java-type="somepkg.orecord"> | ||
+ | <orm:field name="O1" database-type="VARCHAR_TYPE"/> | ||
+ | <orm:field name="O2" database-type="DECIMAL_TYPE"/> | ||
+ | </orm:plsql-record> | ||
+ | <orm:embeddable class="somepkg.orecord" access="VIRTUAL"> | ||
+ | <orm:struct name="SOMEPKG_ORECORD"> | ||
+ | <orm:field>O1</orm:field> | ||
+ | <orm:field>O2</orm:field> | ||
+ | </orm:struct> | ||
+ | <orm:attributes> | ||
+ | <orm:basic name="o1" attribute-type="java.lang.String"> | ||
+ | <orm:column name="O1"/> | ||
+ | </orm:basic> | ||
+ | <orm:basic name="o2" attribute-type="java.lang.String"> | ||
+ | <orm:column name="O2"/> | ||
+ | </orm:basic> | ||
+ | </orm:attributes> | ||
+ | </orm:embeddable> | ||
+ | </orm:entity-mappings> | ||
+ | </source> | ||
==== JAXB Metadata Generation ==== | ==== JAXB Metadata Generation ==== | ||
− | + | The <code>org.eclipse.persistence.tools.dbws.XmlBindingsGenerator</code> class is responsible for generating one or more EclipseLink <code>XmlBindings</code> objects based on a given list of XML descriptors (one <code>XmlBindings</code> per package). | |
− | + | ||
− | ==== | + | If DBWS builder is configured as above for the <code>GETNEWREC</code> PL/SQL stored procedure, the following JAXB metadata would be generated: |
− | + | <source lang="java"> | |
− | + | <?xml version="1.0" encoding="UTF-8" standalone="yes"?> | |
+ | <xml-bindings-list xmlns="http://www.eclipse.org/eclipselink/xsds/persistence/oxm"> | ||
+ | <xml-bindings package-name="somepkg"> | ||
+ | <xml-schema element-form-default="QUALIFIED" namespace="urn:PLSQLRecord"> | ||
+ | <xml-ns namespace-uri="http://www.w3.org/2001/XMLSchema-instance" prefix="xsi"/> | ||
+ | </xml-schema> | ||
+ | <java-types> | ||
+ | <java-type name="orecord" xml-accessor-type="FIELD"> | ||
+ | <xml-type name="somepkg_orecordType" namespace="urn:PLSQLRecord"/> | ||
+ | <xml-root-element name="somepkg_orecordType" namespace="urn:PLSQLRecord"/> | ||
+ | <java-attributes> | ||
+ | <xml-element type="java.lang.String" xml-path="o1/text()" java-attribute="o1"> | ||
+ | <xml-null-policy is-set-performed-for-absent-node="true" xsi-nil-represents-null="true" | ||
+ | empty-node-represents-null="false" null-representation-for-xml="XSI_NIL"/> | ||
+ | <xml-schema-type name="string"/> | ||
+ | </xml-element> | ||
+ | <xml-element type="java.math.BigDecimal" xml-path="o2/text()" java-attribute="o2"> | ||
+ | <xml-null-policy is-set-performed-for-absent-node="true" xsi-nil-represents-null="true" | ||
+ | empty-node-represents-null="false" null-representation-for-xml="XSI_NIL"/> | ||
+ | <xml-schema-type name="decimal"/> | ||
+ | </xml-element> | ||
+ | </java-attributes> | ||
+ | </java-type> | ||
+ | </java-types> | ||
+ | </xml-bindings> | ||
+ | </xml-bindings-list> | ||
+ | </source> | ||
=== Runtime Component === | === Runtime Component === | ||
The DBWS runtime component is responsible for fulfilling SOAP message requests. DBWS makes use of EclipseLink JAXB (static and dynamic), JPA and XR to accomplish this. | The DBWS runtime component is responsible for fulfilling SOAP message requests. DBWS makes use of EclipseLink JAXB (static and dynamic), JPA and XR to accomplish this. | ||
− | ==== Basic Flow ==== | + | ==== Basic Runtime Flow ==== |
The basic flow of the DBWS runtime is as follows: | The basic flow of the DBWS runtime is as follows: | ||
− | # | + | # XR Service is initialized (<code>org.eclipse.persistence.internal.xr.XRServiceFactory</code>) |
− | # SOAP message received and passed to the | + | # SOAP message received and passed to the XR runtime (<code>org.eclipse.persistence.internal.dbws.ProviderHelper</code>) |
− | # SOAP body is unmarshaled into an | + | # SOAP body is unmarshaled into an <code>org.eclipse.persistence.internal.xr.Invocation</code> object (operation name & parameter values) |
− | # | + | # The <code>org.eclipse.persistence.internal.xr.Operation</code> is retrieved from the XR Service based on the Invocation name |
− | # | + | # The Operation is invoked |
− | # | + | #* The DatabaseQuery is retrieved from the query handler |
− | # SOAP response is returned to the caller | + | #* For query operations some modifications are made to the query before execution (as ORM-loaded queries need arg/param mods) |
+ | # The DatabaseQuery is executed | ||
+ | # The query results typically require some processing (such as handling the specific result type, simple-xml-format, collections, binary data, etc.) | ||
+ | # A SOAP response message is generated (<code>org.eclipse.persistence.internal.dbws.SOAPResponseWriter</code>) | ||
+ | # The SOAP response is returned to the caller | ||
− | === | + | ==== Legacy Deployment XML Support ==== |
− | + | To support archives that were generated prior to EclipseLink 2.5, we need to handle loading OR/OX deployment XML at runtime. The basic algorithm is to first attempt to load JAXB/JPA metadata, and if this fails try to load the legacy deployment XML project. | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
=== Tests === | === Tests === | ||
Line 241: | Line 390: | ||
* eclipselink.dbws.builder.test.oracle.server | * eclipselink.dbws.builder.test.oracle.server | ||
− | ==== | + | ==== Configuration ==== |
The following properties need to be set for the tests to run successfully (typically via test.properties in <code>user_home</code>): | The following properties need to be set for the tests to run successfully (typically via test.properties in <code>user_home</code>): | ||
# db.user | # db.user | ||
Line 299: | Line 448: | ||
* <code>ant -f antbuild.xml test-builder</code> | * <code>ant -f antbuild.xml test-builder</code> | ||
* <code>ant -f antbuild.xml test-service</code> <-- deploy generated .war files first | * <code>ant -f antbuild.xml test-service</code> <-- deploy generated .war files first | ||
+ | |||
+ | === Evolution of DBWS === | ||
+ | ==== EclipseLink 2.4 ==== | ||
+ | EclipseLink 2.4 was the first release containing the Oracle DDL parser. This JavaCC based parser was designed to better handle parsing Oracle-specific DDL, providing the ability to process PL/SQL and advanced Oracle JDBC types, such as Varray, Object and Object table types. Another major change was abandoning the visitor pattern that was used to create projects and descriptors based on the types discovered in the DDL. As of 2.4, a database type meta-model is constructed based on results of the DDLParser parse or JDBC driver metadata, and this meta-model is used to build the various required artifacts. | ||
+ | |||
+ | Information about the DDLParser's design and functionality can be found here: http://wiki.eclipse.org/index.php?title=EclipseLink/Development/DBWS/ParseDDLDS . In particular, the "Parsing Technology" section is worth reviewing. | ||
+ | |||
+ | ==== EclipseLink 2.5 ==== | ||
+ | The major change in this release was removal of the legacy EclipseLink deployment XML project files in favor of JPA/JAXB metadata. In addition, much of the code was reworked to accommodate generation of the metadata vs. deployment XML artifacts. | ||
+ | |||
+ | It is important to note that 2.5 and 2.6 DBWS code bases are virtually identical. | ||
+ | |||
+ | ==== Metadata Generation ==== | ||
+ | For 2.6 we were planning on generating a JPA meta-model directly from the DDLParser meta-model objects, as opposed to generating OR/OX projects to use to build the required artifacts, then the JPA meta-model at the end of the process. JAXB would have followed, however, this work was just under way when DBWS was shelved. The code is in the <code>utils</code> project in the <code>org.eclipse.persistence.tools.metadata.generation</code> package. | ||
+ | |||
+ | Note that there are tests for metadata generation in the <code>org.eclipse.persistence.tools.metadata.generation.test</code> package in the <code>utils</code> project; these tests were intended to test JPA metadata generation based on the DDLParser meta-model, and unless that work is continued/completed they aren't relevant. |
Latest revision as of 11:31, 6 May 2014
Contents
- 1 Overview of EclipseLink DBWS
- 1.1 Design-time Component
- 1.1.1 Basic Builder Flow
- 1.1.2 Supported Input
- 1.1.3 User/Tooling Input
- 1.1.4 Command-line Utility
- 1.1.5 Simple XML Format
- 1.1.6 Shadow Types & Anonymous PL/SQL Generation
- 1.1.7 DDL Parser
- 1.1.8 XML Schema Generation
- 1.1.9 XR Service File Generation
- 1.1.10 JPA Metadata Generation
- 1.1.11 JAXB Metadata Generation
- 1.2 Runtime Component
- 1.3 Tests
- 1.4 Evolution of DBWS
- 1.1 Design-time Component
Overview of EclipseLink DBWS
The purpose of this document is to provide a high-level overview of the design and runtime portions of the DBWS component. In addition, an outline of relevant changes from version to version will be provided. The EclipseLink DBWS documentation can be found here: http://www.eclipse.org/eclipselink/documentation/2.5/dbws/toc.htm.
Design-time Component
DBWS builder is responsible for processing input from the user (or tooling, such as JDeveloper), then scraping the database DDL based on this input. The information retrieved from the database is used to construct a meta-model, which is in turn used to generate a number of artifacts required by the DBWS runtime in order to service SOAP message requests.
The following artifacts are generated by the builder:
- JPA metadata
- JAXB metadata
- Sessions XML
- DBWS XML-Relational (XR) file (defines query operations used by the XR runtime)
- XML Schema
- WSDL (JAX-WS 2.0)
- web.xml
- DBWSProvider (Web service provider - deployed in a servlet container)
- ProviderListener (servlet listener impl.)
Typical .war file structure
\---web-inf | | web.xml | +---classes | +---META-INF | | eclipselink-dbws.xml | | eclipselink-dbws-or.xml | | eclipselink-dbws-ox.xml | | eclipselink-dbws-sessions.xml | | | \---_dbws | DBWSProvider.class -- auto-generated JAX-WS 2.0 Provider | ProviderListener.class -- auto-generated Servlet listener | \---wsdl eclipselink-dbws-schema.xsd eclipselink-dbws.wsdl swaref.xsd -- optionally generated if swaRef is being utilized
Basic Builder Flow
The basic flow of DBWS builder is as follows:
- Scrape the database DDL based on user input and build meta-model (
org.eclipse.persistence.tools.oracleddl.metadata
) - Generate OR/OX projects based on meta-model
- Generate XML schema based on OX project
- Generate XR Service Model based on OR project
- Generate WSDL based on Service Model
- Generate Web app and WebService implmentation
- Generate JAXB and JPA metadata files based on OR/OX projects
- Package generated artifacts
Supported Input
Following are the types of input DBWS supports:
- DB Table
- CRUD operations generated by default
- Custom SQL statements - allows additional operations to be defined on a given table, for example, the ability to search for one or more entries based on a field other than the PK field
- Nested procedures are supported
- Secondary SQL
- Allows separate design/runtime SQL statements to be defined
- Design time SQL used to build operations and runtime SQL executed at runtime
- Stored function/procedure
- PL/SQL stored function/procedure
User/Tooling Input
At a minimum, DBWS builder requires database connectivity information and one or more database targets as listed in Supported Input above. The input can be passed to the builder as an XML file or as Java objects (builder creates the Java model based on the input XML, but this model can be created and handed to the builder).
Following is sample input XML:
<?xml version="1.0" encoding="UTF-8"?> <dbws-builder xmlns:xsd=\http://www.w3.org/2001/XMLSchema\> <properties> <property name="projectName">tabletype</property> <property name="logLevel">off</property> <property name="username">scott</property> <property name="password">tiger</property> <property name="url">jdbc:oracle:thin:@localhost:1521:ORCL</property> <property name="driver">oracle.jdbc.OracleDriver</property> <property name="platformClassname">org.eclipse.persistence.platform.database.oracle.Oracle11Platform</property> </properties> <table schemaPattern="SCOTT" tableNamePattern="EMPLOYEE" /> </dbws-builder>
And the equivalent Java code:
DBWSBuilder builder = new DBWSBuilder(); builder.setProjectName("tabletype"); builder.setLogLevel("off"); builder.setUsername("scott"); builder.setPassword("tiger"); builder.setUrl("jdbc:oracle:thin:@localhost:1521:ORCL"); builder.setDriver("oracle.jdbc.OracleDriver"); builder.setPlatformClassname("org.eclipse.persistence.platform.database.oracle.Oracle11Platform"); TableOperationModel tableOpModel = new TableOperationModel(); tableOpModel.setSchemaPattern("SCOTT"); tableOpModel.setTablePattern("EMPLOYEE"); builder.addOperation(tableOpModel);
Note: the 'projectName' property will be used as the default namespace.
Wildcards
Wildcards are supported for the schema, table, and procedure pattern values. For example, if the schemaPattern value is set to "%", all schemas visible to the current user are checked for the target artifact (table, procedure, etc.). Another example would be where all stored procedures starting with "EMP" are to be processed:
And the equivalent Java code:
procOpModel = new ProcedureOperationModel(); procOpModel.setName("EmployeeTest"); procOpModel.setCatalogPattern("TOPLEVEL"); <-- indicates the stored proc is not in a PL/SQL package procOpModel.setProcedurePattern("EMP%");
Command-line Utility
The EclipseLink install comes with a command-line DBWS builder utility (.cmd and .sh). This allows the DBWS builder to be configured and executed easily. The utility is in <eclipselink_home>\utils\dbws
. The JDBC driver must be on the classpath. The utility looks for a DRIVER_CLASSPATH
variable for the .jar location; this variable can be set or simply edited in the .cmd/.sh file to point to the correct location, i.e. set CLASSPATH=%CLASSPATH%;<work_home>\extension.oracle.lib.external\ojdbc6.jar
.
A builder XML configuration file is required, and the staging directory must be set. Following is an example of how to execute the builder (.war packaging with WebLogic server target) using the Windows command script:
dbwsbuilder.cmd -builderFile employee.xml -stageDir stage -packageAs wls employee.war
Simple XML Format
DBWS provides the ability to wrap results in custom XML tags; this is useful where the structure of the results are not known, such as in the case of a ref cursor. If the 'isSimpleXmlFormat' flag is set to true, the results will look like the following:
<?xml version="1.0" encoding="UTF-8"?> <simple-xml-format> <simple-xml> <ID>1</ID> <NAME>Joe Oracle</NAME> </simple-xml> <simple-xml> <ID>2</ID> <NAME>Jane Oracle</NAME> </simple-xml> </simple-xml-format>
Note that both the simple-xml-format
and simple-xml
element names are customizable.
Shadow Types & Anonymous PL/SQL Generation
JDBC does not support transporting Oracle PL/SQL types. In order for DBWS to handle PL/SQL input and output arguments, two tasks are required. First, for each PL/SQL type there must be an equivalent JDBC type. DBWS will generate 'shadow' create/drop DDL for each PL/SQL type; this DDL can be executed on the DB to add and remove these required types. The second thing that is required are PL/SQL functions that covert PL/SQL types to equivalent JDBC types and vice-versa. To support this, anonymous PL/SQL is generated and executed on the database at runtime.
Shadow Type Example
If a given package SOMEPKG
has a PL/SQL record ORECORD
defined as follows:
CREATE OR REPLACE PACKAGE SOMEPKG AS TYPE ORECORD IS RECORD ( O1 VARCHAR2(10), O2 DECIMAL(7,2) ); END PACKAGE SOMEPKG;
DBWS requires that an equivalant JDBC shadow type also exists at the TOPLEVEL (not in a package), i.e.:
CREATE OR REPLACE TYPE SOMEPKG_ORECORD AS OBJECT ( O1 VARCHAR2(10), O2 DECIMAL(7,2) )
Note that the expected naming convention for the shadow JDBC type is packagename_plsqlname
.
ShadowDDLGenerator
DBWS builder generates required shadow create and drop DDL such that the user doesn't have to create the required DDL manually. Following is an example of using the builder-generated shadow DDL:
Connection conn = buildSQLConnection(); PreparedStatement pStmt; // execute shadow type ddl to generate JDBC equivalents of PL/SQL types for (String ddl : builder.getTypeDDL()) { try { pStmt = conn.prepareStatement(ddl); pStmt.execute(); } catch (SQLException e) { // handle exception } }
Note that similar code is used to execute the drop DDL.
Anonymous PL/SQL Code Generation
Considering a PL/SQL stored procedure that returns a new ORECORD
:
CREATE OR REPLACE PACKAGE SOMEPKG AS PROCEDURE GETNEWREC(NEWREC OUT ORECORD); END PACKAGE SOMEPKG;
The following anonymous PL/SQL will be generated and executed against the database at runtime:
DECLARE NEWRECTARGET SOMEPKG.ORECORD; <!-- PL/SQL record declaration --> FUNCTION EL_PL2SQL_0(aPlsqlItem SOMEPKG.ORECORD) <!-- Converts the new PL/SQL record to it's shadow JDBC type --> RETURN SOMEPKG_ORECORD IS aSqlItem SOMEPKG_ORECORD; BEGIN aSqlItem := SOMEPKG_ORECORD(NULL, NULL); aSqlItem.N1 := aPlsqlItem.N1; aSqlItem.N2 := aPlsqlItem.N2; RETURN aSqlItem; END EL_PL2SQL_0; BEGIN SOMEPKG.GETNEWREC(NEWREC=>NEWRECTARGET); <!-- The OUT argument 'NEWREC' from the PL/SQL procedure is passed to --> :1 := EL_PL2SQL_0(NEWRECTARGET); <!-- the anonymous function to be converted to the shadow JDBC type --> END;
Note that this generation occurs in the prepareInternal
method of PLSQLStoredProcedureCall
.
DDL Parser
To provide more complete handling of Oracle PL/SQL and 'advanced' JDBC types (Varray, Object and Object table types), the DDLParser was created. This JavaCC constructed parser is responsible for building database meta-model objects from a given string of Oracle DDL.
More information regarding the DDLParser can be found here:
- http://wiki.eclipse.org/EclipseLink/Development/DBWS/ParseDDLFS
- http://wiki.eclipse.org/EclipseLink/Development/DBWS/ParseDDLDS
Git repo:
JavaCC plugin for Eclipse:
Eclipse Configuration
For running in Eclipse, the DDL parser project requires the Maven (m2e) plugin and the Tycho Configurator. Once the m2e plugin is installed, Tycho can be setup by selecting Preferences > Maven > Discovery > Open Catalog, then selecting the Tycho Configurator.
XML Schema Generation
DBWS builder utilizes the org.eclipse.persistence.internal.oxm.schema.SchemaModelGenerator
class to generate the XML schema. The types generated are referenced in JAXB metadata and the DBWS service file.
If DBWS builder is configured as follows for the GETNEWREC
PL/SQL stored procedure above:
procOpModel = new PLSQLProcedureOperationModel(); procOpModel.setName("GetNewRecordTest"); procOpModel.setCatalogPattern("SOMEPKG"); procOpModel.setProcedurePattern("GETNEWREC");
The following XML schema will be generated:
<?xml version="1.0" encoding="UTF-8"?> <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" targetNamespace="urn:PLSQLRecord" xmlns="urn:PLSQLRecord" elementFormDefault="qualified"> <xsd:complexType name="somepkg_orecordType"> <xsd:sequence> <xsd:element name="o1" type="xsd:string" minOccurs="0" nillable="true"/> <xsd:element name="o2" type="xsd:decimal" minOccurs="0" nillable="true"/> </xsd:sequence> </xsd:complexType> <xsd:element name="somepkg_orecordType" type="somepkg_orecordType"/> </xsd:schema>
XR Service File Generation
The service file contains query information required by the XR component at runtime. Database queries are looked up on the OR session based on the named-query value.
<?xml version="1.0" encoding="UTF-8"?> <dbws xmlns:ns1="urn:PLSQLRecord" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <name>PLSQLRecord</name> <sessions-file>eclipselink-dbws-sessions.xml</sessions-file> <query> <name>GetNewRecordTest</name> <result> <type>ns1:somepkg_orecordType</type> </result> <named-query> <name>GetNewRecordTest</name> </named-query> </query> </dbws>
JPA Metadata Generation
The org.eclipse.persistence.tools.dbws.XmlEntityMappingsGenerator
class is responsible for generating an XMLEntityMappings
instance based on a given OR project's queries and descriptors. In addition to the OR project, a list of composite database types (DDL parser meta-model objects) is used to generate metadata for advanced Oracle and PL/SQL types, and a map of CRUD operations (including the SQL statements for each) is used to create named native queries - in addition to findAll and findByPK queries on each OR descriptor.
If DBWS builder is configured as above for the GETNEWREC
PL/SQL stored procedure, the following JPA metadata would be generated:
<?xml version="1.0" encoding="UTF-8"?> <orm:entity-mappings xsi:schemaLocation="http://www.eclipse.org/eclipselink/xsds/persistence/orm org/eclipse/persistence/jpa/eclipselink_orm_2_5.xsd" xmlns:orm="http://www.eclipse.org/eclipselink/xsds/persistence/orm" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <orm:named-plsql-stored-procedure-query name="GetNewRecordTest" procedure-name="SOMEPKG.GETNEWREC"> <orm:parameter direction="OUT" name="NEWREC" database-type="SOMEPKG.ORECORD"/> </orm:named-plsql-stored-procedure-query> <orm:plsql-record name="SOMEPKG.ORECORD" compatible-type="SOMEPKG_ORECORD" java-type="somepkg.orecord"> <orm:field name="O1" database-type="VARCHAR_TYPE"/> <orm:field name="O2" database-type="DECIMAL_TYPE"/> </orm:plsql-record> <orm:embeddable class="somepkg.orecord" access="VIRTUAL"> <orm:struct name="SOMEPKG_ORECORD"> <orm:field>O1</orm:field> <orm:field>O2</orm:field> </orm:struct> <orm:attributes> <orm:basic name="o1" attribute-type="java.lang.String"> <orm:column name="O1"/> </orm:basic> <orm:basic name="o2" attribute-type="java.lang.String"> <orm:column name="O2"/> </orm:basic> </orm:attributes> </orm:embeddable> </orm:entity-mappings>
JAXB Metadata Generation
The org.eclipse.persistence.tools.dbws.XmlBindingsGenerator
class is responsible for generating one or more EclipseLink XmlBindings
objects based on a given list of XML descriptors (one XmlBindings
per package).
If DBWS builder is configured as above for the GETNEWREC
PL/SQL stored procedure, the following JAXB metadata would be generated:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <xml-bindings-list xmlns="http://www.eclipse.org/eclipselink/xsds/persistence/oxm"> <xml-bindings package-name="somepkg"> <xml-schema element-form-default="QUALIFIED" namespace="urn:PLSQLRecord"> <xml-ns namespace-uri="http://www.w3.org/2001/XMLSchema-instance" prefix="xsi"/> </xml-schema> <java-types> <java-type name="orecord" xml-accessor-type="FIELD"> <xml-type name="somepkg_orecordType" namespace="urn:PLSQLRecord"/> <xml-root-element name="somepkg_orecordType" namespace="urn:PLSQLRecord"/> <java-attributes> <xml-element type="java.lang.String" xml-path="o1/text()" java-attribute="o1"> <xml-null-policy is-set-performed-for-absent-node="true" xsi-nil-represents-null="true" empty-node-represents-null="false" null-representation-for-xml="XSI_NIL"/> <xml-schema-type name="string"/> </xml-element> <xml-element type="java.math.BigDecimal" xml-path="o2/text()" java-attribute="o2"> <xml-null-policy is-set-performed-for-absent-node="true" xsi-nil-represents-null="true" empty-node-represents-null="false" null-representation-for-xml="XSI_NIL"/> <xml-schema-type name="decimal"/> </xml-element> </java-attributes> </java-type> </java-types> </xml-bindings> </xml-bindings-list>
Runtime Component
The DBWS runtime component is responsible for fulfilling SOAP message requests. DBWS makes use of EclipseLink JAXB (static and dynamic), JPA and XR to accomplish this.
Basic Runtime Flow
The basic flow of the DBWS runtime is as follows:
- XR Service is initialized (
org.eclipse.persistence.internal.xr.XRServiceFactory
) - SOAP message received and passed to the XR runtime (
org.eclipse.persistence.internal.dbws.ProviderHelper
) - SOAP body is unmarshaled into an
org.eclipse.persistence.internal.xr.Invocation
object (operation name & parameter values) - The
org.eclipse.persistence.internal.xr.Operation
is retrieved from the XR Service based on the Invocation name - The Operation is invoked
- The DatabaseQuery is retrieved from the query handler
- For query operations some modifications are made to the query before execution (as ORM-loaded queries need arg/param mods)
- The DatabaseQuery is executed
- The query results typically require some processing (such as handling the specific result type, simple-xml-format, collections, binary data, etc.)
- A SOAP response message is generated (
org.eclipse.persistence.internal.dbws.SOAPResponseWriter
) - The SOAP response is returned to the caller
Legacy Deployment XML Support
To support archives that were generated prior to EclipseLink 2.5, we need to handle loading OR/OX deployment XML at runtime. The basic algorithm is to first attempt to load JAXB/JPA metadata, and if this fails try to load the legacy deployment XML project.
Tests
There are tests in two different projects; dbws
and utils
. The tests in the dbws
project focus on XR runtime testing, whereas the tests in the utils
project are end-to-end tests. Additionally, there are server tests in the utils
project that allow the builder to be executed, then the generated .war can be deployed to an app server and the service tested (via SOAP).
XR (dbws
project)
- eclipselink.dbws.test
- eclipselink.dbws.test.oracle
DBWS builder (utils
project)
- eclipselink.dbws.builder.test
- eclipselink.dbws.builder.test.oracle
- eclipselink.dbws.builder.test.oracle.server
Configuration
The following properties need to be set for the tests to run successfully (typically via test.properties in user_home
):
- db.user
- db.pwd
- db.driver
- db.url
- db.platform
- jdbc.driver.jar
- logging.level (optional)
- support.test (should set this to
true
)
For server tests, the following additional properties need to be set:
- server.platform
- server.host
- server.port
- server.datasource
Following is a sample test.properties
file
# Testing environment properties # # Database properties # db.user=SCOTT db.pwd=toger db.driver=oracle.jdbc.OracleDriver db.url=jdbc:oracle:thin:@ottvm000.ca.oracle.com:1521:ORCL db.platform=org.eclipse.persistence.platform.database.oracle.Oracle11Platform jdbc.driver.jar=D:/extension.oracle.lib.external/ojdbc6.jar # # WLS properties # server.platform=wls server.host=localhost server.port=7001 server.datasource=jdbc/DBWStestDS # # TopLink properties # # Logging option for debugging - 'info' for light logging, 'fine' for SQL stmts, 'finest' for everything logging.level=off support.test=true
ANT
The XR/Builder tests can be run by executing:
-
ant -f antbuild.xml run-tests
For the server tests, the following two targets can be executed:
-
ant -f antbuild.xml test-builder
-
ant -f antbuild.xml test-service
<-- deploy generated .war files first
Evolution of DBWS
EclipseLink 2.4
EclipseLink 2.4 was the first release containing the Oracle DDL parser. This JavaCC based parser was designed to better handle parsing Oracle-specific DDL, providing the ability to process PL/SQL and advanced Oracle JDBC types, such as Varray, Object and Object table types. Another major change was abandoning the visitor pattern that was used to create projects and descriptors based on the types discovered in the DDL. As of 2.4, a database type meta-model is constructed based on results of the DDLParser parse or JDBC driver metadata, and this meta-model is used to build the various required artifacts.
Information about the DDLParser's design and functionality can be found here: http://wiki.eclipse.org/index.php?title=EclipseLink/Development/DBWS/ParseDDLDS . In particular, the "Parsing Technology" section is worth reviewing.
EclipseLink 2.5
The major change in this release was removal of the legacy EclipseLink deployment XML project files in favor of JPA/JAXB metadata. In addition, much of the code was reworked to accommodate generation of the metadata vs. deployment XML artifacts.
It is important to note that 2.5 and 2.6 DBWS code bases are virtually identical.
Metadata Generation
For 2.6 we were planning on generating a JPA meta-model directly from the DDLParser meta-model objects, as opposed to generating OR/OX projects to use to build the required artifacts, then the JPA meta-model at the end of the process. JAXB would have followed, however, this work was just under way when DBWS was shelved. The code is in the utils
project in the org.eclipse.persistence.tools.metadata.generation
package.
Note that there are tests for metadata generation in the org.eclipse.persistence.tools.metadata.generation.test
package in the utils
project; these tests were intended to test JPA metadata generation based on the DDLParser meta-model, and unless that work is continued/completed they aren't relevant.