Jump to: navigation, search

Difference between revisions of "EDT:IDE Test Server Regression Testing"

(New page: The following test scenarios should be run each release. Read the top section of the main test server wiki page to understand some basics. <br> '''Dedicated serv...)
 
 
(11 intermediate revisions by one other user not shown)
Line 1: Line 1:
The following test scenarios should be run each release.
+
The following test scenarios should be run each release.  
  
Read the top section of the [[EDT:IDE_Test_Server|main test server wiki page]] to understand some basics.
+
Read the top section of the [[EDT:IDE Test Server|main test server wiki page]] to understand some basics. &nbsp;
  
<br>
+
Tip: To kill the test server so that you can try another scenario, either click the red terminate button in the console, or use the Debug view to kill the process.<br>  
  
'''Dedicated services'''
+
Note: to run a service on the test server you will need to write a simple RUI Handler. The preview pane of the visual editor can be used to run it.
  
TBD
+
<br>
  
 +
'''Dedicated services'''
  
 +
Verify a dedicated service is run on a test server for the main RUI Handler's project, not the service's project. To verify the right project is used, put the dedicated service in a separate project, but make sure it's on the EGL and Java build paths of the RUI project.
  
'''REST services'''
+
Breakpoints in the service should be honored, because the test server is always in debug mode.
  
TBD
+
<br>
  
 +
'''REST services'''
  
 +
Verify a REST service is run on a test server for the service's project, not the RUI Handler's project. To verify the right project is used, put the REST service in a separate project; it does not need to be on the EGL or Java build path of the RUI project.
  
'''Prompting'''
+
Breakpoints in the service should be honored, because the test server is always in debug mode.
  
TBD
+
Note: you neeed to use the workspace:// URI for the binding in your deployment descriptor, AND you need to configure the service to be deployed as REST in its default deployment descriptor file. Failure to do these steps will either cause the test server to not be used, or the test server will not be able to find the REST service.
  
 +
<br>
  
 +
'''Prompting'''
  
'''Database access'''
+
The prompting preferences can be configured in Preferences &gt; EGL &gt; Test Server. Make sure to test each radio button with the below cases &nbsp;
  
TBD (include JNDI)
+
*Classpath changes
 +
**Add or remove a jar or library to a project
 +
**Add or remove a project dependency on the Java build path
 +
*Hot code replace (HCR) failures&nbsp;
 +
**To test "When hot code replace fails":<br>
 +
***Add or remove a global field
 +
**To test "When hot code replace is not supported":<br>
 +
***Don't worry about testing this, it's extremely unlikely these days
 +
**To test "When obsolete methods remain on the stack":<br>
 +
***Try out various changes like modifying function signatures, but if you can't get it to occur then don't worry about it.<br>
 +
***If you CAN get it to occur in a reproducible way, please edit these instructions!
  
 +
<br>
  
 +
'''Other preferences'''
  
'''Deployment descriptors'''
+
*"Show tracing messages in the Console view"
 +
**When disabled, the console is empty (or mostly empty) upon startup and you do not see tracing information during service invocation
 +
**When enabled, the console will have a ton of output on startup, and each service invocation will spit out a lot of tracing information.
  
TBD (include generated DD support)
+
<br> '''Database access'''
 +
 
 +
In your service's deployment descriptor, add an SQL database binding and make sure it's configured to use JNDI. In your service function, add some code that looks up the binding to create an SQLDataSource, then run some SQL statements to verify it can connect correctly.
 +
 
 +
This database scenario should be run twice. Once with an Apache Tomcat runtime defined (you don't need to define the server, just the runtime at Preferences &gt; Server &gt; Runtime Environments), and once with no Apache Tomcat runtime defined. When it's defined, connection pooling will be used; when it's not defined, connection pooling is not used, and you will see a warning in the console about this case.
 +
 
 +
Another variation to add to SQL testing is to make your SQL database binding use application-based authentication (this is not the default). Do this both with and without Tomcat being defined.
 +
 
 +
If you have even more time to burn, you can run all these tests once with the binding referencing a connection profile, and once with the information hard-coded ("Specify connection information" radio button).<br>
 +
 
 +
All these tests should be successful without adding any JDBC jars to your project build paths.
 +
 
 +
<br>
 +
 
 +
'''Deployment descriptors'''
 +
 
 +
A project defines its default deployment descriptor via right-click the project &gt; Properties &gt; EGL Development Deployment Descriptor. When your binding URI does not specify a file, this default file is used for the lookup. Verify that this default is used by using a URI that does not specify a file, e.g. Resources.getResource("binding:myBinding").
 +
 
 +
Verify that non-default deployment descriptors work when running on the test server (e.g. Resources.getResource("binding:file:alternateDD#myBinding")). This non-default deployment descriptor must be located in a project on the test server project's EGL build path. &nbsp;
 +
 
 +
For example when a test server is running for project A, and B is on A's EGL build path, you can reference a deployment descriptor that's in project B. But if you try to use a deployment descriptor from project C, and C is not on A's extended EGL build path, you should get a runtime error about the file not being found. This error should be catchable in your EGL code. Note that it doesn't matter if A's deployment descriptor has an "include" for C's deployment descriptor - ANY deployment descriptor that you wish to access must be on the EGL build path of A. &nbsp;
 +
 
 +
One more test is to verify that "generated" deployment descriptors can be used by the test server. Normally it will use the original *.egldd file, but the "*-bnd.xml" files will also work if the original *.egldd file is not available. To test this out, create a new project, add an SQL binding to its deployment descriptor, deploy it to a temporary web project, then jar up the *-bnd.xml file that was produced in the web project. Now you can delete the temporary web project and the EGL project you deployed. Add the jar containing the *-bnd.xml file to your service project's Java build path; make sure none of the deployment descriptors in the service project's EGL build path have this same name, then reference it by name in your code: Resources.getResource("binding:file:generatedDDName#bindingName"). This code should not throw an error, and the resulting SQLDataSource should be able to retrieve data from the database just like when a *.egldd file is used.
 +
 
 +
[[Category:EDT]]

Latest revision as of 23:23, 14 May 2012

The following test scenarios should be run each release.

Read the top section of the main test server wiki page to understand some basics.  

Tip: To kill the test server so that you can try another scenario, either click the red terminate button in the console, or use the Debug view to kill the process.

Note: to run a service on the test server you will need to write a simple RUI Handler. The preview pane of the visual editor can be used to run it.


Dedicated services

Verify a dedicated service is run on a test server for the main RUI Handler's project, not the service's project. To verify the right project is used, put the dedicated service in a separate project, but make sure it's on the EGL and Java build paths of the RUI project.

Breakpoints in the service should be honored, because the test server is always in debug mode.


REST services

Verify a REST service is run on a test server for the service's project, not the RUI Handler's project. To verify the right project is used, put the REST service in a separate project; it does not need to be on the EGL or Java build path of the RUI project.

Breakpoints in the service should be honored, because the test server is always in debug mode.

Note: you neeed to use the workspace:// URI for the binding in your deployment descriptor, AND you need to configure the service to be deployed as REST in its default deployment descriptor file. Failure to do these steps will either cause the test server to not be used, or the test server will not be able to find the REST service.


Prompting

The prompting preferences can be configured in Preferences > EGL > Test Server. Make sure to test each radio button with the below cases  

  • Classpath changes
    • Add or remove a jar or library to a project
    • Add or remove a project dependency on the Java build path
  • Hot code replace (HCR) failures 
    • To test "When hot code replace fails":
      • Add or remove a global field
    • To test "When hot code replace is not supported":
      • Don't worry about testing this, it's extremely unlikely these days
    • To test "When obsolete methods remain on the stack":
      • Try out various changes like modifying function signatures, but if you can't get it to occur then don't worry about it.
      • If you CAN get it to occur in a reproducible way, please edit these instructions!


Other preferences

  • "Show tracing messages in the Console view"
    • When disabled, the console is empty (or mostly empty) upon startup and you do not see tracing information during service invocation
    • When enabled, the console will have a ton of output on startup, and each service invocation will spit out a lot of tracing information.


Database access

In your service's deployment descriptor, add an SQL database binding and make sure it's configured to use JNDI. In your service function, add some code that looks up the binding to create an SQLDataSource, then run some SQL statements to verify it can connect correctly.

This database scenario should be run twice. Once with an Apache Tomcat runtime defined (you don't need to define the server, just the runtime at Preferences > Server > Runtime Environments), and once with no Apache Tomcat runtime defined. When it's defined, connection pooling will be used; when it's not defined, connection pooling is not used, and you will see a warning in the console about this case.

Another variation to add to SQL testing is to make your SQL database binding use application-based authentication (this is not the default). Do this both with and without Tomcat being defined.

If you have even more time to burn, you can run all these tests once with the binding referencing a connection profile, and once with the information hard-coded ("Specify connection information" radio button).

All these tests should be successful without adding any JDBC jars to your project build paths.


Deployment descriptors

A project defines its default deployment descriptor via right-click the project > Properties > EGL Development Deployment Descriptor. When your binding URI does not specify a file, this default file is used for the lookup. Verify that this default is used by using a URI that does not specify a file, e.g. Resources.getResource("binding:myBinding").

Verify that non-default deployment descriptors work when running on the test server (e.g. Resources.getResource("binding:file:alternateDD#myBinding")). This non-default deployment descriptor must be located in a project on the test server project's EGL build path.  

For example when a test server is running for project A, and B is on A's EGL build path, you can reference a deployment descriptor that's in project B. But if you try to use a deployment descriptor from project C, and C is not on A's extended EGL build path, you should get a runtime error about the file not being found. This error should be catchable in your EGL code. Note that it doesn't matter if A's deployment descriptor has an "include" for C's deployment descriptor - ANY deployment descriptor that you wish to access must be on the EGL build path of A.  

One more test is to verify that "generated" deployment descriptors can be used by the test server. Normally it will use the original *.egldd file, but the "*-bnd.xml" files will also work if the original *.egldd file is not available. To test this out, create a new project, add an SQL binding to its deployment descriptor, deploy it to a temporary web project, then jar up the *-bnd.xml file that was produced in the web project. Now you can delete the temporary web project and the EGL project you deployed. Add the jar containing the *-bnd.xml file to your service project's Java build path; make sure none of the deployment descriptors in the service project's EGL build path have this same name, then reference it by name in your code: Resources.getResource("binding:file:generatedDDName#bindingName"). This code should not throw an error, and the resulting SQLDataSource should be able to retrieve data from the database just like when a *.egldd file is used.