Skip to main content
Jump to: navigation, search

Difference between revisions of "Ant/Test Script"

< Ant
(Debugging)
(Test Passes)
 
(40 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
{{Ant}}
 +
 +
== Test Page Template ==
 +
Test Page Template: [[Ant/Testing/Test Plan Template]]. 
 +
 +
This template must be used to create a new test page for each release that requires testing.
 +
 +
Any new features that are added to the [[Ant/Test Script|test script]] must also be added to the [[Ant/Testing/Test Plan Template#Feature List|feature list]] section of the template.
 +
 
== Testing Notes ==
 
== Testing Notes ==
 
All tests should be done with resource names of varying length, specifically resource names of length one must be tested (the cause of [https://bugs.eclipse.org/bugs/show_bug.cgi?id=323588 bug 323588]).
 
All tests should be done with resource names of varying length, specifically resource names of length one must be tested (the cause of [https://bugs.eclipse.org/bugs/show_bug.cgi?id=323588 bug 323588]).
  
== Feature Testing ==
+
== Test Passes ==
 +
*[[Ant/Testing/Ant-1.9.6|Ant 1.9.6 Update]] (Sep 22, 2015)
 +
*[[Ant/Testing/Ant-1.9.2|Ant 1.9.2 Update]] (July 22, 2013)
 +
*[[Ant/Testing/1.6BREE|Update to 1.6 BREE]] (July 3, 2013)
 +
*[[Ant/Testing/4.3M7|4.3M7]] (May 3, 2013)
 +
*[[Ant/Testing/4.3M4|4.3M4]]
 +
*[[Ant/Testing/1.8.4update|Platform update to Ant 1.8.4]]
 +
*[[Ant/Testing/3.8M6|3.8 M6]]
 +
*[[Ant/Testing/3.8M5|3.8 M5]]
 +
*[[Ant/Testing/3.7.2RC2|3.7.2 RC2]]
 +
*[[Ant/Testing/3.8M4|3.8M4]]
 +
*[[Ant/Testing/3.8M3|3.8M3]]
 +
*[[Ant/Testing/3.8M2|3.8M2]]
 +
*[[Ant/Testing/3.7.1|3.7.1]]
 +
*[[Ant/Testing/End_Game_3.7|3.7 End Game]]
 +
*[[Ant/Testing/3.7M7|3.7M7]]
 +
*[[Ant/Testing/3.7M5|3.7M5]]
 +
*[[Ant/Testing/3.6.2|3.6.2]]
 +
 
 +
== Example Build Files ==
 +
 
 +
A DBCS build file using <code>Shift_JIS</code> encoding and Japanese characters:
 +
 
 +
<source lang="apache">
 +
<?xml version="1.0" encoding="Shift_JIS"?>
 +
<project name="jp_shiftjis_test" default="エコー Hello World">
 +
  <target name="エコー Hello World">
 +
    <echo message="こんにちは!"/>
 +
  </target>
 +
 
 +
  <target name="名前の変更 xml to txt" >
 +
    <echo message="私は名前を変更しました。I rename xml to txt"/>
 +
    <move file="Test.xml" tofile="Test.txt"/>
 +
  </target>
 +
</project>
 +
</source>
 +
 
 +
A DBCS build file using <code>UTF-8</code> encoding and Japanese characters:
 +
 
 +
<source lang="apache">
 +
<?xml version="1.0" encoding="UTF-8"?>
 +
<project name="jp_utf8_test" default="エコー Hello World">
 +
  <target name="エコー Hello World">
 +
    <echo message="こんにちは!"/>
 +
  </target>
 +
 
 +
  <target name="名前の変更 xml to txt" >
 +
    <echo message="私は名前を変更しました。I rename xml to txt"/>
 +
    <move file="Test.xml" tofile="Test.txt"/>
 +
  </target>
 +
</project>
 +
</source>
 +
 
 +
A build file using the Augment task
 +
 
 +
<source lang="apache">
 +
<project>
 +
<path id="path1"/>
 +
<resource id="resource1"/>
 +
<propertyset id="property1"/>
 +
<target name="augment">
 +
<augment id="path1">
 +
<pathelement location="/foo"/>
 +
</augment>
 +
</target>
 +
</project>
 +
</source>
 +
 
 +
A build file using the extension-point target
 +
 
 +
<source lang="apache">
 +
<project default="ep-B">
 +
    <target name="A" />
 +
    <extension-point name="ep-B" depends="A" />
 +
    <target name="B" extensionOf="ep-B" depends="D" />
 +
    <target name="C" depends="ep-B" />
 +
    <target name="D" />
 +
</project>
 +
</source>
  
=== Ant Editor ===
+
== Ant Editor ==
 +
*1.8.x support
 +
**use the extension-point target
 +
***content assist should be available
 +
***mark occurrences should find occurrences
 +
**use the augment task
 +
***content assist should be available
 +
***mark occurrences should find occurrences
 +
***should not throw exception if referencing unknown id
 
*Folding
 
*Folding
 
**make use of the editor context ruler menu
 
**make use of the editor context ruler menu
Line 27: Line 122:
 
**templates should be available and work when applied and run
 
**templates should be available and work when applied and run
 
*Performance
 
*Performance
**"Large" taskdefs (ant-contrib) in Ant Editor
+
**"large" taskdefs (ant-contrib) in Ant Editor
**"Large" imports (need to generate test case) in Ant Editor
+
**"large" imports (need to generate test case) in Ant Editor
**"Large" properties (need to generate test case) in Ant Editor
+
**"large" properties (need to generate test case) in Ant Editor
 +
*Encoding
 +
**using an encoded build file like one of the ones from [[#Example Build Files]] make sure the editor displays the text correctly. On Windows, make sure you have any required language packs installed and that your default encoding has been set correctly
  
=== Ant View ===
+
== Ant View ==
 
*General
 
*General
 
**context menu actions
 
**context menu actions
Line 39: Line 136:
 
**running targets (default and otherwise)
 
**running targets (default and otherwise)
 
**navigation to targets with selection for normal, external entities and imported targets
 
**navigation to targets with selection for normal, external entities and imported targets
 +
*Ant 1.8.x
 +
**Augment task - should show the element it augments as its child
 
*Auto Reconcile
 
*Auto Reconcile
 
**ensure correct functionality on / off (use the editor toolbar action with the "refresh" icon)
 
**ensure correct functionality on / off (use the editor toolbar action with the "refresh" icon)
 
**reconcile triggered on save
 
**reconcile triggered on save
 +
*Encoding
 +
**using an encoded build file like one of the ones from [[#Example Build Files]] make sure the names of tasks / types / etc. display properly. On Windows, make sure you have any required language packs installed and that your default encoding has been set correctly
  
=== Debugging ===
+
== Debugging ==
 
*Buildfiles in the same JRE
 
*Buildfiles in the same JRE
 
**breakpoints (check target breakpoints as well)
 
**breakpoints (check target breakpoints as well)
Line 58: Line 159:
 
**run to line support
 
**run to line support
 
**properties update correctly in the variables view
 
**properties update correctly in the variables view
 +
*Encoding
 +
**using an encoded build file like one of the ones from [[#Example Build Files]] make sure the console output from tasks / types / etc. display properly. On Windows, make sure you have any required language packs installed and that your default encoding has been set correctly
  
=== Import / Export ===
+
== Import / Export ==
Export a Java Project to a buildfile containing a javac task etc.
+
*Export a Java Project to a buildfile containing a javac task etc.
error cases
+
**error cases
classpath including plugin dependencies
+
**classpath including plugin dependencies
Import a Java Project from an external buildfile containing a javac task
+
*Import a Java Project from an external buildfile containing a javac task
error cases
+
**error cases
correct links setup for multi-platform
+
**correct links setup for multi-platform
correct links for various destdir and sourcedir
+
**correct links for various destdir and sourcedir
correct links for various destdir and sourcedir
+
**correct links for various destdir and sourcedir
  
=== Input Handling ===
+
== Input Handling ==
Same JRE
+
*Same JRE
with / without valid args
+
**with / without valid args
cancel
+
**cancel
-noinput
+
**-noinput
Separate JRE
+
*Separate JRE
with / without valid args
+
**with / without valid args
cancel
+
**cancel
-noinput
+
**-noinput
  
=== Launching ===
+
== Launching ==
Correct UI representation of objects contributed to build (same/separate JRE) in the preferences and launch configuration tabs
+
*Correct UI representation of objects contributed to build (same/separate JRE) in the preferences and launch configuration tabs
properties
+
**properties
classpath entries
+
**classpath entries
tasks
+
**tasks
types
+
**types
 +
*Encoding
 +
**using an encoded build file like one of the ones from [[#Example Build Files]] make sure the names of tasks / types / etc. display properly on the tasks tab in the launch configuration dialog. On Windows, make sure you have any required language packs installed and that your default encoding has been set correctly
  
=== External Tools ===
+
== External Tools ==
Tool Builders
+
*Tool Builders
migration from 2.0, 3.0 (with and without prompting)
+
**migration from 2.0, 3.0 (with and without prompting)
Java projects and simple projects
+
**Java projects and simple projects
add / remove (ensure cleanup on disk)
+
**add / remove (ensure cleanup on disk)
rename on creation and as an edit (ensure cleanup on disk)
+
**rename on creation and as an edit (ensure cleanup on disk)
executing an Ant build and program
+
**executing an Ant build and program
same VM / separate VM
+
**same VM / separate VM
making use of environment variables
+
**making use of environment variables
Ant builders set to execute different targets for different build kinds (Ant builder target tab)
+
**Ant builders set to execute different targets for different build kinds (Ant builder target tab)
auto
+
***auto - ensure only the targets specified in Auto Build are executed, except for immediately following a clean command, in which case After a Clean targets should be executed.
manual
+
***manual - with target(s) set in After a Clean, Manual Build targets should only be executed for manual incremental builds. With no target(s) set for After a Clean, Manual Build targets should run following a clean command AND manual incremental builds.
after a clean (full)
+
***after a clean - ensure these targets are executed in the build immediately following a clean command, unless you set no targets to be executed, in which case the targets from Manual Build should be executed.
on a clean
+
***on a clean - ensure these targets are executed DURING the clean, and that After a Clean targets are executed on the following build.
Program builders triggered on clean (Build option tab)
+
**program builders triggered on clean (Build option tab)
Migration of existing builders to new format (3.0.1 builder to 3.1 builder only)
+
**migration of existing builders to new format (3.0.1 builder to 3.1 builder only)
  
=== Headless Builds ===
+
== Headless Builds ==
Run a headless build, ensuring that:
+
*Run a headless build, ensuring that:
no UI is started
+
**no UI is started
no exceptions occur without a UI
+
**no exceptions occur without a UI
the build does not fail (unexpectedly)
+
**the build does not fail (unexpectedly)
  
=== Headless External Tools ===
+
== Headless External Tools ==
Run a headless external tool, ensuring that
+
*Run a headless external tool, ensuring that
no UI (from Eclipse) is started
+
**no UI (from Eclipse) is started
no exceptions occur without a UI
+
**no exceptions occur without a UI
the tool does not fail (unexpectedly)
+
**the tool does not fail (unexpectedly)
Back to top
+
  
=== Refactoring ===
+
== Refactoring ==
In the Ant Editor
+
*In the Ant Editor
Rename in File (Alt-Shift-R) - ensure linked mode is correct and edits are executed correctly
+
**Rename in File (Alt-Shift-R) - ensure linked mode is correct and edits are executed correctly
Rename Projects
+
*Rename Projects
Build files should be updated
+
**build files should be updated
Ant launch configurations should be updated
+
**Ant launch configurations should be updated
Rename Buildfiles
+
*Rename Buildfiles
Build files should be updated
+
**build files should be updated
Ant launch configurations should be updated
+
**Ant launch configurations should be updated
  
===Preferences ===
+
== Preferences ==
All preference pages must have help topics
+
*all preference pages must have help topics
All preference changes must be cancelable
+
*all preference changes must be cancelable
Ensure correct adherance to the user preference settings for:
+
*ensure correct adherance to the user preference settings for:
Ant runtime settings
+
**Ant runtime settings
Editor customization
+
**editor customization
Ignoring errors / problems
+
**ignoring errors / problems
  
=== Test Suite ===
+
== Test Suite ==
 
All tests should run without failure, and should be runnable individually
 
All tests should run without failure, and should be runnable individually
  
Launch the Ant core test suite from org.eclipse.ant.tests.core
+
*launch the Ant core test suite from org.eclipse.ant.tests.core
Launch the Ant ui test suites from org.eclipse.ant.tests.ui
+
*launch the Ant ui test suites from org.eclipse.ant.tests.ui

Latest revision as of 01:45, 23 September 2015

Platform Ant
Website
Download
Community
Mailing ListForumsIRCmattermost
Bugzilla
OpenHelp WantedBug Day
Contribute
Browse Source

Test Page Template

Test Page Template: Ant/Testing/Test Plan Template.

This template must be used to create a new test page for each release that requires testing.

Any new features that are added to the test script must also be added to the feature list section of the template.

Testing Notes

All tests should be done with resource names of varying length, specifically resource names of length one must be tested (the cause of bug 323588).

Test Passes

Example Build Files

A DBCS build file using Shift_JIS encoding and Japanese characters:

<?xml version="1.0" encoding="Shift_JIS"?>
<project name="jp_shiftjis_test" default="エコー Hello World">
  <target name="エコー Hello World">
    <echo message="こんにちは!"/>
  </target>
 
  <target name="名前の変更 xml to txt" >
    <echo message="私は名前を変更しました。I rename xml to txt"/>
    <move file="Test.xml" tofile="Test.txt"/>
  </target>
</project>

A DBCS build file using UTF-8 encoding and Japanese characters:

<?xml version="1.0" encoding="UTF-8"?>
<project name="jp_utf8_test" default="エコー Hello World">
  <target name="エコー Hello World">
    <echo message="こんにちは!"/>
  </target>
 
  <target name="名前の変更 xml to txt" >
    <echo message="私は名前を変更しました。I rename xml to txt"/>
    <move file="Test.xml" tofile="Test.txt"/>
  </target>
</project>

A build file using the Augment task

<project>
	<path id="path1"/>
	<resource id="resource1"/>
	<propertyset id="property1"/>
	<target name="augment">
		<augment id="path1">
			<pathelement location="/foo"/>
		</augment>
	</target>
</project>

A build file using the extension-point target

<project default="ep-B">
    <target name="A" />
    <extension-point name="ep-B" depends="A" />
    <target name="B" extensionOf="ep-B" depends="D" />
    <target name="C" depends="ep-B" />
    <target name="D" />
</project>

Ant Editor

  • 1.8.x support
    • use the extension-point target
      • content assist should be available
      • mark occurrences should find occurrences
    • use the augment task
      • content assist should be available
      • mark occurrences should find occurrences
      • should not throw exception if referencing unknown id
  • Folding
    • make use of the editor context ruler menu
    • key binding actions: Expand All, Enable folding
    • folding is presented for all the partitions in the Ant buildfile. Partion hover is correct.
  • Navigation
    • properties (from files as well)
    • targets
    • referenced elements
    • macrodefs, taskdefs
    • enable / disable
    • key Modifier and modifier manipulation (preference page)
    • links presented for all reasonable Ant elements
  • Mark Occurrences
    • respecting preference and editor toolbar setting to mark occurrences
    • sticky occurrences setting
    • ensure targets references are correctly highlighted
    • ensure properties references are correctly highlighted
  • Content Assist
    • should respect global preferences for auto-completion, activation, colouring, etc
    • should offer reasonable completions
    • templates should be available and work when applied and run
  • Performance
    • "large" taskdefs (ant-contrib) in Ant Editor
    • "large" imports (need to generate test case) in Ant Editor
    • "large" properties (need to generate test case) in Ant Editor
  • Encoding
    • using an encoded build file like one of the ones from #Example Build Files make sure the editor displays the text correctly. On Windows, make sure you have any required language packs installed and that your default encoding has been set correctly

Ant View

  • General
    • context menu actions
    • DnD
    • add / removal
    • search
    • running targets (default and otherwise)
    • navigation to targets with selection for normal, external entities and imported targets
  • Ant 1.8.x
    • Augment task - should show the element it augments as its child
  • Auto Reconcile
    • ensure correct functionality on / off (use the editor toolbar action with the "refresh" icon)
    • reconcile triggered on save
  • Encoding
    • using an encoded build file like one of the ones from #Example Build Files make sure the names of tasks / types / etc. display properly. On Windows, make sure you have any required language packs installed and that your default encoding has been set correctly

Debugging

  • Buildfiles in the same JRE
    • breakpoints (check target breakpoints as well)
    • stepping (step into, step over for constructs like antcall and ant)
    • dependency stack
    • suspend / resume / terminate
    • run to line support
    • properties update correctly in the variables view
  • Buildfiles in a separate JRE
    • breakpoints (check target breakpoints as well)
    • stepping (step into, step over for constructs like antcall and ant)
    • dependency stack
    • suspend / resume / terminate
    • run to line support
    • properties update correctly in the variables view
  • Encoding
    • using an encoded build file like one of the ones from #Example Build Files make sure the console output from tasks / types / etc. display properly. On Windows, make sure you have any required language packs installed and that your default encoding has been set correctly

Import / Export

  • Export a Java Project to a buildfile containing a javac task etc.
    • error cases
    • classpath including plugin dependencies
  • Import a Java Project from an external buildfile containing a javac task
    • error cases
    • correct links setup for multi-platform
    • correct links for various destdir and sourcedir
    • correct links for various destdir and sourcedir

Input Handling

  • Same JRE
    • with / without valid args
    • cancel
    • -noinput
  • Separate JRE
    • with / without valid args
    • cancel
    • -noinput

Launching

  • Correct UI representation of objects contributed to build (same/separate JRE) in the preferences and launch configuration tabs
    • properties
    • classpath entries
    • tasks
    • types
  • Encoding
    • using an encoded build file like one of the ones from #Example Build Files make sure the names of tasks / types / etc. display properly on the tasks tab in the launch configuration dialog. On Windows, make sure you have any required language packs installed and that your default encoding has been set correctly

External Tools

  • Tool Builders
    • migration from 2.0, 3.0 (with and without prompting)
    • Java projects and simple projects
    • add / remove (ensure cleanup on disk)
    • rename on creation and as an edit (ensure cleanup on disk)
    • executing an Ant build and program
    • same VM / separate VM
    • making use of environment variables
    • Ant builders set to execute different targets for different build kinds (Ant builder target tab)
      • auto - ensure only the targets specified in Auto Build are executed, except for immediately following a clean command, in which case After a Clean targets should be executed.
      • manual - with target(s) set in After a Clean, Manual Build targets should only be executed for manual incremental builds. With no target(s) set for After a Clean, Manual Build targets should run following a clean command AND manual incremental builds.
      • after a clean - ensure these targets are executed in the build immediately following a clean command, unless you set no targets to be executed, in which case the targets from Manual Build should be executed.
      • on a clean - ensure these targets are executed DURING the clean, and that After a Clean targets are executed on the following build.
    • program builders triggered on clean (Build option tab)
    • migration of existing builders to new format (3.0.1 builder to 3.1 builder only)

Headless Builds

  • Run a headless build, ensuring that:
    • no UI is started
    • no exceptions occur without a UI
    • the build does not fail (unexpectedly)

Headless External Tools

  • Run a headless external tool, ensuring that
    • no UI (from Eclipse) is started
    • no exceptions occur without a UI
    • the tool does not fail (unexpectedly)

Refactoring

  • In the Ant Editor
    • Rename in File (Alt-Shift-R) - ensure linked mode is correct and edits are executed correctly
  • Rename Projects
    • build files should be updated
    • Ant launch configurations should be updated
  • Rename Buildfiles
    • build files should be updated
    • Ant launch configurations should be updated

Preferences

  • all preference pages must have help topics
  • all preference changes must be cancelable
  • ensure correct adherance to the user preference settings for:
    • Ant runtime settings
    • editor customization
    • ignoring errors / problems

Test Suite

All tests should run without failure, and should be runnable individually

  • launch the Ant core test suite from org.eclipse.ant.tests.core
  • launch the Ant ui test suites from org.eclipse.ant.tests.ui

Back to the top