Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "TPTP Test Tools Project Tests"
(→Manual) |
|||
| Line 5: | Line 5: | ||
== Test Automation == | == Test Automation == | ||
| − | Over time, our goal is to automate | + | Over time, our goal is to automate as many of the manual tests as possible using the [http://www.eclipse.org/tptp/test/documents/gettingstarted/Automated_GUI_Recorder_Getting_Started.html TPTP Automated GUI Recorder (AGR)] to reduce testing costs and improve quality through repeatable tests (for example, BVTs). The below test suites marked in <font color="Red">red</font> are the next candidates for test automation. |
== Build Verification Test (BVT) == | == Build Verification Test (BVT) == | ||
| Line 87: | Line 87: | ||
**/org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_284071.testsuite | **/org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_284071.testsuite | ||
**/org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_deployment_check.testsuite | **/org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_deployment_check.testsuite | ||
| − | |||
*API Recording tests: | *API Recording tests: | ||
| Line 93: | Line 92: | ||
*Datapool tests: | *Datapool tests: | ||
| − | **/org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_importExport.testsuite | + | <font color="Red">**/org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_importExport.testsuite</font> |
**/org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.EncryptedDatapool_editing.testsuite | **/org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.EncryptedDatapool_editing.testsuite | ||
| − | **/org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_editing.testsuite | + | <font color="Red">**/org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_editing.testsuite</font> |
**/org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.Datapool_Running.testsuite | **/org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.Datapool_Running.testsuite | ||
| Line 109: | Line 108: | ||
**/org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.Execution.URLRunner.Remote.testsuite | **/org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.Execution.URLRunner.Remote.testsuite | ||
**/org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.IPV6.testsuite | **/org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.IPV6.testsuite | ||
| + | <font color="Red">**/org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.testsuite<sup>1</sup></font> | ||
| + | |||
| + | <sup>1</sup>: This test suite is covered when executing Test.UI.URLTest.Windows.testsuite or Test.UI.URLTest.Linux.testsuite. | ||
*ASF tests: | *ASF tests: | ||
Revision as of 06:51, 28 September 2009
Contents
Test Project Testing
For more information on the Test Project testing process, see the TPTP Testing Strategy.
Test Automation
Over time, our goal is to automate as many of the manual tests as possible using the TPTP Automated GUI Recorder (AGR) to reduce testing costs and improve quality through repeatable tests (for example, BVTs). The below test suites marked in red are the next candidates for test automation.
Build Verification Test (BVT)
Automated test suites are executed with each Build Verification Test (BVT) (see http://www.eclipse.org/tptp/home/downloads >> Development - <Release Version> >> <Build ID> >> Build Reports >> BVT Results) and reported in each test pass report (see http://www.eclipse.org/tptp >> Development Plan >> Releases Under Development >> Release <Release Version> >> Test Pass Reports).
For more information on the TPTP Build Verification Test (BVT) process, see TPTP Testing Strategy Part 2 - Test Conventions and Best Practices.
JUnit
- Test Tools Core tests:
- /org.eclipse.hyades.test.tools.core.tests/junit/common/util/Test.Execution.CommonRunner.XMLUtil.Test.testsuite
- /org.eclipse.hyades.test.tools.core.tests/junit/Test.HTTP.HttpCookieCacheTest.testsuite
- Test Model tests:
- /org.eclipse.hyades.test.core.tests/junit/model/facade/Test.Model.Facade.Test.testsuite
- /org.eclipse.hyades.test.core.tests/junit/model/Test.Model.Test.Log.Search.Test.testsuite
- /org.eclipse.hyades.test.core.tests/junit/util/Test.Util.EMFUtil.Test.testsuite
- Datapool tests:
- /org.eclipse.hyades.test.ui.datapool.tests/junit/datapool/Test.UI.DatapoolEditor_running.testsuite
- /org.eclipse.hyades.test.ui.datapool.tests/junit/datapool/Test.UI.EncryptedDatapoolEditor_running.testsuite1
- ASF tests:
- /org.eclipse.tptp.test.testservices.tests/junit/Test.IBM15_TestServices.testsuite
- /org.eclipse.tptp.test.testservices.tests/junit/Test.IBM16_TestServices.testsuite
- /org.eclipse.tptp.test.testservices.tests/junit/Test.Sun15_TestServices.testsuite
- /org.eclipse.tptp.test.testservices.tests/junit/Test.Sun16_TestServices.testsuite
1: This TPTP JUnit test suite is NOT executed with the BVTs since it MUST be executed manually (see TPTP JUnit test suite Description).
Full Test Pass
Manual
- Test Core tests:
**/org.eclipse.hyades.test.core.tests/manual/core/Test.Core.Plugin.Test.testsuite
- Test Execution tests:
- /org.eclipse.hyades.execution.harness.tests/manual/defects/Test.Execution.ExecutionHarness.testsuite
- Test Navigator tests:
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_166025.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_195203.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_225157.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_227769.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_231786.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_232070.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_232089.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_232310.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_232397.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_235447.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_235836.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_236426.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_241850.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_242645.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_242701.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_243358.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_243359.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_245539.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_247185.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_247987.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_247989.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_248255.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_248413.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_249587.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_250245.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_250269.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_254459.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_256362.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_262150.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_263267.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_263805.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_263950.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_266728.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_274496.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_277525.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_273111.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_284071.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_deployment_check.testsuite
- API Recording tests:
- /org.eclipse.tptp.test.tools.api.tests/manual/Recorder/Test.UI.APIRecorder.Windows_IA32.testsuite
- Datapool tests:
**/org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_importExport.testsuite
- /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.EncryptedDatapool_editing.testsuite
**/org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_editing.testsuite
- /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.Datapool_Running.testsuite
- JUnit tests:
**/org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.JUnitRunner.testsuite
- JUnit Plug-in tests:
**/org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.JUnitPluginRunner.testsuite
- URL tests:
- /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.Windows.testsuite
- /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.Linux.testsuite
- /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.Execution.URLRunner.Remote.testsuite
- /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.IPV6.testsuite
**/org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.testsuite1
1: This test suite is covered when executing Test.UI.URLTest.Windows.testsuite or Test.UI.URLTest.Linux.testsuite.
- ASF tests:
- /org.eclipse.hyades.test.core.services.tests/manual/Test.Execution.Services.testsuite
- Test Log View tests:
- /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_linux.testsuite
- /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_windows.testsuite
- /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_extensibility.testsuite
- Documentation tests:
- /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.smoke.testsuite
- Executed once per release during TP1 of the last iteration:
- /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.Datapools.technical_review.Windows_IA32.testsuite
- /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.Deployment.technical_review.Windows_IA32.testsuite
- /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.General.technical_review.Windows_IA32.testsuite
- /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.JUnit.technical_review.Windows_IA32.testsuite
- /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.JUnit_Plugin.technical_review.Windows_IA32.testsuite
- /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.Manual.technical_review.Windows_IA32.testsuite
- /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.Test_Log_Viewer.technical_review.Windows_IA32.testsuite
- /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.URL.technical_review.Windows_IA32.testsuite
- /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.ASF.technical_review.Windows_IA32.testsuite
JUnit
- See the JUnit BVT tests.
JUnit Plug-in
- Test Core tests:
- /org.eclipse.hyades.test.core.tests/junit_plugin/launch/facade/Test.Core.Launch.Facade.Deployment.testsuite
- /org.eclipse.hyades.test.core.tests/junit_plugin/launch/facade/Test.Core.Launch.Facade.Test.testsuite
- /org.eclipse.hyades.test.core.tests/junit_plugin/launch/facade/Test.Core.Launch.Facade.TestLog.testsuite
- /org.eclipse.hyades.test.core.tests/junit_plugin/launch/extensions/DeploymentsProviderTest.testsuite
- /org.eclipse.hyades.test.core.tests/junit_plugin/launch/extensions/ExecutionHistoryDefaultsProviderTest.testsuite
- /org.eclipse.hyades.test.core.tests/junit_plugin/launch/extensions/DatapoolHandlerTest.testsuite
- /org.eclipse.hyades.test.core.tests/junit_plugin/model/Test.Model.Datapool.Test.testsuite
- Test Navigator tests:
- /org.eclipse.hyades.test.ui.navigator.tests/junit_plugin/Test.Proxy.FileProxyNodeCacheTest.testsuite
- Datapool tests:
- /org.eclipse.hyades.test.ui.datapool.tests/junit_plugin/datapool/Test.UI.Datapool_Running.testsuite
- /org.eclipse.hyades.test.ui.datapool.tests/junit_plugin/datapool/Test.UI.EncryptedDatapool_Running.testsuite
- /org.eclipse.hyades.test.ui.datapool.tests/junit_plugin/datapool/Test.UI.DatapoolUtilities.testsuite
- JUnit tests:
- /org.eclipse.hyades.test.java.tests/junit/Test.JUnit.CodeSynchronizationTest.testsuite
- /org.eclipse.hyades.test.java.tests/junit/Test.JUnit.JUnitTestGeneration.testsuite
- /org.eclipse.hyades.test.java.tests/junit/Test.JUnit.JUnitTestSuiteFacadeTest.testsuite
- Test Log View tests:
- /org.eclipse.hyades.test.ui.logViewer.tests/junit_plugin/Test.UI.FrameworkEditors_logViewer_UtilitiesTest.testsuite
AGR
- Test Navigator tests:
- /org.eclipse.hyades.test.ui.navigator.tests/gui/Test.UI.TestNavigator_Logical_auto.testsuite
- /org.eclipse.hyades.test.ui.navigator.tests/gui/Test.UI.TestNavigator_Resource_auto.testsuite
- Datapool tests:
- /org.eclipse.hyades.test.ui.datapool.tests/gui/Test.UI.Datapool_Running.testsuite
- JUnit tests:
- /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.IBM15_JUnitRunner.testsuite
- /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.IBM16_JUnitRunner.testsuite
- /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.Sun15_JUnitRunner.testsuite
- /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.Sun16_JUnitRunner.testsuite
- /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.JUnitRunner.Bug186196.testsuite
- /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.JUnitRunner.testsuite
- Test Log View tests:
- /org.eclipse.hyades.test.ui.logViewer.tests/gui/Test.UI.FrameworkEditors_logViewer_linux.testsuite
- /org.eclipse.hyades.test.ui.logViewer.tests/gui/Test.UI.FrameworkEditors_logViewer_windows.testsuite
Smoke Test Pass
Manual
- Test Navigator tests:
- See the Manual Full Test Pass tests.
- API Recording tests:
- See the Manual Full Test Pass tests.
- Datapool tests:
- See the Manual Full Test Pass tests.
- JUnit tests:
- See the Manual Full Test Pass tests.
- JUnit Plug-in tests:
- See the Manual Full Test Pass tests.
- URL tests:
- See the Manual Full Test Pass tests except:
- /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.Linux.testsuite
- /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.IPV6.testsuite
- See the Manual Full Test Pass tests except:
- ASF tests:
- See the Manual Full Test Pass tests.
- Log View tests:
- See the Manual Full Test Pass tests except:
- /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_linux.testsuite
- /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_extensibility.testsuite
- See the Manual Full Test Pass tests except:
JUnit
- See the JUnit Full Test Pass tests.
JUnit Plug-in
- See the JUnit Plug-in Full Test Pass tests.
AGR
- See the AGR Full Test Pass tests.