Difference between revisions of "DSDP/MTJ/Tests/Manual"

From Eclipsepedia

< DSDP‎ | MTJ‎ | Tests
Jump to: navigation, search
(Project Properties)
(Standard usability tests)
Line 151: Line 151:
 
:* <font color="blue">Check:</font> Can change JAD file name
 
:* <font color="blue">Check:</font> Can change JAD file name
 
:* <font color="blue">Check:</font> Can change JAR file name
 
:* <font color="blue">Check:</font> Can change JAR file name
:* '''Library'''
+
===== <font color="green">'''Library'''</font> =====
:** <font color="blue">Check:</font> Can select a library
+
:* <font color="blue">Check:</font> Can select a library
:** <font color="blue">Check:</font> Can deselect a library
+
:* <font color="blue">Check:</font> Can deselect a library
:** <font color="blue">Check:</font> Can view the details of a library
+
:* <font color="blue">Check:</font> Can view the details of a library
:** <font color="blue">Check:</font> The ''Sample library'' is available to be selected
+
:* <font color="blue">Check:</font> The ''Sample library'' is available to be selected
:** <font color="blue">Check:</font> All libraries that use properly the Extension Point available in ''org.eclipse.mtj.core.library'' Plug-in are available to be selected
+
:* <font color="blue">Check:</font> All libraries that use properly the Extension Point available in ''org.eclipse.mtj.core.library'' Plug-in are available to be selected
  
:* '''Signing'''
+
===== <font color="green">'''Signing'''</font> =====
:** <font color="blue">Check:</font> Can enable/disable project specific settings
+
:* <font color="blue">Check:</font> Can enable/disable project specific settings
:** <font color="blue">Check:</font> Can select keystore from project
+
:* <font color="blue">Check:</font> Can select keystore from project
:*** <font color="blue">Check:</font> Keystore password dialog is opened
+
:** <font color="blue">Check:</font> Keystore password dialog is opened
:**** <font color="blue">Check:</font> Can enter Keystore password on dialog
+
:*** <font color="blue">Check:</font> Can enter Keystore password on dialog
:**** <font color="blue">Check:</font> Can save password on workspace Keyring
+
:*** <font color="blue">Check:</font> Can save password on workspace Keyring
:** <font color="blue">Check:</font> Can select keystore from external folder
+
:* <font color="blue">Check:</font> Can select keystore from external folder
:*** <font color="blue">Check:</font> Keystore password dialog is opened
+
:** <font color="blue">Check:</font> Keystore password dialog is opened
:**** <font color="blue">Check:</font> Can enter Keystore password on dialog
+
:*** <font color="blue">Check:</font> Can enter Keystore password on dialog
:**** <font color="blue">Check:</font> Can save password on workspace Keyring
+
:*** <font color="blue">Check:</font> Can save password on workspace Keyring
:** <font color="blue">Check:</font> Can change Keystore password
+
:* <font color="blue">Check:</font> Can change Keystore password
:** <font color="blue">Check:</font> Can select "prompt for password when required" option
+
:* <font color="blue">Check:</font> Can select "prompt for password when required" option
:*** <font color="blue">Check:</font> When selected passwords text fields are disabled
+
:** <font color="blue">Check:</font> When selected passwords text fields are disabled
:** <font color="blue">Check:</font> Can select "save password in workspace keyring" option
+
:* <font color="blue">Check:</font> Can select "save password in workspace keyring" option
:*** <font color="blue">Check:</font> When selected passwords text fields are enabled
+
:** <font color="blue">Check:</font> When selected passwords text fields are enabled
:** <font color="blue">Check:</font> Can select "save password as part of project" option
+
:* <font color="blue">Check:</font> Can select "save password as part of project" option
:*** <font color="blue">Check:</font> When selected passwords text fields are enabled
+
:** <font color="blue">Check:</font> When selected passwords text fields are enabled
:** <font color="blue">Check:</font> Lists all aliases from the selected keystore
+
:* <font color="blue">Check:</font> Lists all aliases from the selected keystore
:*** <font color="blue">Check:</font> When alias is selected key data is displayed
+
:** <font color="blue">Check:</font> When alias is selected key data is displayed
:** <font color="blue">Check:</font> Can create a new key pair
+
:* <font color="blue">Check:</font> Can create a new key pair
:*** <font color="blue">Check:</font> Generate KeyPair dialog is opened
+
:** <font color="blue">Check:</font> Generate KeyPair dialog is opened
:**** <font color="blue">Check:</font> All fields on the dialog are enabled
+
:*** <font color="blue">Check:</font> All fields on the dialog are enabled
:**** <font color="blue">Check:</font> Can create key pair by selecting ok button
+
:*** <font color="blue">Check:</font> Can create key pair by selecting ok button
:**** <font color="blue">Check:</font> Can cancel operation by selecting cancel button
+
:*** <font color="blue">Check:</font> Can cancel operation by selecting cancel button
:** <font color="blue">Check:</font> Can delete key pair
+
:* <font color="blue">Check:</font> Can delete key pair
:** <font color="blue">Check:</font> Can generate Certificate Signing Request (CSR)
+
:* <font color="blue">Check:</font> Can generate Certificate Signing Request (CSR)
:*** <font color="blue">Check:</font> A dialog for selecting a target folder is opened
+
:** <font color="blue">Check:</font> A dialog for selecting a target folder is opened
:*** <font color="blue">Check:</font> A file with the <alias-name>.csr name is generated on the folder selected on the previous step.
+
:** <font color="blue">Check:</font> A file with the <alias-name>.csr name is generated on the folder selected on the previous step.
:** <font color="blue">Check:</font> Can Import Certificate
+
:* <font color="blue">Check:</font> Can Import Certificate
:*** <font color="blue">Check:</font> A dialog for selecting the certificate file is opened
+
:** <font color="blue">Check:</font> A dialog for selecting the certificate file is opened
:*** <font color="blue">Check:</font> The certificate is imported
+
:** <font color="blue">Check:</font> The certificate is imported
:** <font color="blue">Check:</font> Can Import Certificate Response
+
:* <font color="blue">Check:</font> Can Import Certificate Response
:*** <font color="blue">Check:</font> A dialog for selecting the certificate response file is opened
+
:** <font color="blue">Check:</font> A dialog for selecting the certificate response file is opened
:*** <font color="blue">Check:</font> The certificate is imported
+
:** <font color="blue">Check:</font> The certificate is imported
:** <font color="blue">Check:</font> Can set keystore provider
+
:* <font color="blue">Check:</font> Can set keystore provider
:** <font color="blue">Check:</font> Can set keystore type
+
:* <font color="blue">Check:</font> Can set keystore type
:* '''Obfuscation'''
+
===== <font color="green">'''Obfuscation'''</font> =====
:** <font color="blue">Check:</font> If "Enable project specific settings" is <u>enabled</u>, these settings must override the default configuration
+
:* <font color="blue">Check:</font> If "Enable project specific settings" is <u>enabled</u>, these settings must override the default configuration
:** <font color="blue">Check:</font> Can configure obfuscation settings
+
:* <font color="blue">Check:</font> Can configure obfuscation settings
:* '''Packaging'''
+
===== <font color="green">'''Packaging'''</font> =====
:** <font color="blue">Check:</font> If "Enable project specific settings" is <u>enabled</u>, these settings must override the default configuration
+
:* <font color="blue">Check:</font> If "Enable project specific settings" is <u>enabled</u>, these settings must override the default configuration
:** <font color="blue">Check:</font> Can configure packaging settings
+
:* <font color="blue">Check:</font> Can configure packaging settings
:* '''Preverification'''
+
===== <font color="green">'''Preverification'''</font> =====
:** <font color="blue">Check:</font> If "Enable project specific settings" is <u>enabled</u>, these settings must override the default configuration
+
:* <font color="blue">Check:</font> If "Enable project specific settings" is <u>enabled</u>, these settings must override the default configuration
:** <font color="blue">Check:</font> Can configure preverification settings
+
:* <font color="blue">Check:</font> Can configure preverification settings
  
 
==== <font color="green">Jad Editor</font> ====
 
==== <font color="green">Jad Editor</font> ====

Revision as of 13:43, 24 March 2009

This is the MTJ Manual Test Plan. Tasks are outlined very briefly only. Feel free to do some exploratory testing around the suggested items, or suggest new test cases.

Contents

Standard usability tests

When testing, make sure that the settings changed were correctly applied, producing the expected behavior. E.g. If you select an specific device to run a MIDlet in a run dialog, make sure that you run the MIDlet and the device running is the selected one.

Preferences Pages

  • Walk through all of the preferences items in "Mobile Tools for Java" category and change their values
    • Check: The outcome is what you expect?
    • Check: Are the Preferences documented sufficiently?
    • Check: The preferences can be restored to default values?
Device Management Page
  • Check: Can import Devices
  • Check: Can edit Devices
  • Check: Can delete Devices
  • Check: Can duplicate Devices
Editors Page
  • Check: Can change the configuration of the 'Localization Data Highlighting'
  • Check: The configuration of the 'Localization Data Highlighting' reflects on the editor
New MIDlet Project Page
  • Resource folder field
    • Check: Can specify a resource folder name
    • Check: If you specify a folder name, the next project you create uses it as resource folder?
Over the Air Page
  • Check: Can change settings and the changes work as expected?
Packaging Page
  • Check: Can add/remove "Excluded Manifest Entries"
Obfuscation Page
  • Check: Can configure Proguard settings
Preverification Page
  • Check: "Use Built-in preverifier" can't be selected

Import Device Wizard

  • Check: The Finish button is disabled before finding a device?
  • Check: The Select All and Deselect All buttons are disabled before finding a device?
  • Check: The Refresh button is disabled before specifying the search directory?
  • Check: Choosing a folder without any device inside and pressing Refresh will result in no devices found (the device list will remain empty)
  • Check: Choosing a folder with devices inside and pressing Refresh will result in the device list filled with the devices found
  • Check: Pressing Finish button after finding at least one device, will add the device list to the Device Management Page

Device Editor

  • Basic Page
  • Libraries Page
    • Check: can add and remove libraries
    • Check: can define javadoc for library
    • Check: can define source for library
  • Properties Page
    • Check: Can not edit, add or remove properties.

MIDlet Project

New MIDlet Project Wizard

  • Check: "Java ME" category exists
  • "Java ME" Category
    • Check: MIDlet Project in category
  • MIDlet Suite Properties
    • Check: Can manage Devices
    • Check: Can change Application descriptor name
    • Check: The Next and Finish buttons must be disabled in invalid scenarios, such as:
  1. A configuration is added and Application descriptor field is empty
  2. A configuration is added but Application descriptor field has an invalid value (i.e. something.txt)
  3. No configuration is added and Application descriptor field is empty
  4. No configuration is added and Application descriptor field has valid value (i.e. something.jad)
    • Check: When one or more configuration is added and Application descriptor field has valid value (i.e. something.jad) the Next and Finish buttons must be enabled
  • Select a Library
    • Check: Can select a library
    • Check: Can deselect a library
    • Check: Can view the details of a library
    • Check: The Sample library is available to be selected
    • Check: All libraries that use properly the Extension Point available in org.eclipse.mtj.core.library Plug-in are available to be selected
  • Java settings
    • Check: Can edit Java settings
    • Libraries Tab
      • Check: "Java ME Library" corresponds to the device selected in previous page.
      • Check: If another device is selected, the change is reflected in "Java ME Library"
  • Check: The wizard can be canceled at any time.
  • Check: The user can go back to the previous wizard page at any time
  • Check: The wizard can be finished after filling the project name and specifying a device.

New Java ME MIDlet Wizard

    • Check: Can create a new MIDlet
    • Check: Can create a new MIDlet automatically adding it to the Application Descriptor
    • Check: Can create a new MIDlet without adding it to the Application Descriptor
    • Check: Can create a new MIDlet changing other settings
    • Check: Cannot create a new MIDlet with a previously existing MIDlet name in the same project/package.

New Java ME MIDlet From Template Wizard

  • Page 1
    • Check: Can create a new MIDlet based on the template
    • Check: Can create a new MIDlet automatically adding it to the Application Descriptor
    • Check: Can create a new MIDlet without adding it to the Application Descriptor
    • Check: Can change the MIDlet template.
    • Check: The custom fields change on the page 2 when different templates are selected.
    • Check: The details and permissions change when a new template is selected.
  • Page 2
    • Check: Finish Button is initially disabled.
    • Check: The Finish button is not enabled unless the package name has valid format.
    • Check: The Finish button is not enabled unless a source folder is selected.
    • Check: The Finish button is not enabled unless the MIDlet name is set.
    • Check: The code generated by the default values will compile and run.
  • Splash MIDlet Template
    • Check: The template generates a MIDlet with the user defined name.
    • Check: The template generates a Spash.java Class.
    • Check: The template generates a splash_128x160.png image on the "res" folder.
    • Check: The template generates a splash_176x220.png image on the "res" folder.
    • Check: The template generates a splash_240x320.png image on the "res" folder.
  • Display Manager Template
    • Check: The template generates a MIDlet with the user defined name.
    • Check: The template generates a DisplayManager.java Class.
  • Image Downloader Template
    • Check: The template generates a MIDlet with the user defined name.
    • Check: The template generates a DownloadListener.java Class.
    • Check: The template generates a DownloadManager.java Class.
  • SMS Server Template
    • Check: The template generates a MIDlet with the user defined name.
    • Check: The template generates a SMSListener.java Class.
    • Check: The template generates a SMSServer.java Class.
  • Game Template
    • Check: The template generates a MIDlet with the user defined name.
    • Check: The template generates a GameEngine.java Class.
    • Check: The template generates a GameModel.java Class.
    • Check: The template generates a GameView.java Class.

Project Properties

  • Check: "Mobile Tools for Java" category exists
  • Check: Can manage project configurations (If active configuration switched, the change must reflect on project's "Java ME Library")
  • Check: Can change JAD file name
  • Check: Can change JAR file name
Library
  • Check: Can select a library
  • Check: Can deselect a library
  • Check: Can view the details of a library
  • Check: The Sample library is available to be selected
  • Check: All libraries that use properly the Extension Point available in org.eclipse.mtj.core.library Plug-in are available to be selected
Signing
  • Check: Can enable/disable project specific settings
  • Check: Can select keystore from project
    • Check: Keystore password dialog is opened
      • Check: Can enter Keystore password on dialog
      • Check: Can save password on workspace Keyring
  • Check: Can select keystore from external folder
    • Check: Keystore password dialog is opened
      • Check: Can enter Keystore password on dialog
      • Check: Can save password on workspace Keyring
  • Check: Can change Keystore password
  • Check: Can select "prompt for password when required" option
    • Check: When selected passwords text fields are disabled
  • Check: Can select "save password in workspace keyring" option
    • Check: When selected passwords text fields are enabled
  • Check: Can select "save password as part of project" option
    • Check: When selected passwords text fields are enabled
  • Check: Lists all aliases from the selected keystore
    • Check: When alias is selected key data is displayed
  • Check: Can create a new key pair
    • Check: Generate KeyPair dialog is opened
      • Check: All fields on the dialog are enabled
      • Check: Can create key pair by selecting ok button
      • Check: Can cancel operation by selecting cancel button
  • Check: Can delete key pair
  • Check: Can generate Certificate Signing Request (CSR)
    • Check: A dialog for selecting a target folder is opened
    • Check: A file with the <alias-name>.csr name is generated on the folder selected on the previous step.
  • Check: Can Import Certificate
    • Check: A dialog for selecting the certificate file is opened
    • Check: The certificate is imported
  • Check: Can Import Certificate Response
    • Check: A dialog for selecting the certificate response file is opened
    • Check: The certificate is imported
  • Check: Can set keystore provider
  • Check: Can set keystore type
Obfuscation
  • Check: If "Enable project specific settings" is enabled, these settings must override the default configuration
  • Check: Can configure obfuscation settings
Packaging
  • Check: If "Enable project specific settings" is enabled, these settings must override the default configuration
  • Check: Can configure packaging settings
Preverification
  • Check: If "Enable project specific settings" is enabled, these settings must override the default configuration
  • Check: Can configure preverification settings

Jad Editor

  • Check: Any property edited by the Jad Editor must be updated in the <project_jad_file>.jad
    • Overview Page
      • Check: Can manage configurations on Runtime section.
    • MIDlets Page
    • Optional Properties Page
    • Over the Air Properties Page
    • Push Registry Page
    • Used Defined Properties Page
    • Signing Page
      • Check: Can enable/disable project signing.
      • Check: Lists all aliases from the configured keystore.
      • Check: Can manage JAD permissions:
        • Check: Can order;
        • Check: Can add;
        • Check: Can remove;
        • Check: Can scan code to resolve permissions;

Localization Data Editor

  • *If the Localization is enabled.
  • Check: Is possible to configure the Localization Data on the xml file.
  • Check: Is possible to configure the Localization Data using the Localization Data Editor.
  • Check: Is possible to configure the Localization Data on the generated classes (L10nConstants.java and L10nResources.java).
  • Check: Is possible to configure the Localization Data on the generated properties file (one file for each locale).
  • Check: Any property edited on the Localization Data must be updated in all places that the information is displayed (xml file, generated classes, properties file and Localization Data Editor).

Java Editor

  • Library
    • Check: Classes with NON_ACCESSIBLE access rule* of selected libraries can not be used
    • Check: Classes with DISCOURAGED access rule* of selected libraries can be used but a warning must appear
    • Check: Classes with ACCESSIBLE access rule* of selected libraries can be used
    • Check: Classes with no access rule* of selected libraries can be used
    • Check: Libraries that were not selected can not be used
    • * The Access Rules are defined in the plugin.xml file

Project's Context Sensitive Menu

  • MIDlet Project
    • Check: Can create package
    • Check: If "Proguard" is correctly configured, can create obfuscated package
    • Check: If "Antenna" is correctly configured, can export Antenna build files.
  • Regular Java Project/Other Project
    • Check: Can convert to MIDlet Project

Run/Debug Dialog

  • MIDlet tab
    • Check: Can change project
    • Check: Can run a MIDlet specifying the MIDlet class
    • Check: Can run a MIDlet Suite specifying the JAD URL
    • Check: Can run a MIDlet Suite emulating "Over the Air" deployment
  • Emulation tab
    • Check: Can set a specific device to run/debug
    • Check: If "Specific Device" is selected, can manage devices

Localization Support

  • Add Localization Option
    • Check: Select the MIDlet project pop up menu and verify if the "Add Localization" option inside the "Mobile Tools for Java" sub menu is shown
    • Check: Select the "Add Localization" option and verify if the Localization Wizard is shown
  • Localization Wizard
    • Check: Verify that the "Finish" button is disabled
    • Select: Browse for a folder where the properties files will be saved
    • Check: Verify that the "Finish" button is still disabled
    • Select: Browse for a package where the Localization API classes will be saved
    • Check: Verify that the "Finish" button is enabled
    • Select: Press "Finish" button
    • Check: Check that the L10nResources.java source file is stored at the package previously selected
    • Check: Repeat all steps above and press "Cancel" button
    • Check: No changes must be made to the project

User and Legal Documentation

Verify User Docs

  • Walk through tutorial, Context Help, Check Links, Search feature

Verify Legal

  • Feature Descriptions, Licenses in all source features, Overall license