Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "CDT/Archive/cdt-debug-dsf-gdb"

< CDT‎ | Archive
(Automated testing)
m (Jonah.kichwacoders.com moved page CDT/cdt-debug-dsf-gdb to CDT/Archive/cdt-debug-dsf-gdb)
 
(31 intermediate revisions by 4 users not shown)
Line 1: Line 1:
== Overview ==
+
{{warning|Note: The contents of this page refer to design documents reflecting work in progress of Eclipse CDT features and for some features that have never been implemented. Therefore they may not refer to the current state of CDT. The testing aspects have migrated to be part of the [[https://github.com/eclipse-cdt/cdt-infra/issues?utf8=%E2%9C%93&q=is%3Aissue+label%3Aendgame endgame]] plan.}}
  
== Source ==
+
[[Category:CDT:Archive]]
Source plugins and features are located in CVS at [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cdt/dsf-gdb/?root=Tools_Project /cvsroot/tools/org.eclipse.cdt/dsf-gdb].
+
  
=== Launching ===
 
[http://www.eclipse.org/dsdp/dd/development/relnotes/dd_news-1.1.html#mozTocId11668 N&N Entry]
 
* Launch Configurations
 
** Launch an application on local host
 
** Attach to a running process
 
** Launch an application on a remote host
 
* GDB Debugger options
 
** Non-stop mode selection
 
** Reverse debugging mode selection
 
  
=== CDT Breakpoints Support ===
+
== Overview ==
[http://www.eclipse.org/dsdp/dd/development/relnotes/dd_news-1.1.html#mozTocId83314 N&N Entry]
+
 
+
=== Non-stop mode multi-threaded debugging support ===
+
[http://www.eclipse.org/dsdp/dd/development/relnotes/dd_news-1.1.html#mozTocId379326 N&N Entry]
+
 
+
=== Multi-process Debugging ===
+
[http://www.eclipse.org/dsdp/dd/development/relnotes/dd_news-1.1.html#mozTocId379330 N&N Entry]
+
 
+
=== CLI Console ===
+
[http://www.eclipse.org/dsdp/dd/development/relnotes/dd_news-1.1.html#mozTocId232571 N&N Entry]
+
  
 
== Automated testing ==
 
== Automated testing ==
 +
 +
This wiki content applies to new testing framework introduced 18 Mar 2016 for neon M6 (cdt 9.0), if you looking for instruction for older branches use historical version of this wiki.
  
 
DSF-GDB has a JUnit4 test plugin, org.eclipse.cdt.tests.dsf.gdb, that is stored along-side the source code.  This plugin aims at testing the different DSF-GDB services.  There is currently no automated testing for UI operations.  Such tests could eventually be added, maybe using SWTBOT, but until then, a manual test procedure is describe further down in this page.  The rest of this section describes how to use the DSF-GDB JUnit tests.
 
DSF-GDB has a JUnit4 test plugin, org.eclipse.cdt.tests.dsf.gdb, that is stored along-side the source code.  This plugin aims at testing the different DSF-GDB services.  There is currently no automated testing for UI operations.  Such tests could eventually be added, maybe using SWTBOT, but until then, a manual test procedure is describe further down in this page.  The rest of this section describes how to use the DSF-GDB JUnit tests.
  
The DSF-GDB JUnit tests are aimed to be run on a Linux machine. It may be possible to make them run on Windows or Mac, but it was never attempted.
+
The DSF-GDB JUnit tests are usually run on a Linux machine. At some time they were run on a Windows machine but the procedure is unclear.
  
 
=== Things to know ===
 
=== Things to know ===
Line 38: Line 20:
 
* Tests can be run locally but using gdbserver.  This simulates a remote debugging session.  Dedicated remote test suites exists (see below) to trigger such tests.
 
* Tests can be run locally but using gdbserver.  This simulates a remote debugging session.  Dedicated remote test suites exists (see below) to trigger such tests.
 
* Tracing can be turned on using the Tracing launch tab
 
* Tracing can be turned on using the Tracing launch tab
 +
* Testing with different GDB version is supported
 +
* Automatically running the tests through maven is supported
  
=== Launching the tests ===
+
=== Launching the tests manually ===
  
* First make sure you have the proper GDB binaries you want to test.  The tests are aimed at any of GDB 6.6, 6.7, 6.8 and 7.0.  Depending on the test suite you choose, the tests will be looking for the right GDB version in your PATH.  The names of the gdb binaries should be:
+
* First make sure you have the proper GDB binaries you want to test.  The tests are aimed at any of GDB 6.6 - 7.11, and so on until the latest version of GDB available.  Depending on the test suite you choose, the tests will be looking for the right GDB version in your PATH.  The names of the gdb binaries should be:
 
** gdb.6.6
 
** gdb.6.6
 
** gdb.6.7
 
** gdb.6.7
** gdb.6.8
+
** ...
** gdb.7.0
+
** it is also possible to run on default binary "gdb"
  
 
* For 'remote' tests using gdbserver you must have the right gdbserver in your PATH:
 
* For 'remote' tests using gdbserver you must have the right gdbserver in your PATH:
 
** gdbserver.6.6
 
** gdbserver.6.6
 
** gdbserver.6.7
 
** gdbserver.6.7
** gdbserver.6.8
+
** ...
** gdbserver.7.0
+
** or "gdbserver"
  
 +
If you don't want to use your PATH variable to point to the GDB binaries, you can set the vm variable: cdt.tests.dsf.gdb.path to the directory where the GDB binaries are located.  E.g., -Dcdt.tests.dsf.gdb.path=/home/marc/bin/gdbBinaries
  
 
* Create a new "JUnit Plug-in Test" configuration a choose the following options:
 
* Create a new "JUnit Plug-in Test" configuration a choose the following options:
Line 58: Line 43:
 
*** Select "Run a single test"
 
*** Select "Run a single test"
 
*** The project is "org.eclipse.cdt.tests.dsf.gdb"
 
*** The project is "org.eclipse.cdt.tests.dsf.gdb"
*** Choose a test class from the list described below (e.g., org.eclipse.cdt.tests.dsf.gdb.tests.AllSuites)
+
*** Choose a test class from the list described below (e.g., org.eclipse.cdt.tests.dsf.gdb.tests.AutomatedSuite, AllSuites, AllRemoteSuites
 +
or individual test like MIBreakpointsTest)
 
*** "Test runner" should be JUnit 4
 
*** "Test runner" should be JUnit 4
 +
*** Uncheck "Run in UI thread" (!)
 +
** Under the "Main" tab
 +
*** Under "Program to Run", select "Run an application: [No Application] - Headless Mode"
 +
** Under the Arguments tab
 +
*** If you choose suite or test rather then "All*" you have to specify set of gdb's you want to test
 +
**  In VM Arguments tab define "cdt.tests.dsf.gdb.versions" variable as list of gdb's, i.e. -Dcdt.tests.dsf.gdb.versions=gdb.7.7,gdbserver.7.7,gdb.7.11
 +
*** These are all known at this moment -Dcdt.tests.dsf.gdb.versions=gdb.6.6,gdb.6.7,gdb.6.8,gdb.7.0,gdb.7.1,gdb.7.2,gdb.7.3,gdb.7.4,gdb.7.5,gdb.7.6,gdb.7.7,gdb.7.8,gdb.7.9,gdb.7.10,gdb.7.11,gdbserver.6.6,gdbserver.6.7,gdbserver.6.8,gdbserver.7.0,gdbserver.7.1,gdbserver.7.2,gdbserver.7.3,gdbserver.7.4,gdbserver.7.5,gdbserver.7.6,gdbserver.7.7,gdbserver.7.8,gdbserver.7.9,gdbserver.7.10,gdbserver.7.11
 
** Under the "Tracing" tab (if you want to enable 'gdb traces')
 
** Under the "Tracing" tab (if you want to enable 'gdb traces')
 
*** Check "org.eclipse.cdt.dsf.gdb on the left panel, then check 'debug' on the right panel
 
*** Check "org.eclipse.cdt.dsf.gdb on the left panel, then check 'debug' on the right panel
Line 71: Line 64:
 
Each test class tests a different DSF-GDB service.  Tests suites allow running many test classes consecutively.  Below is the list of the different suites and tests that can be run.  Each element of the list can be chosen as the 'test class' in the launch.
 
Each test class tests a different DSF-GDB service.  Tests suites allow running many test classes consecutively.  Below is the list of the different suites and tests that can be run.  Each element of the list can be chosen as the 'test class' in the launch.
  
* org.eclipse.cdt.tests.dsf.gdb.tests.AllSuites - Runs every single test for every GDB version.  You must make sure you have all the GDB binaries in your PATH.
+
* org.eclipse.cdt.tests.dsf.gdb.tests.AllSuites - Runs every single test for every GDB version.  You must make sure you have all the GDB binaries are in your PATH.
* org.eclipse.cdt.tests.dsf.gdb.tests.tests_6_6.Suite_6_6 - Runs every test for gdb.6.6 (must be in your PATH)
+
* org.eclipse.cdt.tests.dsf.gdb.tests.AllRemoteSuites - Runs every single test for every GDB version in remote mode. You must make sure you have all the gdbserver binaries are in your PATH.
* org.eclipse.cdt.tests.dsf.gdb.tests.tests_6_7.Suite_6_7 - Runs every test for gdb.6.7 (must be in your PATH)
+
* org.eclipse.cdt.tests.dsf.gdb.tests.AutomatedSuite - Runs all tests with gdbs specified by "cdt.tests.dsf.gdb.versions" vm variable, by default it run on "gdb,gdbserver" binaries
* org.eclipse.cdt.tests.dsf.gdb.tests.tests_6_8.Suite_6_8 - Runs every test for gdb.6.8 (must be in your PATH)
+
* Each test file can also be run individually, such as below. Again, the proper binary of GDB must be in your PATH
* org.eclipse.cdt.tests.dsf.gdb.tests.tests_7_0.Suite_7_0 - Runs every test for gdb.7.0 (must be in your PATH)
+
** org.eclipse.cdt.tests.dsf.gdb.tests.tests.GDBProcessesTest
  
  
* org.eclipse.cdt.tests.dsf.gdb.tests.tests_6_6.Suite_Remote_6_6 - Runs every test for gdb.6.6 connecting to a gdbserver.6.6 (both must be in your PATH)
+
=== Writing new tests ===
* org.eclipse.cdt.tests.dsf.gdb.tests.tests_6_7.Suite_Remote_6_7 - Runs every test for gdb.6.7 connecting to a gdbserver.6.7 (both must be in your PATH)
+
* org.eclipse.cdt.tests.dsf.gdb.tests.tests_6_8.Suite_Remote_6_8 - Runs every test for gdb.6.8 connecting to a gdbserver.6.8 (both must be in your PATH)
+
* org.eclipse.cdt.tests.dsf.gdb.tests.tests_7_0.Suite_Remote_7_0 - Runs every test for gdb.7.0 connecting to a gdbserver.7.0 (both must be in your PATH)
+
  
 +
Whenever a new feature is added new JUnit tests will probably called for.  If the test falls into an existing DSF-GDB service, the corresponding test class should be enhanced with some new tests.
  
* org.eclipse.cdt.tests.dsf.gdb.tests.AllTests - Deprecated.  Will run the base tests with the GDB binary 'gdb' that is on the system. The problem is that only the base tests are chosen, which does not adapt to different GDB versions
+
If you adding test case in existing ones just make sure you use "assume" test for the right gdb version.
 +
If some functionality only available (or not) in certain gdb's you can use junit4 assume statement.
 +
Parent class provides the following functions for convenience
 +
* assumeGdbVersionAtLeast("7.4") - test will only be run for gdb 7.4 or higher and ignored for the rest
 +
* assumeGdbVersionLowerThen("6.9") - test will be run only for gdb which is stricly lower than 6.9 (which is 6.8 and lower)
 +
* assumeGdbVersionNot("6.8") - skip the test for a specific gdb.
 +
You can also combine them if need be.
 +
You can also assume for remote if you test only remote (or visa versa), i.e       
 +
Assume.assumeTrue("Skipping non-remote", remote);
  
  
* Each test file can also be run individually, such as belowAgain, the proper binary of GDB must be in your PATH
+
If the feature creates a new service, different new test classes will need to be created. If you adding new test case make sure it extends BaseParametrizedTestCase and use Parametrized running (see others as example). And new test case into SuiteGdb.
** org.eclipse.cdt.tests.dsf.gdb.tests.tests_6_6.GDBProcessesTest_6_6
+
 
** org.eclipse.cdt.tests.dsf.gdb.tests.tests_7_0.GDBProcessesTest_7_0
+
The class org.eclipse.cdt.tests.dsf.gdb.tests.ExampleTest is a template for writing new DSF-GDB JUnit test classes.
 +
 
 +
=== Testing using Maven ===
 +
 
 +
When building the CDT using Maven, the DSF-GDB tests will be run automatically using the latest GDB version as specified in  org.eclipse.cdt.tests.dsf.gdb.tests.AutomatedSuite.javaThis is driven by dsf-gdb/org.eclipse.cdt.tests.dsf.gdb/pom.xml.  Both the location of the GDB binaries and which tests to run can be overridden using the following maven environment variables:
 +
* gdbPathOption=-Dcdt.tests.dsf.gdb.path=<path to GDB binaries>
 +
* gdbTestSuite=<full class name of suite or test file to be run>
 +
For example:
 +
 
 +
mvn clean verify -DgdbPathOption=-Dcdt.tests.dsf.gdb.path=/home/marc/bin/gdbBinaries -DgdbTestSuite=Suite_7_8.java
 +
 
 +
will run the Suite_7_8 while looking for the corresponding GDB binary in /home/marc/bin/gdbBinaries.
 +
 
 +
=== Continuous Integration ===
 +
 
 +
Different jobs have been created on CDT's Hudson instance to test the different configurations for the Debug component.
 +
 
 +
* '''cdt-master''' - This is the overall CDT build job for the master branch.  It includes the DSF-GDB JUnit tests for the latest GDB version.
 +
* '''cdt-verify''' - This is the CDT job for patches pushed to Gerrit.  It includes the DSF-GDB JUnit tests for the latest GDB version.
 +
* '''debug-tests-master-all-gdb-versions''' - This jobs runs the local DSF-GDB tests of the master branch using all GDB versions.  It is a separate job that runs once a day only because it takes a long time.
 +
* '''debug-remote-tests-master-all-gdb-versions''' - This jobs runs the DSF-GDB tests of the master branch with gdbserver using all GDB versions.  It is a separate job that runs once a day only because it takes a long time.  It is separate from the 'debug-tests-master-all-gdb-versions' job because the tests have the same names when running locally or with gdbserver, and would overwrite each other if put in the same job.
 +
* '''debug-tests-master-gdb-master''' - This job runs the DSF-GDB tests of the master branch against a GDB built from its own master branch.  The idea is to detect any regression in GDB itself and allow us to notify the GDB community before a release.
 +
* '''debug-tests-master-gdb-<version>-branch''' - This job runs the DSF-GDB tests of the master branch against a GDB built from GDB's <version> branch.  This <version> branch is the one used for an upcoming GDB release.  The idea is to detect any regression in GDB's upcoming maintenance release and allow us to notify the GDB community before the release.
 +
 
 +
=== What to do when a new GDB version is released ===
 +
 
 +
This section lists the steps that should be done in order, soon after a new GDB release, to keep CDT's infrastructure up-to-date.
 +
 
 +
# On master, update the CDT's [http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/tree/dsf-gdb/org.eclipse.cdt.tests.dsf.gdb/scripts/download-build-gdb.sh script that downloads and builds GDBs] to include the new GDB version, e.g. adding 7.11
 +
# Once the download script is updated and committed to master, run the Hudson job called [https://hudson.eclipse.org/cdt/job/debug_manual_gdb_download debug_manual_gdb_download] and specify the new version number when prompted, e.g., 7.11.  This will download and build the new GDB version to CDT's Hudson instance.
 +
# Update the org.eclipse.cdt.tests.dsf.gdb Debug's [http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/tree/dsf-gdb/org.eclipse.cdt.tests.dsf.gdb/pom.xml pom.xml] to point to the suite of tests matching the new GDB version in the surefire plugin setting (This is not needed for maintenance releases since the previous step makes a soflink to it)
 +
# Add the new gdb version constant in the ITestConstants class and to ITestConstants#ALL_SUPPORTED_VERSIONS. (This is not needed for maintenance releases since a step above makes a soflink to it)
 +
 
 +
With the above changes, both the master and verify HIPP jobs will test with the newly released GDB, and the two test jobs running the different GDB versions will include the new one.  As for the "debug-tests-master-gdb-<version>-branch", it should continue pointing to the maintenance branch of the newly released GDB version; this will allow to test the upcoming maintenance release, e.g., 7.11.1.
 +
 
 +
=== What to do when a new GDB branch is created ===
 +
Before a new release, the GDB community will create a new branch to prepare for that release.  At that time, CDT's infrastructure should be updated to test with the new branch instead of the old maintenance branch.  The steps are listed below.
  
== Writing new tests ==
 
  
Whenever a new feature is added new JUnit tests will probably called for. If the test falls into an existing DSF-GDB service, the corresponding test class should be enhanced with some new tests. Usually, it is not necessary to worry about GDB version when adding tests, as long as they are added in the base class.  However, in some cases, the test should be different depending on the GDB version; in those cases, different test classes (based on the GDB version) will need to be updated.
+
* On [https://hudson.eclipse.org/cdt CDT's HIPP], rename the job "debug-tests-master-gdb-<version>-branch" to update the version in the name
 +
* On that same job, change the configuration to use the new branch:
 +
** Under section "Source Code Management"
 +
*** Under the section "URL of repository" of the GDB repo, press "Advanced..."
 +
*** Modify the newly shown "Refspec..." to fetch the new branch, as it is named in the GDB git repo
 +
*** Change the relevant "Branch Specifier..." to the new GDB branch name, as it is named in the GDB git repo
 +
** Under section "Build"
 +
*** Modify the last command part of the '''Execute shell''' section to use a gdb named using the new gdb version, as expected by the JUnit tests created as the first step to this procedure
 +
*** Modify the last line part of the second '''Invoke Maven 3''' section to run the new JUnit test suite for the new GDB version.
 +
** Save the configuration change
 +
** Test the new job by building it right away
 +
* Also update the "debug-tests-master-gdb-master" to change the configuration to use the new GDB version to use the latest tests:
 +
** Under section "Build"
 +
*** Modify the last command part of the '''Execute shell''' section to use a gdb named using the new gdb version, as expected by the JUnit tests created as the first step to this procedure
 +
*** Modify the last line part of the second '''Invoke Maven 3''' section to run the new JUnit test suite for the new GDB version.
 +
** Save the configuration change
 +
** Test the new job by building it right away
  
If the feature creates a new service, different new test classes will need to be created.  Depending on which GDB version will be used with the new service, different test classes will be created: a base test class, and a derived class for each GDB versionAlso, the name of the new test classes must be added to all the relevant test Suite classes.
+
With the above changes, the "debug-tests-master-gdb-<version>-branch" job will testing with the GDB code for the upcoming releaseIf any test failures that are caused by invalid changes to GDB should be reported to the GDB community ASAP in hopes of getting them fixed before the release.
  
 
== Manual Test Procedures ==
 
== Manual Test Procedures ==

Latest revision as of 15:11, 22 January 2020

Warning2.png
Note: The contents of this page refer to design documents reflecting work in progress of Eclipse CDT features and for some features that have never been implemented. Therefore they may not refer to the current state of CDT. The testing aspects have migrated to be part of the [endgame] plan.


Contents

Overview

Automated testing

This wiki content applies to new testing framework introduced 18 Mar 2016 for neon M6 (cdt 9.0), if you looking for instruction for older branches use historical version of this wiki.

DSF-GDB has a JUnit4 test plugin, org.eclipse.cdt.tests.dsf.gdb, that is stored along-side the source code. This plugin aims at testing the different DSF-GDB services. There is currently no automated testing for UI operations. Such tests could eventually be added, maybe using SWTBOT, but until then, a manual test procedure is describe further down in this page. The rest of this section describes how to use the DSF-GDB JUnit tests.

The DSF-GDB JUnit tests are usually run on a Linux machine. At some time they were run on a Windows machine but the procedure is unclear.

Things to know

  • Each JUnit test is independent of the others. To achieve this, each JUnit test is a full-fledged DSF-GDB launch, with its own GDB instance.
  • Tests can be run locally, using a local GDB session
  • Tests can be run locally but using gdbserver. This simulates a remote debugging session. Dedicated remote test suites exists (see below) to trigger such tests.
  • Tracing can be turned on using the Tracing launch tab
  • Testing with different GDB version is supported
  • Automatically running the tests through maven is supported

Launching the tests manually

  • First make sure you have the proper GDB binaries you want to test. The tests are aimed at any of GDB 6.6 - 7.11, and so on until the latest version of GDB available. Depending on the test suite you choose, the tests will be looking for the right GDB version in your PATH. The names of the gdb binaries should be:
    • gdb.6.6
    • gdb.6.7
    • ...
    • it is also possible to run on default binary "gdb"
  • For 'remote' tests using gdbserver you must have the right gdbserver in your PATH:
    • gdbserver.6.6
    • gdbserver.6.7
    • ...
    • or "gdbserver"

If you don't want to use your PATH variable to point to the GDB binaries, you can set the vm variable: cdt.tests.dsf.gdb.path to the directory where the GDB binaries are located. E.g., -Dcdt.tests.dsf.gdb.path=/home/marc/bin/gdbBinaries

  • Create a new "JUnit Plug-in Test" configuration a choose the following options:
    • Under the "Test" tab:
      • Select "Run a single test"
      • The project is "org.eclipse.cdt.tests.dsf.gdb"
      • Choose a test class from the list described below (e.g., org.eclipse.cdt.tests.dsf.gdb.tests.AutomatedSuite, AllSuites, AllRemoteSuites

or individual test like MIBreakpointsTest)

      • "Test runner" should be JUnit 4
      • Uncheck "Run in UI thread" (!)
    • Under the "Main" tab
      • Under "Program to Run", select "Run an application: [No Application] - Headless Mode"
    • Under the Arguments tab
      • If you choose suite or test rather then "All*" you have to specify set of gdb's you want to test
    • In VM Arguments tab define "cdt.tests.dsf.gdb.versions" variable as list of gdb's, i.e. -Dcdt.tests.dsf.gdb.versions=gdb.7.7,gdbserver.7.7,gdb.7.11
      • These are all known at this moment -Dcdt.tests.dsf.gdb.versions=gdb.6.6,gdb.6.7,gdb.6.8,gdb.7.0,gdb.7.1,gdb.7.2,gdb.7.3,gdb.7.4,gdb.7.5,gdb.7.6,gdb.7.7,gdb.7.8,gdb.7.9,gdb.7.10,gdb.7.11,gdbserver.6.6,gdbserver.6.7,gdbserver.6.8,gdbserver.7.0,gdbserver.7.1,gdbserver.7.2,gdbserver.7.3,gdbserver.7.4,gdbserver.7.5,gdbserver.7.6,gdbserver.7.7,gdbserver.7.8,gdbserver.7.9,gdbserver.7.10,gdbserver.7.11
    • Under the "Tracing" tab (if you want to enable 'gdb traces')
      • Check "org.eclipse.cdt.dsf.gdb on the left panel, then check 'debug' on the right panel
    • Press 'Debug'
      • A new window will (slowly) appear
      • You will see header printouts on the console of your development Eclipse for each test
      • If you've enabled tracing, you will see the gdb communication your development Eclipse

Available tests

Each test class tests a different DSF-GDB service. Tests suites allow running many test classes consecutively. Below is the list of the different suites and tests that can be run. Each element of the list can be chosen as the 'test class' in the launch.

  • org.eclipse.cdt.tests.dsf.gdb.tests.AllSuites - Runs every single test for every GDB version. You must make sure you have all the GDB binaries are in your PATH.
  • org.eclipse.cdt.tests.dsf.gdb.tests.AllRemoteSuites - Runs every single test for every GDB version in remote mode. You must make sure you have all the gdbserver binaries are in your PATH.
  • org.eclipse.cdt.tests.dsf.gdb.tests.AutomatedSuite - Runs all tests with gdbs specified by "cdt.tests.dsf.gdb.versions" vm variable, by default it run on "gdb,gdbserver" binaries
  • Each test file can also be run individually, such as below. Again, the proper binary of GDB must be in your PATH
    • org.eclipse.cdt.tests.dsf.gdb.tests.tests.GDBProcessesTest


Writing new tests

Whenever a new feature is added new JUnit tests will probably called for. If the test falls into an existing DSF-GDB service, the corresponding test class should be enhanced with some new tests.

If you adding test case in existing ones just make sure you use "assume" test for the right gdb version. If some functionality only available (or not) in certain gdb's you can use junit4 assume statement. Parent class provides the following functions for convenience

  • assumeGdbVersionAtLeast("7.4") - test will only be run for gdb 7.4 or higher and ignored for the rest
  • assumeGdbVersionLowerThen("6.9") - test will be run only for gdb which is stricly lower than 6.9 (which is 6.8 and lower)
  • assumeGdbVersionNot("6.8") - skip the test for a specific gdb.

You can also combine them if need be. You can also assume for remote if you test only remote (or visa versa), i.e Assume.assumeTrue("Skipping non-remote", remote);


If the feature creates a new service, different new test classes will need to be created. If you adding new test case make sure it extends BaseParametrizedTestCase and use Parametrized running (see others as example). And new test case into SuiteGdb.

The class org.eclipse.cdt.tests.dsf.gdb.tests.ExampleTest is a template for writing new DSF-GDB JUnit test classes.

Testing using Maven

When building the CDT using Maven, the DSF-GDB tests will be run automatically using the latest GDB version as specified in org.eclipse.cdt.tests.dsf.gdb.tests.AutomatedSuite.java. This is driven by dsf-gdb/org.eclipse.cdt.tests.dsf.gdb/pom.xml. Both the location of the GDB binaries and which tests to run can be overridden using the following maven environment variables:

  • gdbPathOption=-Dcdt.tests.dsf.gdb.path=<path to GDB binaries>
  • gdbTestSuite=<full class name of suite or test file to be run>

For example:

mvn clean verify -DgdbPathOption=-Dcdt.tests.dsf.gdb.path=/home/marc/bin/gdbBinaries -DgdbTestSuite=Suite_7_8.java

will run the Suite_7_8 while looking for the corresponding GDB binary in /home/marc/bin/gdbBinaries.

Continuous Integration

Different jobs have been created on CDT's Hudson instance to test the different configurations for the Debug component.

  • cdt-master - This is the overall CDT build job for the master branch. It includes the DSF-GDB JUnit tests for the latest GDB version.
  • cdt-verify - This is the CDT job for patches pushed to Gerrit. It includes the DSF-GDB JUnit tests for the latest GDB version.
  • debug-tests-master-all-gdb-versions - This jobs runs the local DSF-GDB tests of the master branch using all GDB versions. It is a separate job that runs once a day only because it takes a long time.
  • debug-remote-tests-master-all-gdb-versions - This jobs runs the DSF-GDB tests of the master branch with gdbserver using all GDB versions. It is a separate job that runs once a day only because it takes a long time. It is separate from the 'debug-tests-master-all-gdb-versions' job because the tests have the same names when running locally or with gdbserver, and would overwrite each other if put in the same job.
  • debug-tests-master-gdb-master - This job runs the DSF-GDB tests of the master branch against a GDB built from its own master branch. The idea is to detect any regression in GDB itself and allow us to notify the GDB community before a release.
  • debug-tests-master-gdb-<version>-branch - This job runs the DSF-GDB tests of the master branch against a GDB built from GDB's <version> branch. This <version> branch is the one used for an upcoming GDB release. The idea is to detect any regression in GDB's upcoming maintenance release and allow us to notify the GDB community before the release.

What to do when a new GDB version is released

This section lists the steps that should be done in order, soon after a new GDB release, to keep CDT's infrastructure up-to-date.

  1. On master, update the CDT's script that downloads and builds GDBs to include the new GDB version, e.g. adding 7.11
  2. Once the download script is updated and committed to master, run the Hudson job called debug_manual_gdb_download and specify the new version number when prompted, e.g., 7.11. This will download and build the new GDB version to CDT's Hudson instance.
  3. Update the org.eclipse.cdt.tests.dsf.gdb Debug's pom.xml to point to the suite of tests matching the new GDB version in the surefire plugin setting (This is not needed for maintenance releases since the previous step makes a soflink to it)
  4. Add the new gdb version constant in the ITestConstants class and to ITestConstants#ALL_SUPPORTED_VERSIONS. (This is not needed for maintenance releases since a step above makes a soflink to it)

With the above changes, both the master and verify HIPP jobs will test with the newly released GDB, and the two test jobs running the different GDB versions will include the new one. As for the "debug-tests-master-gdb-<version>-branch", it should continue pointing to the maintenance branch of the newly released GDB version; this will allow to test the upcoming maintenance release, e.g., 7.11.1.

What to do when a new GDB branch is created

Before a new release, the GDB community will create a new branch to prepare for that release. At that time, CDT's infrastructure should be updated to test with the new branch instead of the old maintenance branch. The steps are listed below.


  • On CDT's HIPP, rename the job "debug-tests-master-gdb-<version>-branch" to update the version in the name
  • On that same job, change the configuration to use the new branch:
    • Under section "Source Code Management"
      • Under the section "URL of repository" of the GDB repo, press "Advanced..."
      • Modify the newly shown "Refspec..." to fetch the new branch, as it is named in the GDB git repo
      • Change the relevant "Branch Specifier..." to the new GDB branch name, as it is named in the GDB git repo
    • Under section "Build"
      • Modify the last command part of the Execute shell section to use a gdb named using the new gdb version, as expected by the JUnit tests created as the first step to this procedure
      • Modify the last line part of the second Invoke Maven 3 section to run the new JUnit test suite for the new GDB version.
    • Save the configuration change
    • Test the new job by building it right away
  • Also update the "debug-tests-master-gdb-master" to change the configuration to use the new GDB version to use the latest tests:
    • Under section "Build"
      • Modify the last command part of the Execute shell section to use a gdb named using the new gdb version, as expected by the JUnit tests created as the first step to this procedure
      • Modify the last line part of the second Invoke Maven 3 section to run the new JUnit test suite for the new GDB version.
    • Save the configuration change
    • Test the new job by building it right away

With the above changes, the "debug-tests-master-gdb-<version>-branch" job will testing with the GDB code for the upcoming release. If any test failures that are caused by invalid changes to GDB should be reported to the GDB community ASAP in hopes of getting them fixed before the release.

Manual Test Procedures

GDB Basic Sanity Test

    1. Check out and build the sanity test project from /cvsroot/dsdp/org.eclipse.dd.dsf/tests/SanityTest
    2. Launch the DSF debugger with break at main
    3. Follow instructions in the test.

Launching

  • Perspective
    1. In Preferences->Run/Debug->Perspectives, make sure that at least one perspective change option is set to prompt.
    2. Set the java or c++ perspective
    3. Launch a DSF program
      • Verify that there is a prompt to switch perspective to Debug, Answer yes
      • Verify the perspective switches
  • Expanding after launch
      • Verify from the above test that the Debug view has a selection to the top-most thread
    1. Close Debug view
    2. Open Debug view
    3. Verify that the Debug view has a selection to the top-most thread
  • Break on main
    1. Set a valid break-on-main symbol, and launch.
      • Check: the program breaks at the correct location.
    2. Set an invalid break-on-main symbol, and launch.
      • Check: the launch should fail with an error from creating the break-on-main breakpoint
    3. Set a break-on-main symbol that is not reachable by the program, but the program continues running, and launch.
      • Check: the launch should complete as if there was no break-on-main symbol set.
    4. Set a break-on-main symbol that is not reachable by the program, but and the program exits quickly.
      • Check: the launch should complete and the program should run and terminate normally.
  • Remote Launch
    1. Start a gdbserver session with the binary of the eclipse C++ project
    2. Use the DSF Remote launch to connect to that gdbserver
      • Verify that the debugging session is the same as a local debugging session
      • Verify that the Restart button is grayed out
  • Local Attach Launch
    1. Start the binary of your project outside of eclipse (this binary should take a long time to execute to give you enough time to attach to it)
    2. Use the DSF Attach to Local applicaiton launch
      • Verify that a popup window prompts for the process to attach to
    3. Select the binary you started
      • Verify that the debugging session is the same as a local debugging session
      • Verify that the Restart button is grayed out
      • Verify that the output of the binary remains on the terminal where it was started
  • Remote Attach Launch
    1. Start a gdbserver session with the --multi flag and no binary
    2. Start the binary of your project outside of eclipse (this binary should take a long time to execute to give you enough time to attach to it)
    3. Use the DSF Attach launch to connect to that gdbserver using the 'gdbserver Debugger' type
      • Verify that a popup window prompts for the process pid to attach to (no list)
      • Put in the pid of the binary you started
      • Verify that the debugging session is the same as a local debugging session
      • Verify that the Restart button is grayed out
      • Verify that the output of the binary remains on the terminal where it was started
  • Output
    1. Launch a local debug session
    2. Step the program to execute a couple of instructions that have a printf
      • Verify that the output of the program is properly seen in a separate console.
  • Restart
    1. Launch a local debug session
    2. Step the program to execute a couple of instructions
    3. Press the Restart button and verify that the program restarts from the beginning
      • Verify also that the "Break on main option is still respected on the restart
      • Verify also that the output of the program is properly seen in a separate console.

Line Breakpoints (General)

Test Setup

  1. To avoid stack overflow, set memory to 512M
    1. In the launch configuration, Arguments tab, VM arguments section: -Xmx512M
  2. To ensure integrity between the UI and the back-end, all verifications (Check:) are performed from 3 points:
    1. Back-end activities with GDB/MI traces
    2. Back-end breakpoint status with "info break" at the console
    3. Platform breakpoint status in Breakpoint View (with Breakpoint Properties)
  3. Launch the Eclipse workbench
  4. Start a DSF debug session (DsfBreakpoints)

Test 1: Breakpoint setting

    1. In the editor, add a new line breakpoint (BP-1)
      • Check: A new breakpoint is inserted and it is enabled
    2. In the view menu: enable the "Show Full Paths" check box
      • Check: The breakpoint full path is displayed in the view
    3. In the editor, disable the "Show Full Paths" check box
      • Check: The breakpoint full path is not displayed in the view

Test 2: Breakpoint activation

    1. In the editor, disable BP-1
      • Check: BP-1 is disabled
    2. In the editor, enable BP-1
      • Check: BP-1 is enabled
    3. From the breakpoint context menu, disable BP-1
      • Check: BP-1 is disabled
    4. From the breakpoint context menu, enable BP-1
      • Check: BP-1 is enabled

Test 3: Breakpoint condition

    1. Set an invalid condition on BP-1
      • Check: The update is rejected and BP-1 is not changed
    2. Set a valid condition on BP-1
      • Check: The update is accepted and BP-1 has the new condition
    3. Set an invalid condition on BP-1
      • Check: The update is rejected and BP-1 kept the valid condition
    4. Clear the BP-1 condition
      • Check: The update is accepted and BP-1 no longer has a condition

Test 4: Breakpoint ignore count

    1. Set an ignore count on BP-1
      • Check: The update is accepted and BP-1 has the new ignore count
    2. Reset an ignore count on BP-1 (set to 0)
      • Check: The update is accepted and BP-1 has no ignore count
  • Test 5: Breakpoint deletion (simple)
    1. From the GUI, delete BP-1
      • Check: BP-1 is removed

Test 6: Breakpoint persistence

    1. Create 4 breakpoints with the following characteristics:
      1. BP-1: enabled, no condition, no ignore count
      2. BP-2: disabled, with condition, no ignore count
      3. BP-3: disabled, no condition, with ignore count
      4. BP-4: enabled, with condition, with ignore count
    2. Check: The 4 breakpoints are correctly created
    3. Terminate the debugging session
    4. Start a new debugging session
      • Check: The 4 breakpoints are correctly restored
    5. Terminate the debugging session
    6. Enable all breakpoints
    7. Start a new debugging session
      • Check: The 4 breakpoints are correctly restored (and enabled)

Test 7: Import/export breakpoints to file

    1. Create 4 breakpoints with the following characteristics (re-use the ones from previous step):
      1. BP-1: enabled, no condition, no ignore count
      2. BP-2: disabled, with condition, no ignore count
      3. BP-3: disabled, no condition, with ignore count
      4. BP-4: enabled, with condition, with ignore count
      • Check: The 4 breakpoints are correctly created
    2. Export the 4 breakpoints to file
      • Check: The breakpoints file is created
    3. Remove all breakpoints
      • Check: The 4 breakpoints are removed
    4. Import the breakpoint file
      • Check: The 4 breakpoints are correctly restored
    5. Remove all breakpoints
    6. Terminate the debugging session
    7. Start a new debugging session
      • Check: No breakpoint is installed
    8. Import the breakpoint file
      • Check: The 4 breakpoints are correctly restored

Test 8: Breakpoint deletion (multiple)

    1. From the GUI, delete BP-1, BP-2, BP-3 and BP-4
      • Check: the 4 breakpoints are removed

Watchpoints (General)

Test Setup

  1. To avoid stack overflow, set memory to 512M
    1. In the launch configuration, Arguments tab, VM arguments section: -Xmx512M
  2. To ensure integrity between the UI and the back-end, all verifications (Check:) are performed from 3 points:
    1. Back-end activities with GDB/MI traces
    2. Back-end breakpoint status with "info break" at the console
    3. Platform breakpoint status in Breakpoint View (with Breakpoint Properties)
  3. Launch the Eclipse workbench
  4. Start a DSF debug session (DsfBreakpoints)

Test 1: Watchpoint setting

    1. In the Breakpoint View, add a new watchpoint (WP-1)
      • Check: A new watchpoint is inserted and it is enabled

Test 2: Watchpoint activation

    1. From the watchpoint context menu. disable WP-1
      • Check: WP-1 is disabled
    2. From the watchpoint context menu, enable WP-1
      • Check: WP-1 is enabled

Test 3: Watchpoint condition

    1. Set an invalid condition on WP-1
      • Check: The update is rejected and WP-1 is not changed
    2. Set a valid condition on WP-1
      • Check: The update is accepted and WP-1 has the new condition
    3. Set an invalid condition on WP-1
      • Check: The update is rejected and WP-1 kept the valid condition
    4. Clear the WP-1 condition
      • Check: The update is accepted and WP-1 no longer has a condition

Test 4: Watchpoint ignore count

    1. Set an ignore count on WP-1
      • Check: The update is accepted and WP-1 has the new ignore count
    2. Reset an ignore count on WP-1 (set to 0)
      • Check: The update is accepted and WP-1 has no ignore count

Test 5: Watchpoint deletion (simple)

    1. From the GUI, delete WP-1
      • Check: WP-1 is removed

Test 6: Watchpoint persistence

    1. Create 4 watchpoints with the following characteristics:
      1. WP-1 enabled, write, no condition, no ignore count
      2. WP-2 disabled, read, with condition, no ignore count
      3. WP-3 disabled, read/write, no condition, with ignore count
      4. WP-4 enabled, write, with condition, with ignore count
      • Check: The 4 watchpoints are correctly created
    2. Terminate the debugging session
    3. Start a new debugging session
      • Check: The 4 watchpoints are correctly restored
    4. Terminate the debugging session
    5. Enable all watchpoints
    6. Start a new debugging session
      • Check: The 4 watchpoints are correctly restored (and enabled)

Test 7: Import/export watchpoints to file

    1. Create 4 watchpoints with the following characteristics (re-use watchpoints from previous step):
      1. WP-1 enabled, write, no condition, no ignore count
      2. WP-2 disabled, read, with condition, no ignore count
      3. WP-3 disabled, read/write, no condition, with ignore count
      4. WP-4 enabled, write, with condition, with ignore count
      • Check: The 4 watchpoints are correctly created
    2. Export the 4 watchpoints to file
      • Check: The watchpoints file is created
    3. Remove all watchpoints
      • Check: The 4 watchpoints are removed
    4. Import the watchpoints file
      • Check: The 4 watchpoints are correctly restored
    5. Remove all watchpoints
    6. Terminate the debugging session
    7. Start a new debugging session
      • Check: No watchpoints is installed
    8. Import the watchpoints file
      • Check: The 4 watchpoints are correctly restored

Test 8: Watchpoint deletion (multiple)

    1. From the GUI, delete WP-1, WP-2, WP-3 and WP-4
      • Check: The 4 watchpoints are removed

Skip All (Breakpoints) Button

Test Setup

  1. To avoid stack overflow, set memory to 512M
    1. In the launch configuration, Arguments tab, VM arguments section: -Xmx512M
  1. To ensure integrity between the UI and the back-end, all verifications (Check:) are performed from 3 points:
    1. Back-end activities with GDB/MI traces
    2. Back-end breakpoint status with "info break" at the console
    3. Platform breakpoint status in Breakpoint View (with Breakpoint Properties)
  2. Launch the Eclipse workbench
  3. Create 3 line breakpoints (anything will do)
  4. Start a DSF debug session (DsfBreakpoints)
    • Check: Breakpoints are created and enabled

Test 1: Simple ON/OFF

    1. Toggle the 'Skip All Button' SAB to ON
      • Check: BPs 1, 2, 3 are disabled
    2. Toggle the SAB to OFF
      • Check: BPs 1, 2, 3 are enabled

Test 2: ON/OFF with some disabled BPs

    1. Disable BP 2
      • Check: Only BP-2 is disabled
    2. Toggle the SAB to ON
      • Check: All breakpoints are disabled
    3. Toggle the SAB to OFF
      • Check: BPs 1, 3 are enabled
    4. Disable BP 1 and 3
    5. Enable BP 2
      • Check: Only BP-2 is enabled
    6. Toggle the SAB to ON
      • Check: All breakpoints are disabled
    7. Toggle the SAB to OFF
      • Check: Only BP-2 is enabled

Test 3: ON/OFF with all BPs disabled

    1. Disable BPs 1, 2, 3
    2. Toggle the SAB to ON
      • Check: Nothing happens
    3. Toggle the SAB to OFF
      • Check: Nothing happens

Test 4: Change BPs state while OFF

    1. Enable BP 2
    2. Disable BPs 1, 3
    3. Toggle the SAB to ON
      • Check: BP 2 is disabled
    4. Enable BP 1
      • Check: Nothing happens
    5. Disable BP 2
      • Check: Nothing happens
    6. Toggle the SAB to OFF
      • Check: BP 1 is enabled

Test 6: Startup

    1. Terminate DSF debug session
    2. Enable BPs 1, 2, 3
    3. Toggle the SAB to ON
    4. Start DSF debug session
      • Check: BPs 1, 2, 3 are created and disabled
    5. Toggle the SAB to OFF
      • Check: BPs 1, 2, 3 are enabled

Breakpoint Thread Filters

Test Setup

  1. To avoid stack overflow, set memory to 512M
    1. In the launch configuration, Arguments tab, VM arguments section: -Xmx512M
  2. To ensure integrity between the UI and the back-end, all verifications (Check:) are performed from 3 points:
    1. Back-end activities with GDB/MI traces
    2. Back-end breakpoint status with "info break" at the console
    3. Platform breakpoint status in Breakpoint View (with Breakpoint Properties)
  3. Launch the Eclipse workbench
    1. Create a breakpoint on the return statement of the printMsg function ("return NULL")
    2. Start a DSF debug session (DsfThreads)
      • Check: Breakpoint is created and enabled
    3. Run to breakpoint
      • Check: There are 3 threads (1, 2 and 3) in the Debug View

Test 1: Filter out a thread

    1. Set filter for threads 1, 2
      • Check: The original breakpoint is removed
      • Check: 2 identical breakpoints are installed, one for each thread selected

Test 2: Filtered thread substitution

    1. Change filter for threads 1, 3
      • Check: The breakpoint for thread 2 is removed
      • Check: 2 identical breakpoints are installed, one for each thread selected

Test 3: Remove a filtered thread

    1. Change filter for thread 2 only
      • Check: The breakpoints for threads 1, 3 are removed
      • Check: A breakpoint is installed for thread 2

Test 4: Add a filtered thread

    1. Change filter for thread 2, 3
      • Check: 2 identical breakpoints are installed, one for each thread selected

Test 5: Remove thread filters

    1. Remove filter
      • Check: The breakpoints for threads 2, 3 are removed
      • Check: A new breakpoint is set with no thread filter

Test 6: Initial state propagation

    1. Add a valid condition and ignore count to the breakpoint
      • Check: The breakpoint is updated
    2. Add filter for threads 1, 2
      • Check: The original breakpoint is removed
      • Check: 2 new breakpoints are set with the correct condition, ignore count, one for each thread selected

Test 7: Invalid condition propagation

    1. Change to an invalid condition
      • Check: Updates failed
      • Check: Breakpoints aren't changed (previous valid condition is kept)

Test 8: Valid condition propagation

    1. Add a valid condition
      • Check: Condition is properly set for both breakpoints

Test 9: Condition/Ignore count removal propagation

    1. Remove condition and ignore count
      • Check: Condition and ignore count are removed from both breakpoints

Test 10: State persistence

    1. Add a valid condition, ignore count
      • Check: Condition/Ignore count are properly set for both breakpoints
    2. Remove thread filter
      • Check: The breakpoints for threads 1, 2 are removed
      • Check: A new breakpoint is set with the correct condition, ignore count and no thread filter

Test 11: Simultaneous update of state and thread filter

    1. Add filter for threads 1, 2, a valid condition and an ignore count
      • Check: The original breakpoint is removed
      • Check: 2 new breakpoints are set with proper ignore count, condition
    2. Remove thread filter
      • Check: The breakpoints for threads 1, 2 are removed
      • Check: A new breakpoint is set with the correct condition, ignore count and no thread filter

Test 12: Addition of thread filter, invalid condition and ignore count

    1. Add filter for threads 1, 2, invalid condition, ignore count
      • Check: The original breakpoint is removed
      • Check: 2 new breakpoints are set with proper ignore count and old (valid) condition
    2. Remove thread filter
      • Check: The breakpoints for threads 1, 2 are removed
      • Check: A new breakpoint is set with the correct condition, ignore count and no thread filter

Test 13: Enable, disable breakpoints

    1. Disable the breakpoint
      • Check: The breakpoint is disabled
    2. Add filter for threads 1, 2
      • Check: The original breakpoint is removed
      • Check: 2 new disabled breakpoints are set with proper ignore count, condition
    3. Enable the breakpoint
      • Check: Both back-end breakpoints are enabled
    4. Remove thread filter
      • Check: The breakpoints for threads 1, 2 are removed
      • Check: A new (enabled) breakpoint is set with the correct condition, ignore count and no thread filter

Back to the top