Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Batch Mode (Running Experiments)"

 
(14 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
[[Image:STEM TOP BAR.gif]]
 
[[Image:STEM TOP BAR.gif]]
  
'''Overview'''
+
'''[http://wiki.eclipse.org/Tutorials_for_Developers Back]'''
The purpose of Experiments is to allow running
+
  
 +
==''Overview''==
  
An experiment is a collection of modifiers with a link to a scenario.  
+
The purpose of Experiments is to allow running a set of scenarios, one after another, each modifying some aspect of a "base" scenario. As an
An experiment can run just like a scenario can run. When you run an experiment it creates a batch.
+
example, suppose you do not know the transmission rate for a disease. One can set up an experiment that varies the transmission rate through
A batch is an executable that spawns off (a sequence of) simulations.
+
a range of values and the result for each run can be analyzed (via log files).
The experiment uses modifiers to make changes to the scenario and then run a simulation from each.
+
+
  
'''New View: Active Batches (Analogous to Active Simulations)'''
+
STEM uses components called "Modifiers" to modify the base scenario from one run to the next. A modifier is attached to another component
 +
in your scenario. for instance a disease model or a infector. The modifier is responsible for changing one or more attributes of the attached
 +
component between runs.
  
1) Create your scenario.
 
  
2) In the editor you can create modifiers for models, decorators (a disease model), or graphs.
+
An experiment is a collection of modifiers with a link to a scenario, and an experiment can run just like a scenario can run. When you run an experiment it creates a batch
 +
that will spawn off (a sequence of) simulations.
 +
 
 +
 
 +
==''Creating an Experiment''==
 +
 
 +
1) Create a scenario (follow instructions here).
 +
 
 +
2) Create one or more modifiers. Modifiers can be created for for models, disease models, infectors, inoculators or graphs.
 
To create a modifier for a disease, for example, find your disease in the STEM Project Explorer. Select it, right click on your disease, and chose create a modifier as shown in Figure 1.  
 
To create a modifier for a disease, for example, find your disease in the STEM Project Explorer. Select it, right click on your disease, and chose create a modifier as shown in Figure 1.  
  
  
[[Image:Exper1a.jpg|300px]]
+
[[Image:Exper1a.png]]
[[Image:Exper1b.jpg|300px]]
+
[[Image:Exper1b.png]]
  
'''Figure 1a,b:''' Creating a new modifier for a particular disease in an existing scenario
+
'''Figure 1a,b:''' Creating a new modifier for a particular disease in an existing scenario (1a) and the wizard used to specify the features modified by the modifier (1b)
  
  
  
This will launch a new dialog to help you create a modifier that can be added to an experiment (see Figure 2). In the new modifier dialog, for example, under Transmission rate select "range"
+
This will launch a new dialog to help you create a modifier that can be added to an experiment (see Figure 1b). In the new modifier dialog are different [[Modifier Options]] available , for example, under Transmission rate select "range" and specify the range as shown.
(sequence not yet enabled) and specify the range as shown in Figure 2.
+
  
  
 +
3) On top, select "New Experiment" from the toolbar to launch the New Experiment wizard (Figure 2). Specify a name for your experiment and click Finish.
  
3) On top, select "Experiment Model" to launch the experiment wizard (Figure 2).
+
4) Open your new Experiment in the editor and drag the Scenario you created into the experiment.
Drag the scenario into the experiment.
+
Drag any number of modifiers into the experiment.
+
  
Click on the experiment, right click, and run it.
+
5) Drag the modifier you created into your new Experiment. You can drag any number of modifiers into the experiment.
 +
 
 +
5) Save your experiment by hitting ctrl+s
 +
 
 +
6) To run the experiment, right click on it and select run. It's a good idea to enable logging before you start an experiment, or the results will be lost.
 +
 
 +
[[Image:Exper2.png]]
  
[[Image:Exper2.jpg|400px]]
 
 
'''Figure 2:''' The New Experiment Wizard.
 
'''Figure 2:''' The New Experiment Wizard.
  
 +
==''Modifier Options''==
 +
 +
There are four different options to change a value in a modifier:
 +
 +
* None:    Is the default option, the value of the parameter is not changed.
 +
 +
* Single:  These option gives the possibility to change a specific Parameter to an another Value.
 +
 +
* Range:    Set a range of different values between a minimum and a maximum value by a configurable step width.
 +
 +
* Sequence: These option gives the possibility to add a list of different values to a Parameter after finish the Modifier configuration.Look at the [[Sequence Tutorial]].   
 +
 +
==''Sequence Tutorial''==
 +
 +
<ol>
 +
<li>  Create the Modifier( follow the instructions 1,2 in '''Creating an Experiment''')</li>
 +
<li> Choose the Modifier Type: "Sequence"</li>
 +
<li> Click "Next" and "Finish"</li>
 +
<li> Choose the "Modifier" in the "Project Explorer" </li>
 +
<li> Expand the Modifier, all Parameters with Sequence are listed  </li>
 +
<li> Choose via the right click on a Parameter the "Shows Properties View" option  </li>
 +
<li> Choose in the Propertie Window under the Project the row "sequence" and click on the (...) Button in the right corner. </li>
 +
<li> A menu pops up and you can add new values.</li>
 +
<li> Save and Close the menu.</li>
 +
<li> There is a small Bug in the current Experiment. If you want to run the scenarios, it doesn't start because the "Current Sequence Index" writes the actual count of Numbers in the XML-File.</li>
 +
<li> To fix this you can open the directory of your STEM-folder -> Workspace -> Your current STEM Project -> modifiers and open the .MODIFIER File with a Text Editor.</li>
 +
<li> The .modifier files are in XML. There can you configure the "Current Sequence Index", set them back to 0 and save the file. Please change only the value="currentSequenceIndex".<br>
 +
''Example for the changed XML tag:''
 +
<code style="font-size: 10pt;">    <featureModifiers xsi:type="org.eclipse.stem.core.modifier:DoubleSequenceModifier" featureName="beta_Paramter" featureId="23" currentSequenceIndex="0"> </code>
 +
</li>
 +
<li> Refresh the Project in STEM and run the Experiment</li>
 +
</ol>
 +
 +
----
 +
 +
Note:
 +
It is possible to put a bulk of sequence Values into the Project, e.g. you want to run 100 different values for one Parameter.
 +
Currently there is no file Upload in the menu, so you have to put the data directly into the Modifier file with the XML Format.
 +
Please note every value is one scenario so it needs a lot of time, if you have many sequences.   
 +
You can add the different values with: <code style="font-size: 10pt;">    <sequence>1</sequence></code>
 +
 +
----
  
 +
Important clarification: As mentioned, an experiment can contain any number of modifiers. When the experiment runs, all possible combinations of parameter values defined by the modifiers will be enumerated
 +
and a scenario will be created and run for each of the combinations. If you look in Figure 1b however, it is also possible to set up a Modifier to modify more than one feature of a disease model. The distinction here is that when a modifier
 +
affects more than one feature, the sequence of values for those features will be kept in "locked steps" with each other. For instance, if a modifier steps through two values [1.2, 1.5] for transmission rate and two values [0.2, 0.5] for infectious recovery
 +
rate, only two scenarios will be created and run. One with transmission rate 1.2 and infectious recovery rate 0.2, and the other with transmission rate 1.5 and infectious recovery rate 0.5. On the other hand, if you create two modifiers, one for transmission rate and the other for infectious recovery rate and drag into your experiment, all four possible combinations of values will be evaluated.
  
''Save your work at this point by hitting ctrl-s''
 
  
  
You run an experiment the same way you run a scenario. In the STEM Project Explorer click on the experiment, right click, and select run.
+
==''Setting the Stochastic Solver seed''==
 +
Please note that the stochastic solver seed is not modifiable but it can be configured to automatically run with a new seed on each run. please see [[STEM_Solvers#Running_Stochastic_Solvers_in_Batch_Mode]] for more information

Latest revision as of 16:31, 31 March 2018

STEM TOP BAR.gif

Back

Overview

The purpose of Experiments is to allow running a set of scenarios, one after another, each modifying some aspect of a "base" scenario. As an example, suppose you do not know the transmission rate for a disease. One can set up an experiment that varies the transmission rate through a range of values and the result for each run can be analyzed (via log files).

STEM uses components called "Modifiers" to modify the base scenario from one run to the next. A modifier is attached to another component in your scenario. for instance a disease model or a infector. The modifier is responsible for changing one or more attributes of the attached component between runs.


An experiment is a collection of modifiers with a link to a scenario, and an experiment can run just like a scenario can run. When you run an experiment it creates a batch that will spawn off (a sequence of) simulations.


Creating an Experiment

1) Create a scenario (follow instructions here).

2) Create one or more modifiers. Modifiers can be created for for models, disease models, infectors, inoculators or graphs. To create a modifier for a disease, for example, find your disease in the STEM Project Explorer. Select it, right click on your disease, and chose create a modifier as shown in Figure 1.


Exper1a.png Exper1b.png

Figure 1a,b: Creating a new modifier for a particular disease in an existing scenario (1a) and the wizard used to specify the features modified by the modifier (1b)


This will launch a new dialog to help you create a modifier that can be added to an experiment (see Figure 1b). In the new modifier dialog are different Modifier Options available , for example, under Transmission rate select "range" and specify the range as shown.


3) On top, select "New Experiment" from the toolbar to launch the New Experiment wizard (Figure 2). Specify a name for your experiment and click Finish.

4) Open your new Experiment in the editor and drag the Scenario you created into the experiment.

5) Drag the modifier you created into your new Experiment. You can drag any number of modifiers into the experiment.

5) Save your experiment by hitting ctrl+s

6) To run the experiment, right click on it and select run. It's a good idea to enable logging before you start an experiment, or the results will be lost.

Exper2.png

Figure 2: The New Experiment Wizard.

Modifier Options

There are four different options to change a value in a modifier:

  • None: Is the default option, the value of the parameter is not changed.
  • Single: These option gives the possibility to change a specific Parameter to an another Value.
  • Range: Set a range of different values between a minimum and a maximum value by a configurable step width.
  • Sequence: These option gives the possibility to add a list of different values to a Parameter after finish the Modifier configuration.Look at the Sequence Tutorial.

Sequence Tutorial

  1. Create the Modifier( follow the instructions 1,2 in Creating an Experiment)
  2. Choose the Modifier Type: "Sequence"
  3. Click "Next" and "Finish"
  4. Choose the "Modifier" in the "Project Explorer"
  5. Expand the Modifier, all Parameters with Sequence are listed
  6. Choose via the right click on a Parameter the "Shows Properties View" option
  7. Choose in the Propertie Window under the Project the row "sequence" and click on the (...) Button in the right corner.
  8. A menu pops up and you can add new values.
  9. Save and Close the menu.
  10. There is a small Bug in the current Experiment. If you want to run the scenarios, it doesn't start because the "Current Sequence Index" writes the actual count of Numbers in the XML-File.
  11. To fix this you can open the directory of your STEM-folder -> Workspace -> Your current STEM Project -> modifiers and open the .MODIFIER File with a Text Editor.
  12. The .modifier files are in XML. There can you configure the "Current Sequence Index", set them back to 0 and save the file. Please change only the value="currentSequenceIndex".
    Example for the changed XML tag: <featureModifiers xsi:type="org.eclipse.stem.core.modifier:DoubleSequenceModifier" featureName="beta_Paramter" featureId="23" currentSequenceIndex="0">
  13. Refresh the Project in STEM and run the Experiment

Note: It is possible to put a bulk of sequence Values into the Project, e.g. you want to run 100 different values for one Parameter. Currently there is no file Upload in the menu, so you have to put the data directly into the Modifier file with the XML Format. Please note every value is one scenario so it needs a lot of time, if you have many sequences. You can add the different values with: <sequence>1</sequence>


Important clarification: As mentioned, an experiment can contain any number of modifiers. When the experiment runs, all possible combinations of parameter values defined by the modifiers will be enumerated and a scenario will be created and run for each of the combinations. If you look in Figure 1b however, it is also possible to set up a Modifier to modify more than one feature of a disease model. The distinction here is that when a modifier affects more than one feature, the sequence of values for those features will be kept in "locked steps" with each other. For instance, if a modifier steps through two values [1.2, 1.5] for transmission rate and two values [0.2, 0.5] for infectious recovery rate, only two scenarios will be created and run. One with transmission rate 1.2 and infectious recovery rate 0.2, and the other with transmission rate 1.5 and infectious recovery rate 0.5. On the other hand, if you create two modifiers, one for transmission rate and the other for infectious recovery rate and drag into your experiment, all four possible combinations of values will be evaluated.


Setting the Stochastic Solver seed

Please note that the stochastic solver seed is not modifiable but it can be configured to automatically run with a new seed on each run. please see STEM_Solvers#Running_Stochastic_Solvers_in_Batch_Mode for more information

Back to the top