Batch Mode (Running Experiments)

Back

Overview

The purpose of Experiments is to allow running a set of scenarios, one after another, each modifying some aspect of a "base" scenario. As an example, suppose you do not know the transmission rate for a disease. One can set up an experiment that varies the transmission rate through a range of values and the result for each run can be analyzed (via log files).

STEM uses components called "Modifiers" to modify the base scenario from one run to the next. A modifier is attached to another component in your scenario. for instance a disease model or a infector. The modifier is responsible for changing one or more attributes of the attached component between runs.

An experiment is a collection of modifiers with a link to a scenario, and an experiment can run just like a scenario can run. When you run an experiment it creates a batch that will spawn off (a sequence of) simulations.

Creating an Experiment

1) Create a scenario (follow instructions here).

2) Create one or more modifiers. Modifiers can be created for for models, disease models, infectors, inoculators or graphs. To create a modifier for a disease, for example, find your disease in the STEM Project Explorer. Select it, right click on your disease, and chose create a modifier as shown in Figure 1.

Figure 1a,b: Creating a new modifier for a particular disease in an existing scenario (1a) and the wizard used to specify the features modified by the modifier (1b)

This will launch a new dialog to help you create a modifier that can be added to an experiment (see Figure 1b). In the new modifier dialog are different Modifier Options available , for example, under Transmission rate select "range" and specify the range as shown.

3) On top, select "New Experiment" from the toolbar to launch the New Experiment wizard (Figure 2). Specify a name for your experiment and click Finish.

4) Open your new Experiment in the editor and drag the Scenario you created into the experiment.

5) Drag the modifier you created into your new Experiment. You can drag any number of modifiers into the experiment.

5) Save your experiment by hitting ctrl+s

6) To run the experiment, right click on it and select run. It's a good idea to enable logging before you start an experiment, or the results will be lost.

Figure 2: The New Experiment Wizard.

Modifier Options

There are four different options to change a value in a modifier:

• None: Is the default option, the value of the parameter is not changed.

• Single: These option gives the possibility to change a specific Parameter to an another Value.

• Range: Set a range of different values between a minimum and a maximum value by a configurable step width.

• Sequence: These option gives the possibility to add a list of different values to a Parameter after finish the Modifier configuration.Look at the Sequence Tutorial.

Sequence Tutorial

1. Create the Modifier( follow the instructions 1,2 in Creating an Experiment)
2. Choose the Modifier Type: "Sequence"
3. Click "Next" and "Finish"
4. Choose the "Modifier" in the "Project Explorer"
5. Expand the Modifier, all Parameters with Sequence are listed
6. Choose via the right click on a Parameter the "Shows Properties View" option
7. Choose in the Propertie Window under the Project the row "sequence" and click on the (...) Button in the right corner.
8. A menu pops up and you can add new values.
9. Save and Close the menu.
10. There is a small Bug in the current Experiment. If you want to run the scenarios, it doesn't start because the "Current Sequence Index" writes the actual count of Numbers in the XML-File.
11. To fix this you can open the directory of your STEM-folder -> Workspace -> Your current STEM Project -> modifiers and open the .MODIFIER File with a Text Editor.
12. The .modifier files are in XML. There can you configure the "Current Sequence Index", set them back to 0 and save the file. Please change only the value="currentSequenceIndex".
    Example for the changed XML tag: <featureModifiers xsi:type="org.eclipse.stem.core.modifier:DoubleSequenceModifier" featureName="beta_Paramter" featureId="23" currentSequenceIndex="0">
13. Refresh the Project in STEM and run the Experiment

---

Note: It is possible to put a bulk of sequence Values into the Project, e.g. you want to run 100 different values for one Parameter. Currently there is no file Upload in the menu, so you have to put the data directly into the Modifier file with the XML Format. Please note every value is one scenario so it needs a lot of time, if you have many sequences. You can add the different values with: <sequence>1</sequence>

---

Important clarification: As mentioned, an experiment can contain any number of modifiers. When the experiment runs, all possible combinations of parameter values defined by the modifiers will be enumerated and a scenario will be created and run for each of the combinations. If you look in Figure 1b however, it is also possible to set up a Modifier to modify more than one feature of a disease model. The distinction here is that when a modifier affects more than one feature, the sequence of values for those features will be kept in "locked steps" with each other. For instance, if a modifier steps through two values [1.2, 1.5] for transmission rate and two values [0.2, 0.5] for infectious recovery rate, only two scenarios will be created and run. One with transmission rate 1.2 and infectious recovery rate 0.2, and the other with transmission rate 1.5 and infectious recovery rate 0.5. On the other hand, if you create two modifiers, one for transmission rate and the other for infectious recovery rate and drag into your experiment, all four possible combinations of values will be evaluated.

Setting the Stochastic Solver seed

Please note that the stochastic solver seed is not modifiable but it can be configured to automatically run with a new seed on each run. please see STEM_Solvers#Running_Stochastic_Solvers_in_Batch_Mode for more information