Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Welcome STEM Developers"

(Books)
(Introduction)
 
(176 intermediate revisions by 10 users not shown)
Line 1: Line 1:
 +
[[Image:STEM TOP BAR.gif|800px]]
 +
{| align="right"
 +
  | __TOC__
 +
  |}
 +
 +
'''[http://wiki.eclipse.org/index.php/STEM STEM Contents Page]'''
 +
 
=Introduction=
 
=Introduction=
 
This article is intended to be the starting point for developers working on the '''Spatio-Temporal Epidemiological Modeler Project (STEM)''' code base.   
 
This article is intended to be the starting point for developers working on the '''Spatio-Temporal Epidemiological Modeler Project (STEM)''' code base.   
Line 8: Line 15:
 
document  contains or by clarifying any descriptions.  As a  newcomer
 
document  contains or by clarifying any descriptions.  As a  newcomer
 
to the project you have a totally unique and extremely valuable
 
to the project you have a totally unique and extremely valuable
perspective, if you can't find something or it doesn't make sense to
+
perspective. If you can't find something or it doesn't make sense to
 
you then you're probably not alone.  When you find out the answer to
 
you then you're probably not alone.  When you find out the answer to
 
your quandary, please record it here so those that come after you can
 
your quandary, please record it here so those that come after you can
“stand on your shoulders”.  Welcome aboard!
+
“stand on your shoulders. Welcome aboard!
  
 
The STEM code base is written in Java<FONT FACE="Times New Roman, serif">™</FONT>
 
The STEM code base is written in Java<FONT FACE="Times New Roman, serif">™</FONT>
Version 5 and is organized as a set of well defined components using
+
Version 6 and is organized as a set of well defined components using
 
the eclipse (<FONT COLOR="#000080"><U><SPAN STYLE="background: transparent">[http://www.eclipse.org/ www.eclipse.org]</SPAN></U></FONT>)
 
the eclipse (<FONT COLOR="#000080"><U><SPAN STYLE="background: transparent">[http://www.eclipse.org/ www.eclipse.org]</SPAN></U></FONT>)
 
plug-in tool framework. The components are integrated through a well
 
plug-in tool framework. The components are integrated through a well
 
defined “extension point” mechanism that makes the entire code
 
defined “extension point” mechanism that makes the entire code
base highly extensible. The use of the eclipse framework also
+
base highly extensible. The use of the Eclipse framework also
 
provides for multi-platform portability.
 
provides for multi-platform portability.
<P>Interestingly, eclipse is also the STEM project's Java<FONT FACE="Times New Roman, serif">™</FONT>
+
<P>Interestingly, Eclipse is also the STEM project's Java<FONT FACE="Times New Roman, serif">™</FONT>
 
development environment.  If this seems a bit strange at first, don't
 
development environment.  If this seems a bit strange at first, don't
worry, as you understand more about eclipse and the project this will
+
worry. As you understand more about eclipse and the project this will
 
make more sense.
 
make more sense.
 
The project is organized and managed as an open source project.  
 
The project is organized and managed as an open source project.  
Line 29: Line 36:
 
Karl Fogel (see the references later in this document).</SPAN>
 
Karl Fogel (see the references later in this document).</SPAN>
  
===Step-by-step for OHF STEM developers===
+
=== Step-by-step for&nbsp;STEM developers ===
 
+
#Read this article.
+
#When you find missing, confusing, erroneous information please contribute, by fixing the problem. This Wiki allows any registered user to update content. Please do!
+
#Visit the [http://www.eclipse.org/ohf/ OHF project page].  Bookmark it.
+
#Subscribe to the project [http://dev.eclipse.org/mailman/listinfo/ohf-dev mailing list]. The mailing list is used for all internal developer communication.  It is preferred to send private Emails.
+
#Install Java 5.0 and Eclipse 3.2]
+
#Obtain the source code for the STEM project from the CVS repository.
+
 
+
=STEM Installation Instructions=
+
==Installing  Java™ 5 JVM==
+
STEM requires Java Version 1.5.
+
IBMers should obtain the IBM version of Java 1.5.  Others can use either IBM's Java 1.5 or the Sun Java 1.5 [[http://java.sun.com/j2se/1.5.0/download.jsp]].
+
 
+
For the IBM Java:
+
*Register with [http://w3.hursley.ibm.com/java/jim/ JIM], IBM's “Java™ Information Manager”
+
When you register you need to specify why you need access to Java™.
+
*When you receive authorization in the form of an email download the file (e.g., <FONT FACE="Courier New, monospace">ibm-java2-sdk-50-win-i386</FONT>)
+
*Run the installation program by double clicking on the file.
+
 
+
==Installing the Eclipse Environment==
+
 
+
Goto [[STEM Eclipse Setup]]
+
 
+
==Installing/Running the STEM application==
+
 
+
'''Running Stem as a standalone application'''
+
If you want to first tryout STEM as an application then we provide a standalone version of STEM that only requires that you have Java 1.5 or higher installed.
+
 
+
You can get the file by going to the
+
[http://www.eclipse.org/ohf/stem project page]:
+
and then clicking on "Files".  Download stem0.1.0.zip (it is a very large file) and unzip it to your favorite directory.
+
 
+
This is a self-contained stand-alone version of STEM that is launched by double-clicking on the "STEM.exe" executable file in the root directory of the expanded archive.  You'll need to have Java1.5 installed for it to run, but if you do and it's on the path,  you should be greeted by the STEM "splash screen" while it starts up. 
+
  
*Go to Windows->Preferences->STEM->Simulation Management and enable "Pause simulation after each cycle" and set the seconds to pause to 2.
+
*Read this article.
*Open the Simulation perspective.
+
*When you find missing, confusing, erroneous information, please contribute by fixing the problem. This Wiki allows any Bugzilla registered user to update content. Please do!
*Open the Map view with Window->Map to see the spread of the disease.  
+
*Visit the&nbsp;[http://www.eclipse.org/stem/ STEM project page] and bookmark it.  
*Select the Spanish Flu Scenario under STEM Internal Data Test in the Scenario's view
+
*Obtain an [https://dev.eclipse.org/site_login/createaccount.php Eclipse.org account] to access bugzilla, the newsgroup, etc.
*Select Stem->Run from the menubar or toolbar.
+
*Subscribe to the stem-dev [http://dev.eclipse.org/mailman/listinfo/stem-dev mailing list]. The mailing list is used for all internal developer communication.  
 +
*Visit the [http://www.eclipse.org/forums/index.php?t=thread&frm_id=72 Newsgroup]
 +
*Install Java 8.0 and Eclipse 4.x [http://wiki.eclipse.org/index.php/Installation_Guide#STEM_Installation_Guide]
 +
*Obtain the source code for the STEM project from the code repository. [http://wiki.eclipse.org/index.php/STEM_Source_Code]
  
The Scenario only has the USA and Mexico at this time.
+
You can also review the article on '''Eclipse Development Resources''' http://wiki.eclipse.org/index.php/Development_Resources
  
To run the Stem-GoogleEarth view:
+
= Software Engineering =
*You must have installed GoogleEarth V4
+
*Select the GoogleEarth View. Window->Other->Stem->GoogleEarth View
+
*Select Window-Preferences->Stem->GoogleEarth and set:
+
**Check Method -> Direct Launch
+
**Check "Automatically process every simulation"
+
*Select the Spanish Flu Scenario under STEM Internal Data Test in the Scenario's view
+
*Select Stem->Run from the menubar or toolbar.
+
  
<H1 CLASS="western">Software Engineering</H1>
 
 
==Coding Conventions and Guidelines==
 
==Coding Conventions and Guidelines==
 
As with any product being built by a team, there are various areas where standards, conventions, and other guidelines can play a role in helping to ensure that the resulting product presents to developers and customers as a unified whole rather than as a loose collection of parts worked on by a variety of individuals each with their own styles and ways of working.
 
As with any product being built by a team, there are various areas where standards, conventions, and other guidelines can play a role in helping to ensure that the resulting product presents to developers and customers as a unified whole rather than as a loose collection of parts worked on by a variety of individuals each with their own styles and ways of working.
Line 88: Line 57:
 
See the [http://wiki.eclipse.org/index.php/Development_Conventions_and_Guidelines Eclipse Development Conventions and Guidelines].
 
See the [http://wiki.eclipse.org/index.php/Development_Conventions_and_Guidelines Eclipse Development Conventions and Guidelines].
  
The most important guidelines for STEM code are that it have good JavaDoc documentation and not generate compiler warnings.  
+
The most important guidelines for STEM code are to have good JavaDoc documentation and not generate compiler warnings.  
  
 
The Eclipse compiler verifies the code against a preference list and will generate warning messages for code that does not follow rules specified in the compiler preferences.  For example, it will check for "unchecked Generic type operations" and give a warning.   
 
The Eclipse compiler verifies the code against a preference list and will generate warning messages for code that does not follow rules specified in the compiler preferences.  For example, it will check for "unchecked Generic type operations" and give a warning.   
We ask you removes all of the Eclipse compiler warnings before submitting the code.  Also preferences for the following JavaDoc options should be changed to generate warnings and the warnings removed by fixing the code and/or creating accurate and informative Javadoc comments,
+
We ask you remove all of the Eclipse compiler warnings before submitting the code.  Also preferences for the following JavaDoc options should be changed to generate warnings and the warnings removed by fixing the code and/or creating accurate and informative Javadoc comments,
  
 
<code>
 
<code>
Line 101: Line 70:
 
</code>
 
</code>
  
If you are the owner or creater of a STEM subproject you can create more restrictive standards by checking in a set of project preferences.  However, please do not remove warnings by checking in project preferences that ignore bad coding practices.
+
If you are the owner or creator of a STEM subproject, you can create more restrictive standards by checking in a set of project preferences.  However, please do not remove warnings by checking in project preferences that ignore bad coding practices.
  
  
  
'''Copyright Statement'''
+
=== Copyright Statement ===
  
Each source file should include the following copyright statement.  It should follow the '''package''' statement and proceed the '''import''' statements.
+
A copyright should be on any human created text document including HTML,Java, properties, XML, XSD. Auto generated/machine made text files does not need to have a copyright. Such files are compiled from a human created file and it is not practical to add copyright to them (as we wouldn't add copyright to class files). To be specific, we are referring to auto generated JavaDocs (HTML) and EMF models (JAVA). For .java files the copyright statement should follow the '''package''' statement and proceed the '''import''' statements. The Copyright statement is further described [http://www.eclipse.org/legal/copyrightandlicensenotice.php here]
  
 +
<br><code></code>
  
<code>
+
  package org.eclipse.stem;  
  package org.eclipse.ohf.stem;
+
  /*************************************************************************
 
+
  * Copyright (c) 2006,2007 IBM Corporation and others.
  /*******************************************************************************
+
  * Copyright (c) 2006 IBM Corporation and others.
+
 
  * All rights reserved. This program and the accompanying materials
 
  * All rights reserved. This program and the accompanying materials
 
  * are made available under the terms of the Eclipse Public License v1.0
 
  * are made available under the terms of the Eclipse Public License v1.0
Line 122: Line 90:
 
  * Contributors:
 
  * Contributors:
 
  *    IBM Corporation - initial API and implementation
 
  *    IBM Corporation - initial API and implementation
  *******************************************************************************/  
+
  *************************************************************************/  
 +
import ...&nbsp;;
  
import ... ;
 
</code>
 
  
  
'''National Language Support'''
+
To have the above automatically inserted in new code, do the following:
  
STEM supports languages other than English.  The basic process is to provide properly named properties files that mirror the "native" English properties files.  These files are grouped into a "plug-in fragment" which acts as an "add on" to a plug-in and adds the files of the fragment to the plug-in as if they were there originally.  We separate them so that additional languages can be added without changing the original plug-in.  
+
*Select the above comment (from /* to */)
 +
*Select '''Window-&gt;preferences'''
 +
*Select '''Java-&gt;Code Style-&gt;CodeTemplates-&gt;Types'''
 +
*Select '''edit'''
 +
*Paste the copyright statement in place of the existing "Author" statement.  
 +
*Select '''OK'''
  
If a native file is named "messages.properties", then the corresponding file with Spanish translations for the messages would be named "messages_es.properties".  It is also possible to provide translations that are specific to a particular country, for instance, for Canadian English, the corresponding file would be "messages_en_CA.properties".  For Californian English it would be "messages_en_US_CA.properties".  This follows the regular Java conventions.
+
===National Language Support===
  
To get us started, I've created one new plug-in fragment called "org.eclipse.ohf.stem.ui.nl1" which contains the properties files for the main UI component of STEM in the plug-in "org.ecliopse.ohf.stem.ui".  Each of the other plug-ins with translatable strings will require their own (yet to be created) plug-in fragment.  I also created properly named, untranslated, properties files for Californian English, Canadian English, Spanish, Hebrew, Tamil and Chinese in the new fragment.
+
: '' See page on [[STEM_National_Language_Support | STEM National Language Support]] ''
  
To start STEM (Eclipse) in a language different from the working language of the operating system, you need to use the "-nl" command line parameter.  For example, use "-nl en_US_CA" to start STEM in Californian English, or "-nl es" to start it in Spanish.  You might find it useful to create a new launch configuration in Eclipse with the parameter specified.  I have one for each language.
+
== Bug Reporting ==
  
1) Use the "Run..." menu item to open the "Create, manage, and run configurations" dialogue.
+
Eclipse uses the Bugzilla system for reporting and processing Bug reports and enhancement request for the&nbsp;STEM project. Once a problem or enhancement has been submitted to Bugzilla, the Bugzilla entry should be used for subsequent discussion of the issue.  
2) Duplicate the Eclipse application you use to launch STEM and rename the new one to reflect the language (e.g., "STEM Californian")
+
3) Select the "Arguments" tab
+
4) In the "Program arguments" text box enter the command line parameter (E.g., "-nl en_US_CA" without the quotes, for Californian English).  
+
5) Apply
+
6) Run
+
  
Open the Active Simulations View and you should see a different title than normal.  
+
*The following page is used for creation of a Bugzilla account. [https://bugs.eclipse.org/bugs/createaccount.cgi]
 +
*The following page is used to set the user email preferences. [https://bugs.eclipse.org/bugs/userprefs.cgi]
 +
*The bug reporting page is: [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=OHF] Specify STEM as the '''component'''
 +
*The following page can be used to find bugs in&nbsp;STEM https://bugs.eclipse.org/bugs/query.cgi
 +
**Enter ''Technology'' as the '''Classification'''
 +
**Enter ''STEM'' as the '''Product'''
  
 +
The following article shows the life cycle of a bug from entering the system to being '''closed'''. http://www.eclipse.org/projects/dev_process/bugzilla-use.php
  
Here are some good resources for more detailed information:  
+
Before reporting a bug, please read the Eclipse [https://bugs.eclipse.org/bugs/page.cgi?id=bug-writing.html bug writing guidelines] and please search to see if the bug has already been reported.
  
http://www.eclipse.org/articles/Article-Internationalization/how2I18n.html
+
There are some important considerations when you submit a Bugzilla report. If it is a problem that you are reporting, then the promptness of a resolution is very much dependent on the completeness of the report. In most cases, a problem needs to be reproduced in order to fix it. Before you press '''submit''' read your report over and think about whether if you were the STEM developer that gets the problem, would you have the information you need to reproduce the problem.  
  
http://www.eclipse.org/tptp/home/documents/process/development/translation_rules_of_thumb.html
+
An important help in problem determination is the '''Error Log''' view on the STEM window. If any exceptions or serious errors occurred they will show up there. These entries can be exported to a file and attached to the bugzilla entry.  
  
"Building Commercial Quality Plug-ins", Clayberg,  
+
Most of the development of STEM was done on Windows with Eclipse 3.8 or above. If you are using a different platform, be sure to mention this just in case this is a platform problem.
  
"The Java Developer's Guide to ECLIPSE", D'Anjou, et al.
+
If you are submitting a suggestion for a new feature or enhancement, include some justification and enough details that it can be evaluated. Since others may want to contribute to a discussion of the issue, it would be a good idea to post a pointer to the Bugzilla entry on the STEM newsgroup.
  
<H2 CLASS="western">Source Code Control</H2>
+
== STEM Mailing List ==
<P>All of the files that constitute the STEM project are maintained
+
in a “source code control” system. Anyone can obtain a copy of
+
the files in the system, but only people designated as “contributors”
+
are allowed to add new files or make other changes to the repository.
+
The only files that are in the repository are those that cannot be
+
derived from others.  For instance, Java<FONT FACE="Times New Roman, serif">™</FONT>
+
source files would be in the repository, but not Java<FONT FACE="Times New Roman, serif">™</FONT>
+
class files. 
+
</P>
+
== Plans to move STEM to Open Source repository ==
+
  
Soon the STEM repository will be moved to the official Eclipse CVS repository and
+
Send stem-dev mailing list submissions to&nbsp;'''&lt;stem-dev@eclipse.org&gt;'''
the following Subversion information will no longer apply.
+
  
Anyone will be able to checkout the STEM source code and data from the CVS repository as described later.
+
To subscribe or unsubscribe via the World Wide Web, visit
  
To submit to the Eclipse CVS repository you will need a [https://bugs.eclipse.org/bugs/userprefs.cgi Bugzilla account] and be added to the list of committers.
+
:[https://dev.eclipse.org/mailman/listinfo/dev https://dev.eclipse.org/mailman/listinfo/stem-dev]
  
== Subversion Source Code Control System ==
+
or, via email, send a message with subject of body 'help' to
  
<P>The project uses the  [http://subversion.tigris.org/ Subversion] source code control system.  This is an open source revision
+
&nbsp;&nbsp;&nbsp;&nbsp; [mailto:stem-dev-request@eclipse.org stem-dev-request@eclipse.org]
control system that is designed to be the successor to CVS.
+
Subversion was conceived and implemented by developers who are
+
intimately familiar with CVS and so was designed to be a superior
+
replacement.  It includes a number of features missing from CVS
+
including atomic operations, full version support for folders and
+
better performance (this is an incomplete list). Most new open source
+
projects are selecting Subversion to manage their source code
+
repositories. 
+
</P>
+
<P>While Subversion performs a similar function as CVS, it differs
+
from the later in a very fundamental way.  In CVS, each file has a
+
version or revision number which changes as it is changed and
+
checked-in to the repository.  In Subversion, there is only one
+
global revision number for the entire repository and each time any
+
change (a “commit”) is made to the repository, this number is
+
increment.  Subversion is, in essence, a “versioning file system”.
+
The concept is simple, any state of the file system since it came
+
into existence can be recreated, all one needs is the “revision
+
number”of the desired state and it can be “checked out” from
+
the system.  For instance, the state of the file system after the
+
three hundred and thirty fourth change is represented by the revision
+
number “r334” (the prefix “r” is required).  Understanding
+
this, it should be clear that from one revision to another, a file
+
can be exactly the same even though “its” revision number is
+
different.  The key point to remember is that the revision number is
+
not for the file, but for the repository.
+
</P>
+
<P>For example, a newly created repository is at revision “r0”.
+
If we add file “foo.java” and commit it to the repository, the
+
revision number of the repository would become “r1”.  If we add
+
another file “bar.java” and commit it, the revision number of the
+
repository will increase to “r2”.
+
</P>
+
<P>Subversion also differs from CVS and other repository systems in
+
the way it handles “branches” and “tags” of the repository. A
+
branch is an alternative version of the repository usually used to do
+
parallel development without affecting the main development path.  A
+
tag is basically a label attached to a version of the repository
+
used to name some significant state such as a release.  In CVS these
+
concepts are specifically implemented and maintained by the system,
+
in Subversion these concepts have no specific support.  Instead they
+
are implemented by creating  new “sub-folders” in the repository
+
and then copying the appropriate file system tree in the repository
+
to the new sub-folder. On the surface this seems very strange and
+
inefficient. The trick is that Subversion is implemented in such a
+
way that copies are very fast and very efficient.  When a file is
+
copied in the repository nothing much changes except for a few
+
bookkeeping entries.
+
</P>
+
<P>By convention, but not enforced by the system, the root of a
+
Subversion source code repository contains three top-level folders
+
named “trunk”, “branches” and “tags”.  The trunk folder
+
holds the main copy of the code being developed.  The branches folder
+
contains sub-folders that define different branch copies of the
+
contents of trunk. The tags folder contains sub-folders that each
+
define different “tagged” copies of the contents of trunk.  The
+
names of the folders in both the branches and tags folders act as the
+
labels of the corresponding “branch” and “tag”. Development
+
occurs (i.e., files are committed) under trunk and branches, but not
+
tags.  The development efforts on the branches sub-folders should
+
have a life-cycle that causes their contents to be merged back into
+
the trunk (how to do that is another story) or abandoned, and then
+
ultimately, regardless of their fate, deleted from the current
+
revision of the repository.  The tags folder should never be changed.
+
It represents a labeled snapshot of the trunk.
+
</P>
+
<P>&lt;diagram of trunk/branches/tags here&gt;</P>
+
  
==Getting Access to the STEM Source Code Repository==
+
You can reach the person managing the list at
There are three ways to access the STEM source code repository,
+
through the Subclipse eclipse plug-in, through the tortoiseSVN
+
integrated windows shell tool, and through the command line using
+
Subversion commands.  Installation of Subclipse was described
+
earlier in this document and we give specific instructions on how to
+
use ''subclipse'' but you are free to use TortoiseSVN or the command line if you prefer.
+
  
==Accessing the STEM Source Code Repository with Subclipse==
+
&nbsp;&nbsp;&nbsp; [mailto:stem-dev-owner@eclipse.org stem-dev-owner@eclipse.org]  
These instructions assume that the Subclipse plug-in has been
+
installed in eclipse.
+
*Select Window->Open Perspective->Other->SVN Repository Exploring
+
*In the “SVN Repository” perspective right click on the view to bring up the context menu and select “New→Repository Location”
+
*In the dialog there will be a box labeled “Url:”, enter https://svn.opensource.ibm.com/svn/stem/trunk This is the URL of the source code repositories “trunk”.
+
*In the “SVN Repository” view you will now have a list of all of the current projects.  Select the ones you want (usually all) and from the "right click" select “Check Out...” This will get all of the projects into your eclipse workspace.
+
===Accessing the STEM Source Code Repository with tortoiseSVN===
+
TortoiseSVN is a Subversion client, implemented as a windows shell extension.
+
One would download and install TortoiseSVN from the Tigis.org [[http://tortoisesvn.tigris.org/]] web site.
+
  
===Accessing the STEM Source Code Repository with CVS===
+
When replying, please edit your Subject line so it is more more specific than&nbsp;"Re: Contents of stem-dev digest..."&nbsp;
  
''For use when STEM is on Eclipse.OHF''
+
<br>
  
At the current time, both users and developers of STEM must access the code and data from the CVS repository in order to use the system. However all elements of the Eclipse STEM project is available for anonymous read-only access to the development CVS repository. Using anonymous access you can checkout the code and data to your local system.  You can then use the system and if you wish modify it locally. However you can not use anonymous access to write it back to the repository. This is handy if you would like to fix a bug or add a feature. Get the code via anonymous access, do your work and then pass the work on to a committer (in a patch attached to a bug report) for inclusion in the repository. Committers love to have bugs fixed for them!
+
The&nbsp;developer's mailing list should be used to communicate with other STEM developers. It is preferred over private emails.
  
To use anonymous access from a running Eclipse platform, simply go to a Repository View and add a new CVS repository location. Use the following information to connect:
+
== STEM Newsgroup  ==
  
{| style="width:60%;" border="0" cellpadding="2"
+
The Newsgroup for STEM is '''news://news.eclipse.org/eclipse.technology'''  
|+Anonymous CVS Connection Information
+
|-
+
| '''Host''' || dev.eclipse.org  
+
|-
+
| '''Repository Paths''' || /cvsroot/technology  
+
|-
+
| '''User''' || anonymous
+
|-
+
| '''Password''' || leave blank
+
|-
+
| '''Connection Type''' || pserver
+
|-
+
|}
+
  
Expand the list from:
+
To access the newsgroup you will need to go to the following page and request a userid and password.  
  HEAD -> org.eclipse.ohf -> stem
+
*org.eclipse.ohf.stem.common
+
*org.eclipse.ohf.stem.core
+
*org.eclipse.ohf.stem. '''...'''
+
*org.eclipse.ohf.stem.utility
+
  
Select all of the stem projects and
+
:http://www.eclipse.org/newsgroups/
'''checkout''' the projects to your Eclipse workspace (e.g. c:/stem)
+
  
The repository is also available for browsing  [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ohf/?cvsroot=Technology_Project here].
+
A special Newreader userid and password will be emailed to you. This userid and password is used to subscribe to the Newsgroup using whatever news reader you choose.  
  
 +
<br>The newsgroup is for all of the components of Eclipse, not just STEM. We suggest that when you post to the newsgroup, you include '''STEM''' in the title to ensure it being read by the STEM community.
  
*Committer access to CVS
+
More information about access to the newsgroup is here: [http://www.eclipse.org/newsgroup.php http://www.eclipse.org/newsgroup.php]
  
Developers with commit rights have individual user ids and passwords in the Eclipse project development repository. As a committer you can use SSH (Secure SHell) to connect to the CVS repository as follows. Go to a Repository View and add a new CVS repository location. When asked for the repository location enter the following:
+
<br>For those who use FireFox and Thunderbird as their web browser and email client, the following may help in setting up Thunderbird to read the STEM newsgroup.  
  
{| style="width:60%;" border="0" cellpadding="2"
+
*From menubar, select '''Tools-&gt;Account Setting'''
|+Committer CVS Connection Information
+
*Select '''Add Account'''
|-
+
*Select '''Newsgroup Account''' then '''Next'''
| '''Host''' || dev.eclipse.org  
+
*Enter your name and email address then '''Next''' .
|-
+
*Enter '''news.eclipse.org''' for Newsgroup Server then '''Next'''  
| '''Repository Paths''' || /cvsroot/technology
+
*Enter the userid and password that were sent to you when you signed up.
|-
+
*Enter the account name. The default of News.Eclipse.org is OK.
| '''User''' || (your committer user id, supplied by the webmaster)
+
*You should now have "News.Eclipse.Org" listed in your "Folders"
|-
+
*Select the '''News.Eclipse.org''' folder and then select '''Manage NewsGroup Subscriptions'''  
| '''Password''' || (your committer password)
+
*Expand the list of newsgroups until you find '''Eclipse.technology''' and select it.
|-
+
*You should now have the list of previous postings to the newsgroup and it will be updated every time you get your mail from the server.
| '''Connection Type''' || extssh
+
|-
+
|}
+
  
Once your information is authenticated, you can browse the repository and add projects to your workspace. If you do some changes that you'd like to contribute, after testing and ensuring that you have followed the contribution guidelines, you are free to release your changes to the repository. Of course, you can only release changes to projects for which you have commit rights. The current list of committers for OHF can be found [http://www.eclipse.org/ohf/committers/index.php here].
 
  
Note that you can use the SSH protocol and your Eclipse user id to access projects for which you are not a committer but you will not be able to release changes.
 
  
More information about CVS usage with Eclipse is available [http://wiki.eclipse.org/index.php/CVS_Howto  here].
+
== Submitting code to the STEM project ==
  
==Creating a project in the STEM Source Code Repository with Subclipse==
+
If you are not a project member with full '''committer''' access to SVN, the best way to contribute source code changes is to follow these steps:
When you create a new Eclipse project you are going to want to be able to save it in the source code repository both so it can be backed up and so that you can share it with others. Basically you will create the project using eclipse and then initially store it as a "branch" in the Source Code Repository.  You can invite others to access your branch when you get ready to have others try it.  At some future point, it may be decided that this project should be migrated to the Source Code Trunk. 
+
  
The process is as follows:
+
*Obtain the source code with anonymous access to SVN.
*Create the Eclipse project using the Eclipse wizards.
+
*Make your changes using Eclipse.  
*"Right click" on the project name and select Tean->Share Project 
+
*Test the changes using the most recent version of STEM.
*On the "Share Project with SVN" screen, select "create new repository location"
+
*Prepare a patch as described below.
*On the "Enter Repository Location ..." screen in the "URL" field enter "https://svn.opensource.ibm.com/svn/stem/branches" and click "Next"
+
*If there is a Bugzilla item that describes the need for the change that you are making. then post a description of the patch to the Bugzilla entry and attach the patch.  
*On the "enter FolderName" screen, select "Use project name as folder name" and click "Finish"
+
*If there is no Bugzilla entry, please create one and attach your patch to it. It is much preferred that all patches be related to an appropriate bugzilla entry.  
 +
*If for some reason a bugzilla entry is not appropriate, email your patch to a member of the&nbsp;STEM team who has committer authority, along with a note describing the patch.
  
If it complains about the name, that may be because someone has already created a project with that same name and you need to choose a different name.  Question?  Should there be a project name registry?
+
'''Preparing a patch (Eclipse users)'''
  
If you want to share your project with others,  give them the name of the project and tell them to access the project using the Subclipse command to access it. For example, if you created a project "com.ibm.almaden.stem.dogflu" then they would do the following:
+
Eclipse can create and apply patches in the unified diff format. Assuming you are working out of an Eclipse SVN project created from the&nbsp;STEM SVN repository, you can create a (possibly, multi-file) patch by selecting the desired resource scope in the package explorer and doing "Team/Create Patch...". Because the patch must be applied to the same Eclipse resource it was generated from it is probably safest to select the project itself.
*Go to the SVN Repository perspective.
+
*If "https://svn.opensource.ibm.com/svn/stem/branches" is not listed then "right click" and select New->Repository Location
+
*Enter "https://svn.opensource.ibm.com/svn/stem/branches" as the URL
+
*If the required project is not listed, "right click"->refresh.
+
*Choose "com.ibm.almaden.stem.dogflu" and select "Checkout"
+
  
 +
== Creating a new standalone STEM application ==
  
 +
At some point you may want to create a new version of the standalone Stem application. Or more likely, you may need to verify that changes you have made will work in the standalone version. To create a new temporary version of the STEM application for testing do the following.
  
 +
*Select the '''org.eclipse.stem.ui''' project
 +
**Select '''feature.product'''
 +
**From the Overview tab
 +
***Select '''Export'''
  
 +
It should request the name of the zip file to be generated and build it. It will run for many minutes. The resulting zip file can be expanded in a new directory like "c:\teststem" and invoked from a command window.
  
 +
== Software Engineering Documentation To Do's ==
 +
''TODO''
 +
*Describe the basic development philosophy of the project and the basic ideas behind “agile” software development.
 +
*Explain how to test the system.
 +
*Explain how to submit a bug report.
 +
*Explain how to use JUnit.
  
 +
=== Building The System  ===
  
== Software Engineering Documentation To Do's ==
+
STEM developers can use the headless build mechanism (build plug-ins automatically outside the Eclipse IDE) for building distributions for various OS platforms.<br>
<P>Describe the basic development philosophy of the project and the
+
 
basic ideas behind “agile” software development.</P>
+
The instructions below are for running the build on a Linux machine.
<P>Explain how to build the system</P>
+
 
<P>Explain how to test the system</P>
+
Before using the headless build, make sure you have the following software prerequisites:
<P>Explain how to submit a bug report</P>
+
 
<P>Explain how to use JUnit</P>
+
*Eclipse Platform 4.0 or higher, make sure that the following Eclipse features are also installed:
<P><BR><BR>
+
**BIRT (and all its prerequisite plugins)
</P>
+
*(Alternatively, you can download the latest version of BIRT Report Designer All-in-one)
 +
*Eclipse DeltaPack that matches the version of the above Eclipse
 +
*JDK 8.0 or higher
 +
*SVN for Ant (from [http://subclipse.tigris.org/files/documents/906/43359/svnant-1.2.1.zip here])
 +
 
 +
<br>To build STEM using this mechanism you will need to follow these steps:
 +
 
 +
*Check-out from STEM SVN the plugin '''org.eclipse.stem.releng'''
 +
*Copy the '''local.sh-template''' to '''local.sh''', edit it and change the values within it to fit your local platform:
 +
**''MAJOR_VERSION'' - The version number of the STEM product you are building (e.g., 3.0.0)
 +
**''JAVA_HOME'' - Path to the JDK to be used
 +
**''ECLIPSE_HOME'' - The Eclipse SDK to be used for building
 +
*Put the ''lib'' directory from the SVN for Ant Zip file under a directory named '.ant' in your home dir
 +
*From a command line, change the directory into the above plugin directory and execute the '''build.sh''' script
 +
 
 +
<br>The build script will first fetch the latest plugins code from the SVN and then build those plugins. The last step would be to pack those plugins into an executable product package.<br>If the build finishes successfully, you should see a "BUILD SUCCESSFUL" message at the end of logging messages.<br>The output of the build process is a set of Zip files, one for each different OS platform. Those are located under folder: ''build\I.WeeklyBuild''.
  
 
=STEM subprojects=  
 
=STEM subprojects=  
 
STEM consist of a basic core system and (in the future) many projects that either use the basic system or enhance it.  The following section describe the subprojects that use STEM or enhance it.
 
STEM consist of a basic core system and (in the future) many projects that either use the basic system or enhance it.  The following section describe the subprojects that use STEM or enhance it.
 +
*'''GoogleEarth'''
 +
**Interface to the GoogleEarth application.
 +
*'''Reports'''
 +
**Interface to the Eclipse BIRT product.
 +
*'''DataLog'''
 +
**Simple support for logging STEM data.
 +
 +
  
 
===STEM - GoogleEarth Interface ===  
 
===STEM - GoogleEarth Interface ===  
Line 373: Line 262:
 
To run the STEM-GoogleEarth Interface you need to do the following:
 
To run the STEM-GoogleEarth Interface you need to do the following:
  
* Build the '''ge''' project
+
====Prerequisites====
** Checkout the '''org.eclipse.ohf.stem.ui.ge''' project into Eclipse. (See earlier instructions)
+
** cd to the above '''ge''' project directory.
+
* Start STEM application
+
* Display the STEM-GE Window
+
** windows->open views->Other->Stem->GoogleEarth
+
  
There are numerous ways that you can use GoogleEarth with your simulations.
+
You must have installed the GoogleEarth application which is available for personal use from the
We will not describe them all. The STEM-GE Preference Page is described below. That is where you specify many options that control what the STEM-GE interface does.  
+
[http://earth.google.com/download-earth.html GoogleEarth download site].
The most common use of STEM-GE is to run a simulation and have it write KML files to a folderSimultaneously GoogleEarth is reading these KML files via a webServer and displaying the results of the simulation on the GoogleEarth screen.
+
You should verify that GoogleEarth works correctly on your machine by starting it and verifying that you can browse the 3D image because some older computers do not have the 3D graphics capabilities required by GoogleEarth.  
 +
It is a fun application to play with and when you are done you can leave it running or notIf it is not active, STEM will start it.
  
For example, if you have a predefined scenario that you want to run, then do the following:
+
==== Running STEM and GoogleEarth ====
  
*  
+
*If you are running from the STEM source distribution:
 +
**Update STEM to the latest code level and ensure it is refreshed and rebuilt.
 +
**From project '''org.eclipse.stem.ui.ge''' select '''servlet.xml'''
 +
**Select '''RunAs-&gt;AntBuild'''
 +
**From Eclipse, Run Stem using stem.product in org.eclipse.stem.ui (Described earlier [http://wiki.eclipse.org/index.php/Welcome_STEM_Developers#Running_STEM_as_a_STEM_developer here].
 +
*If you are running the STEM standalone executable, the above steps were already done for you and you just need to start STEM.exe.
  
    '''To be completed'''
+
<br>
  
 +
*In the STEM workspace window
 +
**Windows-&gt;GoogleEarth-&gt;View
 +
***A window will be displayed that contains a Display button and
  
'''STEM-GoogleEarth Preferences'''
+
a set of Radio buttons that select the disease aspect to be displayed.
  
The STEM-GoogleEarth interface has a set of preference that control how the application works.  This preference page is accessed by going to ''windows->preferences'' and selecting the ''GoogleEarth'' entry.  
+
***At this point the GoogleEarth application should start if it was not already started.
 +
**Optionally, select Windows-&gt;Preferences-&gt;STEM-&gt;visualization-&gt;GoogleEarth
 +
***Specify any preferences that you want to use. The defaults are probably OK.
 +
**Using the Scenarios view: select your desired scenario.
 +
***For example: STEM-&gt;Geography-&gt;Continent-&gt;NorthAmerica
 +
***Doubleclick on '''Spanish Flu''' to start the simulation.
 +
**After the simulation has run for 7 or 8 cycles
 +
***Pause the simulation
 +
***Click the display button in the GoogleEarth View.
 +
***You should see the GoogleEarth map showing some red area in the Northeast areas of the USA.
 +
***Start the simulation again
 +
***Pause and Click the GoogleEarth Display button every few cycles. These red areas should grow as the infections spread both across county borders and across the airline connections.
 +
***You can use the GoogleEarth view to change the aspect displayed to any of:
 +
****Infections - Red
 +
****Exposed - Yellow
 +
****Recovered - Green
 +
****Susceptible - Blue
 +
 
 +
As distributed, the GoogleEarth interface runs in manual mode. Because of memory usage of both STEM and GoogleEarth they do not run well together unless your workstation has more than 1 Gigabyte of memory. If you do have lots of memory, you can use the GoogleEarth preferences to change so that as the simulation runs the results are shown simultaneously on the GoogleEarth display.
 +
 
 +
<br>There are numerous ways that you can use GoogleEarth with your simulations. We will not describe them all. The STEM-GE Preference Page is described below. That is where you specify many options that control what the STEM-GE interface does. The most common use of STEM-GE is to run a simulation and have it write KML files to a folder. Simultaneously GoogleEarth is reading these KML files via a webServer and displaying the results of the simulation on the GoogleEarth screen.
 +
 
 +
====STEM-GoogleEarth Preferences====
 +
 
 +
The STEM-GoogleEarth interface has a set of preference that control how the application works.  This preference page is accessed by going to ''windows->preferences->visualization'' and selecting the ''GoogleEarth'' entry.  
  
 
You will get the following window:
 
You will get the following window:
Line 399: Line 316:
  
 
* Preferences
 
* Preferences
** Choose the Method used to display STEM Results
+
<ul>
*** LogOnly <br>With this option, the KML files are written but not displayed by GoogleEarth.
+
  <li><em>Choose the Method used to display STEM Results</em></li>
*** Log+Servlet <br>With this option, the KML files are written and then displayed by GoogleEarth. GoogleEarth actually requests the file from a webserver Servlet whcih actually reads the file.
+
  <ul>
*** AsysncServlet <br>The KML is written on every cycle, overlaying the previous cycle. GoogleEarth asks for new data on a predetermined interval and is sent the current KML.
+
<li><em>LogOnly</em><br>With this option, the KML files  
*** AutoLaunch <br>With this option, the KML files are directly sent to GoogleEarth. This can cause problems because GoogleEarth may get files sent at inopportune times but it is more efficient.
+
are written but not displayed by GoogleEarth.  
*** Manual Launch <br> The map is generated by user selection of ''MapDisplay'' from the context menu (Right Click).
+
This would be used when you are either going to display
** Folder for KML Logging <br>This is the folder where STEM will write the KML files that GoogleEarth will read.  If it already contains KML files, the user will be given the oportunity to delete them, keep them or choose a new folder.
+
the GoogleEarth visualization at a later time or if you are
** Hostname:port for external webserver: <br>This is the required hostname and port for an external webserver. Normally the internal webserver would be used so this is not needed but there are cases where one might want to use an existing web server.   
+
going to run GoogleEarth from another system with the KML
** Use internal webserver <br>This is used to cause the webserver built into Eclipse to be used.
+
files on a shared disk.</li>
** Automatically startup GoogleEarth <br>If specified then when the STEM-GoogleEarth view is started, then the GoogleEarth application is also launched.
+
<li><em>"Log+Servlet"</em><br>With this option,  
** Automatically log KML for every simulation <br>if specifid, then when you start any simulation running, it will automatically log files to GoogleEarth.  Only the first one will be displayed by GE since it would be counterproductive to show 2 different views at the same time.
+
the KML files are written and then displayed by GoogleEarth.  
** Only write a KML file file every Nth cycle <br>If the simulation does not change rapidly from cycle to cycle, significant overhead can be saved by only sending data to GoogleEarth every Nth cycle.
+
GoogleEarth actually requests the file from a webserver  
** Chose the STEM Aspect to be displayed
+
Servlet which reads the file and sends it to GoogleEarth.</li>
*** Susceptible/Exposed/infectious/Recovered
+
<li><em>"AsyncServlet"</em><br>The KML is written on every cycle,  
*** Any/All <br>The graphical display can only display one of the disease attributes at a time. Normally you would select one  of S E I or R but '''Any''' would be specified when you are displaying an existing folder and you want whatever was logged to be displayed. '''All''' would be specified when you want all of the disease statistics to be logged. (4 files for each cycle).
+
overlaying the file written for the previous cycle. GoogleEarth  
** Allow debug output on Console <br>This is used for problem determination.
+
asks for new data on a predetermined interval and is sent the  
 +
current KML. The advantage over the previous method is that
 +
it helps keep GoogleEarth from falling to far behind the STEM processing.</li>
 +
<li><em>"DirectLaunch"</em><br>With this option, the KML files  
 +
are directly sent to GoogleEarth without using an intermediate
 +
web server. This can cause problems because GoogleEarth may get  
 +
files faster than it can process them but it is more efficient.</li>
 +
<li><em>"ManualDisplay"</em><br> The map is generated by user  
 +
clicking the <b>Display</b> button. This is the default option.</li>
 +
  </ul>
 +
  <li><em>"Folder for KML logging:"</em>This is the folder where  
 +
STEM will write the KML files that GoogleEarth will read.   
 +
If it already contains KML files, the user will be given  
 +
the oportunity to delete them, keep them or choose a new folder.</li>
 +
  <li><em>"User internal webserver"</em><br>This is used to cause the
 +
    webserver built into Eclipse to be used.</li>
 +
  <li><em>"Hostname:port for external webserver"</em>
 +
    <br>This is the required hostname and port for an external webserver.  
 +
    Normally the internal webserver would be used so this is not needed  
 +
    but there are cases where one might want to use an existing web server. </li>
 +
   <li><em>"Automatically startup GoogleEarth"</em>
 +
    <br>If specified then when the STEM-GoogleEarth view is started,
 +
    then the GoogleEarth application is also launched.</li>
 +
  <li><em>"Automatically process every simulation"</em>
 +
    <br>if specified, then when you start any simulation running,  
 +
    it will automatically have its processing be mapped to GoogleEarth.   
 +
    Only the first one will be displayed by GE since it would  
 +
    be counterproductive to show 2 different views at the same time. </li>
 +
  <li><em>"Write KML files only every N th cycle"</em>
 +
    <br>If the simulation does not change rapidly from cycle to cycle,  
 +
    significant overhead can be saved by only sending data to  
 +
    GoogleEarth every Nth cycle.</li>
 +
</ul>
 +
 
 +
==STEM Reports SubProject==
 +
The Reports subproject provides an interface to the Eclipse BIRT component.  
 +
BIRT is an open source Eclipse-based reporting system that integrates with STEM to produce custom reports.
 +
 
 +
** instructions to be provided later **
 +
 
 +
 
  
 
=STEM Data=
 
=STEM Data=
Line 451: Line 408:
 
A list of all the county (Administration 0) codes is here: http://www.davros.org/misc/iso3166.html
 
A list of all the county (Administration 0) codes is here: http://www.davros.org/misc/iso3166.html
  
==Data Location and Usage==
+
==An Introduction to STEM's Properties Files==
'''STEM Projects that contain data'''
+
In STEM II, we store data about a country and its administrative divisions in properties files. Properties files are used to define identifiers and to set population and area data
The following STEM projects contain data files that are used in STEM. In most cases these projects also contains code that processes the data.
+
at all levels (i.e. country, state, or county level). There are four main types of properties files: area, population, nodes, and names. The purpose of each property file will be
* Eclipse Project names
+
explained later.
** '''org.eclipse.ohf.stem.geography''' <br>Contains the Latitude/Longitude data for the Administration areas.
+
 
** '''org.eclipse.ohf.stem.internal.data''' <br>Contains the properties files that describe all of the data in STEM.
+
In STEM II, most of the data is standardized ISO 3166 or FIPS (Federal Information Processing Standards) data. According to Wikipedia, "ISO 3166 is a three-part geographic coding standard for
** '''org.eclipse.ohf.stem.data''' <br>No longer actively used and should be ignored.
+
coding the names of countries and dependent areas, and the principal subdivisions thereof." ISO 3166 specifies standard alpha-2 codes, alpha-3 codes, and three-digit country codes as well. For example,
** '''com.ibm.almaden.stem.utility''' <br>Code and data used to prepare the properties files
+
for the USA, the alpha-2 code would be US, the alpha-3 code is USA, and the three-digit country code is 840. In addittion, we use FIPS codes whenever possible to create identifiers that represent a
 +
political administration. For instance, the FIPS for New York County (i.e., Manhattan) is 36061 while Kings County (i.e., Brooklyn) is 36047. The first two digits of the FIPS identify the state.
 +
 
 +
==Administrative Levels and Properties Files==
 +
 
 +
Administrative levels correspond to political divisions of a country. A level 0 administration identifies an entire country (e.g. USA or Mexico). A level 1 administration corresponds to a subdivision of a country
 +
which can be a state, territory, parish, or a province. As an example we have that California, Colorado, and New York are all level 1 administrations of the USA. Similarly, level 2 administrations are subdivisions of a level 1 administration. Orange County, Monterey County, and Napa County are all level 2 administrations that belong to California.
 +
 
 +
A property file is a plain-text file that contains either area data, population data, or other relevant data related to the political divisions of a country at different administration levels. Properties files are located
 +
under org.eclipse.ohf.stem.internal.data\resources\data\country. There are four types of properties files :
 +
 
 +
*Names property file :  This file defines the identifiers for every administrative division in a country at each level. For example, for the USA, at level 0 we would have "USA = United States". At level 1, we would have "US-CA = California" , "US-CO = Colorado", and "US-NY = New York".
 +
At level 2, for Orange, Monterey, and Napa counties within California we have "US-CA-06059 = Orange County", "US-CA-06053 = Monterey County", and "US-CA-06055 = Napa County". There is a single names property
 +
file for every country. The corresponding names property file for the USA is USA_names.properties. For level 2 administrations, the five digits found on the identifiers (i.e.
 +
"06053" for identifier "US-CA-06053" are defined as follows : the leftmost two digits identify the level 1 administration ( "06" -> California ) while the remaining
 +
digits, which can be up to four, identify the level 2 administration ("053" -> Monterey County).     
 +
 
 +
*Nodes property file : This file provides additional information about identifiers of administrative divisions. There is a nodes property file for each administration level. There is a fixed format that is used in these files and it is as follows:
 +
<code>
 +
  '''# Format: Code = Name, ISO-3166-2 numeric code, Two letter code''' 
 +
  For example, at level 0 for the USA we have
 +
  "USA = United States, 840, US".
 +
  At level 1, for California, Colorado, and New York we have :
 +
    "US-CA = California, 06, CA" , 
 +
    "US-CO = Colorado, 08, CO", and 
 +
    "US-NY = New York, 36, NY".
 +
</code>
 +
The nodes property files corresponding to the USA are USA_0_node.properties, USA_1_node.properties, and USA_2_node.properties.
 +
 
 +
*Area property file : This file contains area data (in square kilometers) for administrative divisions. There is an area property file for each administration level.
 +
In the case of the USA at level 0, we have "USA = 9161923".
 +
At level 1, we have "US-CA = 163695.57", "US-CO = 104093.57",
 +
and "US-NY = 54556.00" for California, Colorado and New York respectively. At level 2, for Orange, Monterey, and Napa counties within California
 +
we have "US-CA-06059 = 2043.5006", "US-CA-06053 = 8603.9405", and "US-CA-06055 = 1952.8510". The area property files corresponding to the USA are USA_0_area.properties, USA_1_area.properties, and USA_2_area.properties.
 +
 
 +
 
 +
*Population property file : This file contains population data for administrative divisions. There is a population property file for each administration level.  In the case of the USA at level 0, we have "USA = 298444215".
 +
At level 1, we have "US-CA = 33871648 ", "US-CO = 4301261", and "US-NY = 18976457" for California, Colorado and New York respectively. At level 2, for Orange, Monterey, and Napa counties within California
 +
we have "US-CA-06059 = 2846289", "US-CA-06053 = 401762", and "US-CA-06055 = 124279". The population property files corresponding to the USA are USA_0_human.properties, USA_1_human.properties, and USA_2_human.properties.
 +
 
 +
== Updating values in a property file ==
 +
 
 +
To update a value in a property file you should follow a few simple steps. First, given that you know the name of the country and/or administrative division to be updated, look under org.eclipse.stem.internal.data\resources\data\country\&lt;three letter identifier for the country&gt; for the folder that belongs to a given country. In the case of the USA, we would go to org.eclipse.stem.internal.data\resources\data\country\USA where all the property files are located. Second, search in the names property file to find the identifier of the administration we want to update. Third, open the corresponding file (area, population, or node file) where the update will take place. Finally, search for the identifier and once found, update its value. As an example, let's say we want to update by increasing 100 square kilometers the area of Orange County which belongs to California, USA. Then, we would open org.eclipse.stem.internal.data\resources\data\country\USA\USA_names.properties and do a search for the identifier of "Orange County" which is "US-CA-06059". Then, since we learn from the names property file that Orange County is a level 2 administration, we would open "org.eclipse.stem.internal.data\resources\data\country\USA\USA_2_area.properties" and do a new search for identifier "US-CA-06059". Our search will find ""US-CA-06059 = 2043.5006" Finally, we do the update by adding 100 square kilometers to the current value "US-CA-06059 = 2143.5006".
 +
 
 +
Once we've&nbsp;updated the correct file(s), we need to run org.eclipse.stem.internal.data\resources\src\Main.java. By running Main.java we create a new serialized file that will take the changes into account. The new serialized file can be found under org.eclipse.stem.internal.data\temp\data\scenario\disease. Next time STEM II executes a scenario, it will load the new serialized file.
 +
 
 +
== An Introduction to STEM Map Files ==
 +
 
 +
'''STEM Maps'''
 +
 
 +
Maps for STEM are XML files that follow a GML format. These files are located under: '''org.eclipse.stem.geography\resources\data\geo\country\XXX''' where XXX is the three letter identifier for the country. There is also a second set of files at '''org.eclipse.stem.geography\resources\data\geo\country_reduced''' Which set is used is determined by a preference at '''Windows-&gt;Preferences-&gt;STEM-&gt;Visualization-&gt;MapDataManagement'''
 +
 
 +
There can be multiple map files for a country, one for each administration level. As an example, for the USA we have the following two maps: USA_1_MAP.xml and USA_2_MAP.xml. According to Wikipedia, GML is an XML grammar defined by the Open Geospatial Consortium (OGC) to express geographical features. As an example, the set of GML elements that define Orange county are: <code></code>
 +
 
 +
  //Sample taken from
 +
  org.eclipse.stem.geography\resources\
 +
    data\geo\country\USA\USA_2_MAP.xml
 +
  ...
 +
  &lt;gml:Polygon gml:id="US-CA-06053"&gt;
 +
  &lt;gml:outerBoundaryIs&gt;
 +
  &lt;gml:LinearRing&gt;
 +
  &lt;gml:posList&gt;
 +
  36.9186 -121.7019 36.9197 -121.6999 ... 36.9186 -121.7019
 +
  &lt;/gml:posList&gt;
 +
  &lt;/gml:LinearRing&gt;&lt;/gml:outerBoundaryIs&gt;
 +
  &lt;/gml:Polygon&gt;
 +
  ...
 +
 
 +
 
 +
'''Updating a Map'''
 +
 
 +
To update a map, that is, to provide more accurate latitude and longitude data for a given location we follow a few steps. We find the identifier for the location we are trying to update. This step is similar to the step described for updating values in a property file. Once we have found it, we replace the latitute and longitude values contained within the &lt;gml:postList&gt; &lt;/gml:postList&gt; tags. We need to make sure that the starting and ending latitute and longitude values are the same, otherwise it wont be accepted as a closed polygon. In the example above, note that for Monterey County ("US-CA-06053") the starting latitude and longitude values "36.9186 -121.7019" are the same as the ending ones "36.9186 -121.7019".
 +
 
 +
The '''country_reduced''' files have had the border data smoothed to reduce the number of lat/long points used to describe a border. It can take sometimes thousands of lat/long points to describe a border accurately. By smoothing the border data so that the border is described by many fewer points, we can save both CPU and memory usage.
 +
 
 +
== Data Location and Usage ==
 +
 
 +
'''STEM Projects that contain data''' The following STEM projects contain data files that are used in STEM. In most cases these projects also contains code that processes the data.  
 +
 
 +
*Eclipse Project names  
 +
**'''org.eclipse.stem.geography''' <br>Contains the Latitude/Longitude data for the Administration areas.  
 +
**'''org.eclipse.stem.internal.data''' <br>Contains the properties files that describe all of the data in STEM.  
 +
**'''org.eclipse.stem.data''' <br>No longer actively used and should be ignored.  
 +
**'''org.eclipse.stem.utility''' <br>Code and data used to prepare the properties files.
 +
 
 +
'''Property files in org.eclipse.stem.internal.data''' The following property files are found in the following directory. ''NNN'' is the 3 character Country code and N is the administration level.
  
''' Property files in org.eclipse.ohf.stem.internal.data'''
 
The following property files are found in the following directory. 
 
''NNN'' is the 3 character Country code and N is the administration level.
 
 
 
*'''resource/data/country/XXX/''' There is a separate subdirectory for each country.  
 
*'''resource/data/country/XXX/''' There is a separate subdirectory for each country.  
**''XXX_N_area.properties'' This property file contains the area of the administration level in square kilometers. The Key is the key for the administration level ( level 0 = USA; Level 1 = US-CA; Level 2 = US-CA-06075
+
**''XXX_N_area.properties'' This property file contains the area of the administration level in square kilometers. The Key is the key for the administration level ( level 0 = USA; Level 1 = US-CA; Level 2 = US-CA-06075  
**''XXX_N_human_2006_population.properties'' This property file contains the population of the administration level.  
+
**''XXX_N_human_2006_population.properties'' This property file contains the population of the administration level.  
**''XXX_N_node.properties'' This property file contains the full name of the administration area. >> we need an example of how to access it <<
+
**''XXX_N_node.properties'' This property file contains the full name of the administration area. &gt;&gt; we need an example of how to access it &lt;&lt;
**''XXX_names.properties'' This property file contains a xref of both Level 1 and level 2 names.  
+
**''XXX_names.properties'' This property file contains a xref of both Level 1 and level 2 names.
  
 
*'''resource/data/relationship''' This contains the following subdirectories.  
 
*'''resource/data/relationship''' This contains the following subdirectories.  
**''airtransport''
+
**''airtransport''  
**''commonborder'' See below.
+
**''commonborder'' See below.  
**''airtransport''
+
**''airtransport''  
 
**''airtransport''
 
**''airtransport''
  
+
<br>
*'''resource/data/decorator/disease'''  This contains property files that describe existing diseases.
+
  
 +
*'''resource/data/decorator/disease''' This contains property files that describe existing diseases.
 +
 +
<br>'''Spacial Data (Geographic Coordinates)''' that provides the latitude and longitude coordinates for the borders of the Administration areas.
  
'''Spacial Data (Geographic Coordinates)''' that provides the latitude and longitude coordinates for the borders of the Administration areas.
 
 
  <code>   
 
  <code>   
   Project: org.eclipse.ohf.stem.geography
+
   Project: org.eclipse.stem.geography
 
   Path:  resource/data/geo/country/XXX/
 
   Path:  resource/data/geo/country/XXX/
 
       XXX_0_MAP.xml
 
       XXX_0_MAP.xml
Line 488: Line 527:
 
       XXX_2_MAP.xml
 
       XXX_2_MAP.xml
 
    
 
    
   Example URI: platform:/plugin/org.eclipse.ohf.stem.geography/
+
   Example URI: platform:/plugin/org.eclipse.stem.geography/
 
                     resources/data/geo/country/USA/USA_2_MAP.xml
 
                     resources/data/geo/country/USA/USA_2_MAP.xml
</code>
+
</code>
 
  <code>
 
  <code>
 
   Key Format:  
 
   Key Format:  
Line 512: Line 551:
  
  
'''Common Border Data''' that describes which Administration areas have common borders.
+
 
 +
<br>'''Common Border Data''' that describes which Administration areas have common borders.  
 +
 
 
  <code>   
 
  <code>   
   Project: org.eclipse.ohf.stem.internal.data
+
   Project: org.eclipse.stem.internal.data
 
   Path:  resource/data/relationship/commonborder/
 
   Path:  resource/data/relationship/commonborder/
 
       XXX_2_XXX_2.properties   
 
       XXX_2_XXX_2.properties   
Line 524: Line 565:
 
       The property files will list the bordering admin level 2 areas.  
 
       The property files will list the bordering admin level 2 areas.  
 
   
 
   
   Example URI: platform:/plugin/org.eclipse.ohf.stem.internal.data/
+
   Example URI: platform:/plugin/org.eclipse.stem.internal.data/
 
                 resources/data/relationship/
 
                 resources/data/relationship/
 
                 commonborder/AFG_2_CHN_2.properties
 
                 commonborder/AFG_2_CHN_2.properties
 +
</code>
 +
  
 
   Generated by:  NeighborUtility.java       
 
   Generated by:  NeighborUtility.java       
   Located in:    org.eclipse.ohf.stem.internal.data
+
   Located in:    org.eclipse.stem.internal.data
</code>
+
  
 
=Other Information=
 
=Other Information=
Line 557: Line 599:
 
</TD>
 
</TD>
 
<TD WIDTH=23%>
 
<TD WIDTH=23%>
<P>V2.0.1</P>
+
<P>V3.6</P>
 
</TD>
 
</TD>
 
<TD WIDTH=49%>
 
<TD WIDTH=49%>
Line 565: Line 607:
 
<TR VALIGN=TOP>
 
<TR VALIGN=TOP>
 
<TD WIDTH=28%>
 
<TD WIDTH=28%>
<P>IBM's Java<FONT FACE="Times New Roman, serif">™</FONT> 5 JDK</P>
+
<P>IBM's Java<FONT FACE="Times New Roman, serif">™</FONT> 6 JDK</P>
 
</TD>
 
</TD>
 
<TD WIDTH=23%>
 
<TD WIDTH=23%>
<P>5</P>
+
<P>6</P>
 
</TD>
 
</TD>
 
<TD WIDTH=49%>
 
<TD WIDTH=49%>
Line 575: Line 617:
 
</TD>
 
</TD>
 
</TR>
 
</TR>
<TR VALIGN=TOP>
+
<TD WIDTH=28%>
+
<P>Subversion</P>
+
</TD>
+
<TD WIDTH=23%>
+
<P>V1.30</P>
+
</TD>
+
<TD WIDTH=49%> [http://subversion.tigris.org/ http://subversion.tigris.org/]
+
</TD>
+
</TR>
+
<TR VALIGN=TOP>
+
<TD WIDTH=28%>
+
<P>TortoiseSVN</P>
+
</TD>
+
<TD WIDTH=23%>
+
<P>V1.4</P>
+
</TD>
+
<TD WIDTH=49%>
+
[http://tortoisesvn.tigris.org/ http://tortoisesvn.tigris.org/]
+
                        </TD>
+
</TR>
+
 
<TR VALIGN=TOP>
 
<TR VALIGN=TOP>
 
<TD WIDTH=28%>
 
<TD WIDTH=28%>
Line 602: Line 624:
 
</TD>
 
</TD>
 
<TD WIDTH=23%>
 
<TD WIDTH=23%>
V3.2.1
+
V3.7
 
</TD>
 
</TD>
 
<TD WIDTH=49%>
 
<TD WIDTH=49%>
Line 634: Line 656:
 
</TABLE>
 
</TABLE>
  
== Eclipse Features and Plug-ins ==
+
== Optional Eclipse Features and Plug-ins ==
 
{| class="wikitable" border="0" cellpadding="4" cellspacing="1"
 
{| class="wikitable" border="0" cellpadding="4" cellspacing="1"
 
! Plug-in !! Description !! URL  
 
! Plug-in !! Description !! URL  
Line 642: Line 664:
 
| UMLet
 
| UMLet
 
| UML class diagram drawing tool
 
| UML class diagram drawing tool
| [http://homepage.mac.com/martin.auer/umlet/ http://homepage.mac.com/martin.auer/umlet]
+
| [http://www.umlet.com/ http://www.umlet.com/]
 
|-
 
|-
 
| Metrics
 
| Metrics
Line 652: Line 674:
 
| [http://sourceforge.net/projects/merlingenerator/ http://sourceforge.net/projects/merlingenerator]
 
| [http://sourceforge.net/projects/merlingenerator/ http://sourceforge.net/projects/merlingenerator]
 
|-
 
|-
| EMF 2.2
+
|  
| Latest EMF downloads
+
| [http://download.eclipse.org/tools/emf/scripts/downloads.php http://download.eclipse.org/tools/emf/scripts/downloads.php]
+
 
|}
 
|}
  
Line 661: Line 681:
  
  
<H1 CLASS="western">Books</H1>
+
==Books==
 
<P>[http://www.oreilly.com/catalog/javanut5/]Java in a Nutshell, Fifth Edition  Flanagan, D., O'Reilly, 2005,
 
<P>[http://www.oreilly.com/catalog/javanut5/]Java in a Nutshell, Fifth Edition  Flanagan, D., O'Reilly, 2005,
 
ISBN 0-596-00773-6. <I>This is a great reference for Java<FONT FACE="Times New Roman, serif">™</FONT>.</I>
 
ISBN 0-596-00773-6. <I>This is a great reference for Java<FONT FACE="Times New Roman, serif">™</FONT>.</I>
Line 695: Line 715:
 
ISBN 0-596-00759-0. This is a great book on how to manage software development in general not just open source.  We're using it
 
ISBN 0-596-00759-0. This is a great book on how to manage software development in general not just open source.  We're using it
 
as a guide for this project.</I></P>
 
as a guide for this project.</I></P>
 
== Web Sites ==
 
'''STEMTODO'''
 
*The STEM II project [http://www.eclipse.org/ohf/stem/ main web site] provides access to project documents, bug tracking, and a web interface to the source code repository
 
*The STEM SVN [http://w3.opensource.ibm.com/plugins/scmsvn/viewcvs.php/root=stem repository browser].
 
*The main web site of the [http://www.eclipse.org/ eclipse project].
 
*A collection of [http://eclipse-plugins.2y.net/eclipse/index.jsp eclipse plug-ins].
 
 
== Transition to Eclipse OHF project ==
 
 
'''STEMTODO'''
 
 
During the next month (Feb 2007) we will be moving STEM to be a sub project of the Eclipse Open Healthcare Framework
 
 
* Eclipse OHF website
 
:The following link will take you to the OHF page on the eclipse.org website.
 
:http://www.eclipse.org/ohf/
 
---
 
* OHF WIKI
 
:The following is the OHF top Wiki page.
 
::http://wiki.eclipse.org/index.php/OHF
 
---
 
* OHF Mailing List
 
:Open Healthcare Framework Mailing list <ohf-dev@eclipse.org>
 
:https://dev.eclipse.org/mailman/listinfo/ohf-dev
 
 
:The archives for ohf-dev mailing list is at:
 
:http://dev.eclipse.org/mhonarc/lists/ohf-dev/maillist.html
 
---
 
* Newsgroup
 
:The Newsgroup for OHF is:
 
:news://news.eclipse.org/eclipse.technology.ohf
 
 
:To access the newsgroup you will need to go to the following page and request a userid and password.  (Note: you apparently have no choice in the selection of either the userid or the password)
 
 
:http://www.eclipse.org/newsgroups/
 
---
 
* Bug reporting
 
:Eclipse uses the Bugzilla system for reporting and processing Bug reports for the OHF project.
 
 
:The following page has a link for creation of a Bugzilla account.
 
:https://bugs.eclipse.org/bugs/
 
 
:The bug reporting page is:
 
:https://bugs.eclipse.org/bugs/enter_bug.cgi?product=OHF
 
 
:The following page can be used to find bugs in OHf.
 
:https://bugs.eclipse.org/bugs/query.cgi
 
:Enter OHF as the '''PRODUCT'''
 

Latest revision as of 13:47, 21 October 2015

STEM TOP BAR.gif

STEM Contents Page

Introduction

This article is intended to be the starting point for developers working on the Spatio-Temporal Epidemiological Modeler Project (STEM) code base. It contains a detailed description of how the project is organized, how to find all of the resources associated with the project, and how to install, configure and use the necessary development environment.

One of the advantages of this document is that it gets newcomers “up and running” faster and it allows them to contribute to the project immediately by correcting any inaccuracies or omissions the document contains or by clarifying any descriptions. As a newcomer to the project you have a totally unique and extremely valuable perspective. If you can't find something or it doesn't make sense to you then you're probably not alone. When you find out the answer to your quandary, please record it here so those that come after you can “stand on your shoulders.” Welcome aboard!

The STEM code base is written in Java Version 6 and is organized as a set of well defined components using the eclipse (www.eclipse.org) plug-in tool framework. The components are integrated through a well defined “extension point” mechanism that makes the entire code base highly extensible. The use of the Eclipse framework also provides for multi-platform portability.

Interestingly, Eclipse is also the STEM project's Java development environment. If this seems a bit strange at first, don't worry. As you understand more about eclipse and the project this will make more sense. The project is organized and managed as an open source project. Much of the inspiration, philosophy and techniques for its management are taken directly from the book producing open source software by Karl Fogel (see the references later in this document).

Step-by-step for STEM developers

  • Read this article.
  • When you find missing, confusing, erroneous information, please contribute by fixing the problem. This Wiki allows any Bugzilla registered user to update content. Please do!
  • Visit the STEM project page and bookmark it.
  • Obtain an Eclipse.org account to access bugzilla, the newsgroup, etc.
  • Subscribe to the stem-dev mailing list. The mailing list is used for all internal developer communication.
  • Visit the Newsgroup
  • Install Java 8.0 and Eclipse 4.x [1]
  • Obtain the source code for the STEM project from the code repository. [2]

You can also review the article on Eclipse Development Resources http://wiki.eclipse.org/index.php/Development_Resources

Software Engineering

Coding Conventions and Guidelines

As with any product being built by a team, there are various areas where standards, conventions, and other guidelines can play a role in helping to ensure that the resulting product presents to developers and customers as a unified whole rather than as a loose collection of parts worked on by a variety of individuals each with their own styles and ways of working.

The STEM project will use the conventions and guidelines used by the Eclipse project. See the Eclipse Development Conventions and Guidelines.

The most important guidelines for STEM code are to have good JavaDoc documentation and not generate compiler warnings.

The Eclipse compiler verifies the code against a preference list and will generate warning messages for code that does not follow rules specified in the compiler preferences. For example, it will check for "unchecked Generic type operations" and give a warning. We ask you remove all of the Eclipse compiler warnings before submitting the code. Also preferences for the following JavaDoc options should be changed to generate warnings and the warnings removed by fixing the code and/or creating accurate and informative Javadoc comments,

 window->preferences->java->compiler->javadoc:
    Process Javadoc comments
      Malformed Javadoc Comments:         warning
      Missing Javadoc tags(public)        warning
      Missing Javadoc comments(Public)    warning

If you are the owner or creator of a STEM subproject, you can create more restrictive standards by checking in a set of project preferences. However, please do not remove warnings by checking in project preferences that ignore bad coding practices.


Copyright Statement

A copyright should be on any human created text document including HTML,Java, properties, XML, XSD. Auto generated/machine made text files does not need to have a copyright. Such files are compiled from a human created file and it is not practical to add copyright to them (as we wouldn't add copyright to class files). To be specific, we are referring to auto generated JavaDocs (HTML) and EMF models (JAVA). For .java files the copyright statement should follow the package statement and proceed the import statements. The Copyright statement is further described here


package org.eclipse.stem;   
/*************************************************************************
* Copyright (c) 2006,2007 IBM Corporation and others.
* All rights reserved. This program and the accompanying materials
* are made available under the terms of the Eclipse Public License v1.0
* which accompanies this distribution, and is available at
* http://www.eclipse.org/legal/epl-v10.html
*
* Contributors:
*     IBM Corporation - initial API and implementation
*************************************************************************/ 
import ... ;


To have the above automatically inserted in new code, do the following:

  • Select the above comment (from /* to */)
  • Select Window->preferences
  • Select Java->Code Style->CodeTemplates->Types
  • Select edit
  • Paste the copyright statement in place of the existing "Author" statement.
  • Select OK

National Language Support

See page on STEM National Language Support

Bug Reporting

Eclipse uses the Bugzilla system for reporting and processing Bug reports and enhancement request for the STEM project. Once a problem or enhancement has been submitted to Bugzilla, the Bugzilla entry should be used for subsequent discussion of the issue.

  • The following page is used for creation of a Bugzilla account. [3]
  • The following page is used to set the user email preferences. [4]
  • The bug reporting page is: [5] Specify STEM as the component
  • The following page can be used to find bugs in STEM https://bugs.eclipse.org/bugs/query.cgi
    • Enter Technology as the Classification
    • Enter STEM as the Product

The following article shows the life cycle of a bug from entering the system to being closed. http://www.eclipse.org/projects/dev_process/bugzilla-use.php

Before reporting a bug, please read the Eclipse bug writing guidelines and please search to see if the bug has already been reported.

There are some important considerations when you submit a Bugzilla report. If it is a problem that you are reporting, then the promptness of a resolution is very much dependent on the completeness of the report. In most cases, a problem needs to be reproduced in order to fix it. Before you press submit read your report over and think about whether if you were the STEM developer that gets the problem, would you have the information you need to reproduce the problem.

An important help in problem determination is the Error Log view on the STEM window. If any exceptions or serious errors occurred they will show up there. These entries can be exported to a file and attached to the bugzilla entry.

Most of the development of STEM was done on Windows with Eclipse 3.8 or above. If you are using a different platform, be sure to mention this just in case this is a platform problem.

If you are submitting a suggestion for a new feature or enhancement, include some justification and enough details that it can be evaluated. Since others may want to contribute to a discussion of the issue, it would be a good idea to post a pointer to the Bugzilla entry on the STEM newsgroup.

STEM Mailing List

Send stem-dev mailing list submissions to <stem-dev@eclipse.org>

To subscribe or unsubscribe via the World Wide Web, visit

https://dev.eclipse.org/mailman/listinfo/stem-dev

or, via email, send a message with subject of body 'help' to

     stem-dev-request@eclipse.org

You can reach the person managing the list at

    stem-dev-owner@eclipse.org

When replying, please edit your Subject line so it is more more specific than "Re: Contents of stem-dev digest..." 


The developer's mailing list should be used to communicate with other STEM developers. It is preferred over private emails.

STEM Newsgroup

The Newsgroup for STEM is news://news.eclipse.org/eclipse.technology

To access the newsgroup you will need to go to the following page and request a userid and password.

http://www.eclipse.org/newsgroups/

A special Newreader userid and password will be emailed to you. This userid and password is used to subscribe to the Newsgroup using whatever news reader you choose.


The newsgroup is for all of the components of Eclipse, not just STEM. We suggest that when you post to the newsgroup, you include STEM in the title to ensure it being read by the STEM community.

More information about access to the newsgroup is here: http://www.eclipse.org/newsgroup.php


For those who use FireFox and Thunderbird as their web browser and email client, the following may help in setting up Thunderbird to read the STEM newsgroup.

  • From menubar, select Tools->Account Setting
  • Select Add Account
  • Select Newsgroup Account then Next
  • Enter your name and email address then Next .
  • Enter news.eclipse.org for Newsgroup Server then Next
  • Enter the userid and password that were sent to you when you signed up.
  • Enter the account name. The default of News.Eclipse.org is OK.
  • You should now have "News.Eclipse.Org" listed in your "Folders"
  • Select the News.Eclipse.org folder and then select Manage NewsGroup Subscriptions
  • Expand the list of newsgroups until you find Eclipse.technology and select it.
  • You should now have the list of previous postings to the newsgroup and it will be updated every time you get your mail from the server.


Submitting code to the STEM project

If you are not a project member with full committer access to SVN, the best way to contribute source code changes is to follow these steps:

  • Obtain the source code with anonymous access to SVN.
  • Make your changes using Eclipse.
  • Test the changes using the most recent version of STEM.
  • Prepare a patch as described below.
  • If there is a Bugzilla item that describes the need for the change that you are making. then post a description of the patch to the Bugzilla entry and attach the patch.
  • If there is no Bugzilla entry, please create one and attach your patch to it. It is much preferred that all patches be related to an appropriate bugzilla entry.
  • If for some reason a bugzilla entry is not appropriate, email your patch to a member of the STEM team who has committer authority, along with a note describing the patch.

Preparing a patch (Eclipse users)

Eclipse can create and apply patches in the unified diff format. Assuming you are working out of an Eclipse SVN project created from the STEM SVN repository, you can create a (possibly, multi-file) patch by selecting the desired resource scope in the package explorer and doing "Team/Create Patch...". Because the patch must be applied to the same Eclipse resource it was generated from it is probably safest to select the project itself.

Creating a new standalone STEM application

At some point you may want to create a new version of the standalone Stem application. Or more likely, you may need to verify that changes you have made will work in the standalone version. To create a new temporary version of the STEM application for testing do the following.

  • Select the org.eclipse.stem.ui project
    • Select feature.product
    • From the Overview tab
      • Select Export

It should request the name of the zip file to be generated and build it. It will run for many minutes. The resulting zip file can be expanded in a new directory like "c:\teststem" and invoked from a command window.

Software Engineering Documentation To Do's

TODO

  • Describe the basic development philosophy of the project and the basic ideas behind “agile” software development.
  • Explain how to test the system.
  • Explain how to submit a bug report.
  • Explain how to use JUnit.

Building The System

STEM developers can use the headless build mechanism (build plug-ins automatically outside the Eclipse IDE) for building distributions for various OS platforms.

The instructions below are for running the build on a Linux machine.

Before using the headless build, make sure you have the following software prerequisites:

  • Eclipse Platform 4.0 or higher, make sure that the following Eclipse features are also installed:
    • BIRT (and all its prerequisite plugins)
  • (Alternatively, you can download the latest version of BIRT Report Designer All-in-one)
  • Eclipse DeltaPack that matches the version of the above Eclipse
  • JDK 8.0 or higher
  • SVN for Ant (from here)


To build STEM using this mechanism you will need to follow these steps:

  • Check-out from STEM SVN the plugin org.eclipse.stem.releng
  • Copy the local.sh-template to local.sh, edit it and change the values within it to fit your local platform:
    • MAJOR_VERSION - The version number of the STEM product you are building (e.g., 3.0.0)
    • JAVA_HOME - Path to the JDK to be used
    • ECLIPSE_HOME - The Eclipse SDK to be used for building
  • Put the lib directory from the SVN for Ant Zip file under a directory named '.ant' in your home dir
  • From a command line, change the directory into the above plugin directory and execute the build.sh script


The build script will first fetch the latest plugins code from the SVN and then build those plugins. The last step would be to pack those plugins into an executable product package.
If the build finishes successfully, you should see a "BUILD SUCCESSFUL" message at the end of logging messages.
The output of the build process is a set of Zip files, one for each different OS platform. Those are located under folder: build\I.WeeklyBuild.

STEM subprojects

STEM consist of a basic core system and (in the future) many projects that either use the basic system or enhance it. The following section describe the subprojects that use STEM or enhance it.

  • GoogleEarth
    • Interface to the GoogleEarth application.
  • Reports
    • Interface to the Eclipse BIRT product.
  • DataLog
    • Simple support for logging STEM data.


STEM - GoogleEarth Interface

STEM is a computer software system for defining and visualizing simulations of the geographical spread of contagious diseases. One way to visualize this geographical information is to overlay it on the GoogleEarth 3D world model.

The STEM-GoogleEarth project (STEM-GE) enables the logging of the spatial data with disease statistics included as it runs. Either simultaneously or after the fact, the logged data(in the form of KML files) is read by GoogleEarth and displayed by mapping disease state to color intensity.

To run the STEM-GoogleEarth Interface you need to do the following:

Prerequisites

You must have installed the GoogleEarth application which is available for personal use from the GoogleEarth download site. You should verify that GoogleEarth works correctly on your machine by starting it and verifying that you can browse the 3D image because some older computers do not have the 3D graphics capabilities required by GoogleEarth. It is a fun application to play with and when you are done you can leave it running or not. If it is not active, STEM will start it.

Running STEM and GoogleEarth

  • If you are running from the STEM source distribution:
    • Update STEM to the latest code level and ensure it is refreshed and rebuilt.
    • From project org.eclipse.stem.ui.ge select servlet.xml
    • Select RunAs->AntBuild
    • From Eclipse, Run Stem using stem.product in org.eclipse.stem.ui (Described earlier here.
  • If you are running the STEM standalone executable, the above steps were already done for you and you just need to start STEM.exe.


  • In the STEM workspace window
    • Windows->GoogleEarth->View
      • A window will be displayed that contains a Display button and

a set of Radio buttons that select the disease aspect to be displayed.

      • At this point the GoogleEarth application should start if it was not already started.
    • Optionally, select Windows->Preferences->STEM->visualization->GoogleEarth
      • Specify any preferences that you want to use. The defaults are probably OK.
    • Using the Scenarios view: select your desired scenario.
      • For example: STEM->Geography->Continent->NorthAmerica
      • Doubleclick on Spanish Flu to start the simulation.
    • After the simulation has run for 7 or 8 cycles
      • Pause the simulation
      • Click the display button in the GoogleEarth View.
      • You should see the GoogleEarth map showing some red area in the Northeast areas of the USA.
      • Start the simulation again
      • Pause and Click the GoogleEarth Display button every few cycles. These red areas should grow as the infections spread both across county borders and across the airline connections.
      • You can use the GoogleEarth view to change the aspect displayed to any of:
        • Infections - Red
        • Exposed - Yellow
        • Recovered - Green
        • Susceptible - Blue

As distributed, the GoogleEarth interface runs in manual mode. Because of memory usage of both STEM and GoogleEarth they do not run well together unless your workstation has more than 1 Gigabyte of memory. If you do have lots of memory, you can use the GoogleEarth preferences to change so that as the simulation runs the results are shown simultaneously on the GoogleEarth display.


There are numerous ways that you can use GoogleEarth with your simulations. We will not describe them all. The STEM-GE Preference Page is described below. That is where you specify many options that control what the STEM-GE interface does. The most common use of STEM-GE is to run a simulation and have it write KML files to a folder. Simultaneously GoogleEarth is reading these KML files via a webServer and displaying the results of the simulation on the GoogleEarth screen.

STEM-GoogleEarth Preferences

The STEM-GoogleEarth interface has a set of preference that control how the application works. This preference page is accessed by going to windows->preferences->visualization and selecting the GoogleEarth entry.

You will get the following window: File:GEPreferences.jpg

  • Preferences
  • Choose the Method used to display STEM Results
    • LogOnly
      With this option, the KML files are written but not displayed by GoogleEarth. This would be used when you are either going to display the GoogleEarth visualization at a later time or if you are going to run GoogleEarth from another system with the KML files on a shared disk.
    • "Log+Servlet"
      With this option, the KML files are written and then displayed by GoogleEarth. GoogleEarth actually requests the file from a webserver Servlet which reads the file and sends it to GoogleEarth.
    • "AsyncServlet"
      The KML is written on every cycle, overlaying the file written for the previous cycle. GoogleEarth asks for new data on a predetermined interval and is sent the current KML. The advantage over the previous method is that it helps keep GoogleEarth from falling to far behind the STEM processing.
    • "DirectLaunch"
      With this option, the KML files are directly sent to GoogleEarth without using an intermediate web server. This can cause problems because GoogleEarth may get files faster than it can process them but it is more efficient.
    • "ManualDisplay"
      The map is generated by user clicking the Display button. This is the default option.
  • "Folder for KML logging:"This is the folder where STEM will write the KML files that GoogleEarth will read. If it already contains KML files, the user will be given the oportunity to delete them, keep them or choose a new folder.
  • "User internal webserver"
    This is used to cause the webserver built into Eclipse to be used.
  • "Hostname:port for external webserver"
    This is the required hostname and port for an external webserver. Normally the internal webserver would be used so this is not needed but there are cases where one might want to use an existing web server.
  • "Automatically startup GoogleEarth"
    If specified then when the STEM-GoogleEarth view is started, then the GoogleEarth application is also launched.
  • "Automatically process every simulation"
    if specified, then when you start any simulation running, it will automatically have its processing be mapped to GoogleEarth. Only the first one will be displayed by GE since it would be counterproductive to show 2 different views at the same time.
  • "Write KML files only every N th cycle"
    If the simulation does not change rapidly from cycle to cycle, significant overhead can be saved by only sending data to GoogleEarth every Nth cycle.

STEM Reports SubProject

The Reports subproject provides an interface to the Eclipse BIRT component. BIRT is an open source Eclipse-based reporting system that integrates with STEM to produce custom reports.

    • instructions to be provided later **


STEM Data

In Process documentation This section is in the very early stages of documentation. Please help complete it.


An important component of STEM is the large collection of data that it contains. This section will attempt to describe the different data that is available, where it is and how it is used.

Definitions

Administration Level The administration level for geographic data refers to the "political resolution" of the data sets. The United Nations defines political divisions called "Administration Levels".

The highest level is "UN Administration Level 0" which corresponds to "countries". We have 244 such "countries", we get our definition of a country from the ISO-3166-1 codes, of which there are 244. There is a ISO-3166-1 code for the United States (a country), and one for Puerto Rico (not a country, but a slightly separate political division). For the United States the ISO-3166-1 2 character code is US and the ISO-3166-1 3 character code is USA.

The next level down is "UN Administration Level 1", which for the United States are the states and for Canada are the provinces and territories. Below that is "UN Administration Level 2" The definition of these areas varies from country to country which is why we use the "level" instead. Level 2 is the limit on our current data sets. In the future we could add higher resolutions. For instance, for a small country like Israel we might go to level 3 and use something like a census track as the location.


Latitude: Latitude gives the location of a place on Earth north or south of the Equator. Latitude is an angular measurement in degrees (marked with °) ranging from 0° at the Equator to 90° at the poles (90° N for the North Pole or 90° S for the South Pole). In Stem, latitudes north of the Equator are positive values and south of the equator are negative.

Longitude: Longitude describes the location of a place on Earth east or west of a north-south line called the Prime Meridian. Longitude is given as an angular measurement ranging from 0° at the Prime Meridian to +180° eastward and −180° westward. The Greenwich meridian is the universal prime meridian or zero point of longitude.

ISO3166 code ISO 3166 is a three-part geographic coding standard for coding the names of countries and dependent areas, and the principal subdivisions thereof. The official name is Codes for the representation of names of countries and their subdivisions.

  • ISO 3166-1 codes for country and dependent area names.
    • ISO 3166-1 alpha-2 two-letter country codes
    • ISO 3166-1 alpha-3 three-letter country codes
    • ISO 3166-1 numeric three-digit country codes
    • ISO 3166-2 Codes for the representation of names of countries and their subdivisions -- Part 2: Country subdivision code - defines codes for the principal subdivisions of a country or dependent area.

A list of all the county (Administration 0) codes is here: http://www.davros.org/misc/iso3166.html

An Introduction to STEM's Properties Files

In STEM II, we store data about a country and its administrative divisions in properties files. Properties files are used to define identifiers and to set population and area data at all levels (i.e. country, state, or county level). There are four main types of properties files: area, population, nodes, and names. The purpose of each property file will be explained later.

In STEM II, most of the data is standardized ISO 3166 or FIPS (Federal Information Processing Standards) data. According to Wikipedia, "ISO 3166 is a three-part geographic coding standard for coding the names of countries and dependent areas, and the principal subdivisions thereof." ISO 3166 specifies standard alpha-2 codes, alpha-3 codes, and three-digit country codes as well. For example, for the USA, the alpha-2 code would be US, the alpha-3 code is USA, and the three-digit country code is 840. In addittion, we use FIPS codes whenever possible to create identifiers that represent a political administration. For instance, the FIPS for New York County (i.e., Manhattan) is 36061 while Kings County (i.e., Brooklyn) is 36047. The first two digits of the FIPS identify the state.

Administrative Levels and Properties Files

Administrative levels correspond to political divisions of a country. A level 0 administration identifies an entire country (e.g. USA or Mexico). A level 1 administration corresponds to a subdivision of a country which can be a state, territory, parish, or a province. As an example we have that California, Colorado, and New York are all level 1 administrations of the USA. Similarly, level 2 administrations are subdivisions of a level 1 administration. Orange County, Monterey County, and Napa County are all level 2 administrations that belong to California.

A property file is a plain-text file that contains either area data, population data, or other relevant data related to the political divisions of a country at different administration levels. Properties files are located under org.eclipse.ohf.stem.internal.data\resources\data\country. There are four types of properties files :

  • Names property file : This file defines the identifiers for every administrative division in a country at each level. For example, for the USA, at level 0 we would have "USA = United States". At level 1, we would have "US-CA = California" , "US-CO = Colorado", and "US-NY = New York".

At level 2, for Orange, Monterey, and Napa counties within California we have "US-CA-06059 = Orange County", "US-CA-06053 = Monterey County", and "US-CA-06055 = Napa County". There is a single names property file for every country. The corresponding names property file for the USA is USA_names.properties. For level 2 administrations, the five digits found on the identifiers (i.e. "06053" for identifier "US-CA-06053" are defined as follows : the leftmost two digits identify the level 1 administration ( "06" -> California ) while the remaining digits, which can be up to four, identify the level 2 administration ("053" -> Monterey County).

  • Nodes property file : This file provides additional information about identifiers of administrative divisions. There is a nodes property file for each administration level. There is a fixed format that is used in these files and it is as follows:

 # Format: Code = Name, ISO-3166-2 numeric code, Two letter code   
 For example, at level 0 for the USA we have 
 "USA = United States, 840, US". 
 At level 1, for California, Colorado, and New York we have :
   "US-CA = California, 06, CA" ,  
   "US-CO = Colorado, 08, CO", and  
   "US-NY = New York, 36, NY". 

The nodes property files corresponding to the USA are USA_0_node.properties, USA_1_node.properties, and USA_2_node.properties.

  • Area property file : This file contains area data (in square kilometers) for administrative divisions. There is an area property file for each administration level.

In the case of the USA at level 0, we have "USA = 9161923". At level 1, we have "US-CA = 163695.57", "US-CO = 104093.57", and "US-NY = 54556.00" for California, Colorado and New York respectively. At level 2, for Orange, Monterey, and Napa counties within California we have "US-CA-06059 = 2043.5006", "US-CA-06053 = 8603.9405", and "US-CA-06055 = 1952.8510". The area property files corresponding to the USA are USA_0_area.properties, USA_1_area.properties, and USA_2_area.properties.


  • Population property file : This file contains population data for administrative divisions. There is a population property file for each administration level. In the case of the USA at level 0, we have "USA = 298444215".

At level 1, we have "US-CA = 33871648 ", "US-CO = 4301261", and "US-NY = 18976457" for California, Colorado and New York respectively. At level 2, for Orange, Monterey, and Napa counties within California we have "US-CA-06059 = 2846289", "US-CA-06053 = 401762", and "US-CA-06055 = 124279". The population property files corresponding to the USA are USA_0_human.properties, USA_1_human.properties, and USA_2_human.properties.

Updating values in a property file

To update a value in a property file you should follow a few simple steps. First, given that you know the name of the country and/or administrative division to be updated, look under org.eclipse.stem.internal.data\resources\data\country\<three letter identifier for the country> for the folder that belongs to a given country. In the case of the USA, we would go to org.eclipse.stem.internal.data\resources\data\country\USA where all the property files are located. Second, search in the names property file to find the identifier of the administration we want to update. Third, open the corresponding file (area, population, or node file) where the update will take place. Finally, search for the identifier and once found, update its value. As an example, let's say we want to update by increasing 100 square kilometers the area of Orange County which belongs to California, USA. Then, we would open org.eclipse.stem.internal.data\resources\data\country\USA\USA_names.properties and do a search for the identifier of "Orange County" which is "US-CA-06059". Then, since we learn from the names property file that Orange County is a level 2 administration, we would open "org.eclipse.stem.internal.data\resources\data\country\USA\USA_2_area.properties" and do a new search for identifier "US-CA-06059". Our search will find ""US-CA-06059 = 2043.5006" Finally, we do the update by adding 100 square kilometers to the current value "US-CA-06059 = 2143.5006".

Once we've updated the correct file(s), we need to run org.eclipse.stem.internal.data\resources\src\Main.java. By running Main.java we create a new serialized file that will take the changes into account. The new serialized file can be found under org.eclipse.stem.internal.data\temp\data\scenario\disease. Next time STEM II executes a scenario, it will load the new serialized file.

An Introduction to STEM Map Files

STEM Maps

Maps for STEM are XML files that follow a GML format. These files are located under: org.eclipse.stem.geography\resources\data\geo\country\XXX where XXX is the three letter identifier for the country. There is also a second set of files at org.eclipse.stem.geography\resources\data\geo\country_reduced Which set is used is determined by a preference at Windows->Preferences->STEM->Visualization->MapDataManagement

There can be multiple map files for a country, one for each administration level. As an example, for the USA we have the following two maps: USA_1_MAP.xml and USA_2_MAP.xml. According to Wikipedia, GML is an XML grammar defined by the Open Geospatial Consortium (OGC) to express geographical features. As an example, the set of GML elements that define Orange county are:

  //Sample taken from
 org.eclipse.stem.geography\resources\
   data\geo\country\USA\USA_2_MAP.xml
 ...
 <gml:Polygon gml:id="US-CA-06053">
 <gml:outerBoundaryIs>
 <gml:LinearRing>
 <gml:posList>
 36.9186 -121.7019 36.9197 -121.6999 ... 36.9186 -121.7019
 </gml:posList>
 </gml:LinearRing></gml:outerBoundaryIs>
 </gml:Polygon>
 ...
 

Updating a Map

To update a map, that is, to provide more accurate latitude and longitude data for a given location we follow a few steps. We find the identifier for the location we are trying to update. This step is similar to the step described for updating values in a property file. Once we have found it, we replace the latitute and longitude values contained within the <gml:postList> </gml:postList> tags. We need to make sure that the starting and ending latitute and longitude values are the same, otherwise it wont be accepted as a closed polygon. In the example above, note that for Monterey County ("US-CA-06053") the starting latitude and longitude values "36.9186 -121.7019" are the same as the ending ones "36.9186 -121.7019".

The country_reduced files have had the border data smoothed to reduce the number of lat/long points used to describe a border. It can take sometimes thousands of lat/long points to describe a border accurately. By smoothing the border data so that the border is described by many fewer points, we can save both CPU and memory usage.

Data Location and Usage

STEM Projects that contain data The following STEM projects contain data files that are used in STEM. In most cases these projects also contains code that processes the data.

  • Eclipse Project names
    • org.eclipse.stem.geography
      Contains the Latitude/Longitude data for the Administration areas.
    • org.eclipse.stem.internal.data
      Contains the properties files that describe all of the data in STEM.
    • org.eclipse.stem.data
      No longer actively used and should be ignored.
    • org.eclipse.stem.utility
      Code and data used to prepare the properties files.

Property files in org.eclipse.stem.internal.data The following property files are found in the following directory. NNN is the 3 character Country code and N is the administration level.

  • resource/data/country/XXX/ There is a separate subdirectory for each country.
    • XXX_N_area.properties This property file contains the area of the administration level in square kilometers. The Key is the key for the administration level ( level 0 = USA; Level 1 = US-CA; Level 2 = US-CA-06075
    • XXX_N_human_2006_population.properties This property file contains the population of the administration level.
    • XXX_N_node.properties This property file contains the full name of the administration area. >> we need an example of how to access it <<
    • XXX_names.properties This property file contains a xref of both Level 1 and level 2 names.
  • resource/data/relationship This contains the following subdirectories.
    • airtransport
    • commonborder See below.
    • airtransport
    • airtransport


  • resource/data/decorator/disease This contains property files that describe existing diseases.


Spacial Data (Geographic Coordinates) that provides the latitude and longitude coordinates for the borders of the Administration areas.

  
 Project: org.eclipse.stem.geography
 Path:   resource/data/geo/country/XXX/
      XXX_0_MAP.xml
      XXX_1_MAP.xml
      XXX_2_MAP.xml
 
 Example URI: platform:/plugin/org.eclipse.stem.geography/
                    resources/data/geo/country/USA/USA_2_MAP.xml


 Key Format: 
   For level 1 data, the key is the ISO3166-2 code. 
     An ISO3166-2 code is composed as follows: 
       Two letter country code followed by up to three 
       alphanumeric characters for the level 1 administration.
   For level 2 data, the key is the ISO3166-2 code followed by
   up to six digits. 
     The leftmost two digits indicate the level 1 container of a 
     level 2 administration (i.e. California is a level 1
     container for Orange County; a level 2 administration).  
     The two digits were taken from a lexicographic sorting of 
     all the level 1 administrations within a country. 
     Similarly, the four leftmost digits indicate the level 2
     administration. 
     Again, these four digits are an index into the lexicographic
     sorting of all level 2 administrations within a level 1
     administration.



Common Border Data that describes which Administration areas have common borders.

  
 Project: org.eclipse.stem.internal.data
 Path:   resource/data/relationship/commonborder/
      XXX_2_XXX_2.properties  
                Common Borders between admin areas within country XXX 
      XXX_2_YYY_2.properties  
                Common Borders between admin areas of country XXX to 
                the admin areas of Country YYY where XXX and YYY have 
                a common border.
      The property files will list the bordering admin level 2 areas. 

 Example URI: platform:/plugin/org.eclipse.stem.internal.data/
                resources/data/relationship/
                commonborder/AFG_2_CHN_2.properties


 Generated by:  NeighborUtility.java      
 Located in:    org.eclipse.stem.internal.data

Other Information

Tools

The following is a list of software that is useful to have while working on the STEM project.

Tool

Version

URL

OpenOffice

V3.6

http://www.openoffice.org

IBM's Java 6 JDK

6

http://w3.hursley.ibm.com/java/jim

Eclipse IDE

V3.7

http://www.eclipse.org/downloads/

Jdraw

V1.1,4

[http://www.j-domain.de/homepage.php?page=20

Optional Eclipse Features and Plug-ins

Plug-in Description URL
Subclipse Subversion client Subversion client
UMLet UML class diagram drawing tool http://www.umlet.com/
Metrics Java code metrics computations http://metrics.sourceforge.net
Merline Generator GEF generator from EMF http://sourceforge.net/projects/merlingenerator

Reference Sources

Books

<P>[6]Java in a Nutshell, Fifth Edition Flanagan, D., O'Reilly, 2005, ISBN 0-596-00773-6. This is a great reference for Java.

Indispensable.

[7]Eclipse Modeling Framework Budinsky, F., et al, Addison-Wesley. 2003. ISBN 0131425420. This is THE book on the eclipse modeling framework (EMF). Note- a second edition is due out very soon, and is available for pre-order

[8]Official Eclipse 3.0 FAQs Arthorne, J., Laffra, C., Addison-Wesley, 2004, ISBN 0321268385. This is an excellent source of quick answers to great questions.

[9]Eclipse:Building Commercial-Quality Plug-Ins Clayberg, E., Rubel, D., Addison-Wesley, 2004. ISBN: 0321228472. Good reference, but starts out a bit too simply.

[10]Eclipse Distilled Carlson, D., Addison-Wesley, 2005. ISBN 0321288157. A concise guide with advanced explanations of how to exploit the features of eclipse. Very useful for people who are already familiar with eclipse because it adds an extra layer of sophistication to the reader's bag-of-tricks.

[11]Eclipse Rich Client Platform: Designing, Coding, and Packaging Java Applications McAffer, J., et al., Addison-Wesley, 2005, ISBN 0321334612.

[12]Eclipse Cookbook Holzner, S., O'Reilly, 2004, ISBN 0-596-00710-8. Good “how to” guide to use eclipse, less for eclipse plug-in development itself.</P> [13] Eclipse Holzner, S., O'Reilly, 2004, ISBN 0-596-00641-1. Good starter for new eclipse users/developers.</P> "The Definitive Guide to SWT and JFace Harris, R., Warner, R., Apress, 2004, ISBN 1-509059-325-1.</P>

[14]UML for Java Programmers Martin, R., Prentice Hall, 2003, ISBN 0131428489. Good “what you need to know” introduction to UML.

[15]Producing Open Source Software Fogel, K., O'Reilly, 2005, ISBN 0-596-00759-0. This is a great book on how to manage software development in general not just open source. We're using it as a guide for this project.</I>

Copyright © Eclipse Foundation, Inc. All Rights Reserved.