Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Performance Test Fingerprint Guidelines

Revision as of 07:22, 30 April 2007 by Daniel Megert.ch.ibm.com (Talk | contribs) (New page: This page describes the guidelines for tests that show up in the global or component performance fingerprint (aka summary). The fingerprints have to be targeted at users that want to get a...)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

This page describes the guidelines for tests that show up in the global or component performance fingerprint (aka summary). The fingerprints have to be targeted at users that want to get a quick overview of what became faster or slower when using Eclipse. We will have to improve the support for developers, e.g. allow to get the bar charts for tests that aren't in the fingerprint but that's outside the 3.3 scope.

Component Fingerprint

  1. the test must cover a common scenario that the user sees/experiences e.g. startup, full build, opening a concrete wizard or part that exists in the workbench, perspective switch, etc.
  2. the test must be relevant for that component i.e. if it turns red you must not say, e.g. "the test is not really significant because..."
  3. the test should at least take 100ms but longer is of course better
  4. the test must be stable i.e. don't take one that looks like a sawtooth
  5. give the test a name that fits the scenario and mention the amount of iterations e.g. use "Activate 30 Java Editors" instead of just "Activating an Editor"
  6. at the end of a release a degraded test must not be red but gray with an explanation why it degraded
  7. don't put more than 20 tests into the fingerprint as it should give an overview that shows the big picture

Global Fingerprints

Each component contributes up to 2 most common component fingerprints to the global list.

Back to the top