Jump to: navigation, search

Orion/Server scalability

This plan lays out the work areas required to re architect the current Orion Java server for very large scale deployment. This is not all work we will be able to scope within a single release, but is intended to provide a roadmap of the work required over the coming releases to make this happen. Work items are roughly sorted by order of priority.

Assessment of current state

Work areas for scalability

Metadata backing store

As of Orion 2.0, the server metadata is stored in Equinox preferences. There are four preference trees for the various types of metadata: Users, Workspaces, Projects, and Sites. These are currently persisted in four files, one per preference tree. There are a number of problems with this implementation:

  1. Storing large amounts of metadata in a single file that needs to be read on most requests is a severe bottleneck.
  2. Migration of metadata format across builds/releases is ad-hoc and error prone
  3. Individual preference node access/storage is thread safe, but many Orion API calls require multiple read/writes to multiple nodes. There is no way to synchronize access/update across nodes.
  4. Metadata file format does not lend itself to administrative tools for managing users, migrating, etc.

Move search engine/indexer to separate process(es)

Move long running Git operations out of process

Move Java session state out of process

Externalize log configuration

Search for implicit assumptions about system tools (exec calls)