Revision as of 10:35, 29 April 2013

The EMF-IncQuery Event-driven Virtual Machine

Overview

Nowadays, collaboration and scalability challenges in modeling tools are typically addressed with dedicated problem-specific solutions e.g.:

• On-the-fly constraint evaluation engines (to provide scalability for model validation)
• Incremental model transformation tools (to address performance issues of e.g. model synchronization)
• Incremental model comparison algorithms (to support versioning and model merge in collaborative scenarios)
• Design space exploration tools (to optimize models towards a design goal).

The common recurring task in these applications is to capture and process not only the models, but also their changes as a stream of events (operations that affect models). We generalized this approach to provide a common conceptual framework for an event-driven virtual machine (EVM) architecture.

The EVM is a rule-based system with a special focus on versatile model transformations, with built-in support for reacting to model and query result changes and user interactions. The EVM integrates various execution schemes into a uniform and flexible architecture, to provide a common framework that even allows for combinations of advanced model transformation scenarios, e.g. by the interleaving of various execution strategies (batch, live/triggered and exploratory) within a single transformation program.

Core architecture

Overview of the Event-driven VM

The event-driven virtual machine allows the central management of executable actions defined on event sources and can execute these actions automatically with a predefined schedule.

• An activation is wrapper of a pattern match with a corresponding rule instance and state.
  • Activation states are: inactive, appeared, fired, updated, disappeared
• A rule instance manages the activations corresponding to a rule specification in a given EMF-IncQuery engine.
• A rule specification defines the life cycle for changing the activation state in response to events and the possible actions that can be executed on an activation in a given state.
  • Events related to a life cycle are: match appears/disappears/updates, activation fires
  • State transitions in a life cycle always have a source state, an event and a target state. There can be only one (source state, event) pair in the life cycle, thus the target state is always deterministic.
  • Jobs are atomic actions that are performed if an activation is fired when in a state defined by the job.
  • An activation is enabled if there is at least one job that is defined for the current state of the activation.
• An agenda is a collection of rule instances with an added responsibility of ordering the enabled activations of all rule instances related to the same EMF-IncQuery engine
  • The agenda keeps track of the activations of the rule instances by an activation notification mechanism. Rule instances notify the agenda if one of their activations changed state in response to an event.
• An executor is responsible for executing enabled activations in the agenda when it is scheduled to do so, and to provide an execution context to store execution results or other data related to execution.
• A scheduler is defined to respond to some kind of global event (e.g. transaction commit, user request, or EMF-IncQuery Base update callback) by scheduling its executor.
• A rule engine is created for a given EMF-IncQuery engine and an optional set of rule specifications, and has its own agenda.
• A execution schema is a special rule engine, that also has a scheduler set up.

Example code

We illustrate the two main usage modes of the EVM with UML models. First, we have to define the preconditions with patterns, then define the rule specifications which can be added to a rule engine or execution schema.

The example project is on the repository: UML EVM Example

Precondition pattern definition

The code example below shows the possibleSuperClass and onlyInheritedOperations patterns of that are based on the UML example, the complete query definition can be found in our repository: preconditions.eiq

/* Precondition for add generalization rule */
pattern possibleSuperClass(cl : Class, sup : Class) {
	neg find superClass(cl, _otherSup);
	neg find superClass(_cl2, sup);
}
/* Precondition for create owned operation */
pattern onlyInheritedOperations(cl : Class) {
	find hasOperation(cl, _op);
	neg find ownsOperation(cl, _ownOp);
}

Rule specifications

We define two rule specifications, both encapsulated by a method in UMLexampleForEVM.java.

The first rule specification uses the possibleSuperClass pattern as a precondition and when executed for a given class pair, it creates a new Generalization element to set the class sup as a superclass for cl. The life-cycle is the most simple, where neither the updated, nor the disappeared state is used.

// the job specifies what to do when an activation is fired in the given state
Job job = Jobs.newStatelessJob(ActivationState.APPEARED, new PossibleSuperClassProcessor() {
  @Override
  public void process(Class cl, Class sup) {
    System.out.println("Found cl " + cl + " without superclass");
    Generalization generalization = UMLFactory.eINSTANCE.createGeneralization();
    generalization.setGeneral(sup);
    generalization.setSpecific(cl);
  }
});
// the life-cycle determines how events affect the state of activations
DefaultActivationLifeCycle lifecycle = DefaultActivationLifeCycle.DEFAULT_NO_UPDATE_AND_DISAPPEAR;
// the factory is used to initialize the matcher for the precondition
IMatcherFactory<PossibleSuperClassMatcher> factory = PossibleSuperClassMatcher.factory();
// the rule specification is a model-independent definition that can be used to instantiate a rule
RuleSpecification spec = Rules.newSimpleMatcherRuleSpecification(factory, lifecycle, Sets.newHashSet(job));

The second rule specification is similar, it uses the onlyInheritedOperations pattern and when executed it creates a new operation with name newOp and adds it to the class which had no own property before.

Job job = Jobs.newStatelessJob(ActivationState.APPEARED, new OnlyInheritedOperationsProcessor() {
  @Override
  public void process(Class cl) {
    System.out.println("Found class " + cl + " without operation");
    Operation operation = UMLFactory.eINSTANCE.createOperation();
    operation.setName("newOp");
    operation.setClass_(cl);
  }
});
DefaultActivationLifeCycle lifecycle = DefaultActivationLifeCycle.DEFAULT_NO_UPDATE_AND_DISAPPEAR;
IMatcherFactory<OnlyInheritedOperationsMatcher> factory = OnlyInheritedOperationsMatcher.factory();
RuleSpecification spec = Rules.newSimpleMatcherRuleSpecification(factory, lifecycle, Sets.newHashSet(job));

Fire activations manually using a rule engine

The first option when using the EVM is creating a rule engine, which manages the set of activations, but will not fire the enabled activations. A rule engine has no context of its own, so the user can create one to use when firing activations. The rule specifications above are returned by the two getter methods, and the addRule method is used on the rule engine to instantiate the rules.

// create rule engine over IncQueryEngine
RuleEngine ruleEngine = RuleEngines.createIncQueryRuleEngine(engine);
// create context for execution
Context context = Context.create();
// prepare rule specifications
RuleSpecification createGeneralization = getCreateGeneralizationRule();
RuleSpecification createOperation = getCreateOperationRule();
// add rule specifications to engine
ruleEngine.addRule(createGeneralization);
ruleEngine.addRule(createOperation);

Once a rule specification is added to the rule engine, the existing activations of a given rule can be retrieved from the rule engine and can fire them manually. Alternatively, the next activation as selected by the conflict resolver (which is a simple hash set without ordering) can be retrieved and fired.

// check rule applicability
Set<Activation> createClassesActivations = ruleEngine.getActivations(createGeneralization);
if (!createClassesActivations.isEmpty()) {
    // fire activation of a given rule
    createClassesActivations.iterator().next().fire(context);
}
// check for any applicable rules
while (!ruleEngine.getConflictingActivations().isEmpty()) {
    // fire next activation as long as possible
    ruleEngine.getNextActivation().fire(context);
}

As long as the rule engine exists, it will keep on managing the activations of added rules. It is possible to remove a single rule, which will remove all its activations from the rule engine, and the rule engine can be disposed when not needed anymore.

// rules that are no longer needed can be removed
ruleEngine.removeRule(createGeneralization);
// rule engine manages the activations of the added rules until disposed
ruleEngine.dispose();

Fire activations automatically with an execution schema

Usage scenarios

Both the data binding and validation frameworks of EMF-IncQuery use the EVM for handling events.

• Data binding: observable match result collections are created in the createRuleSpecification method of ObservableCollectionHelper (org.eclipse.incquery.databinding.runtime.collection package)
• Validation constraints are created in the constructor of ConstraintAdapter (org.eclipse.incquery.validation.runtime package)

Programming against the EVM API

Efficiently reacting to pattern match set changes

TODO

Design decisions and code style

These guidelines are derived from the main decision to create a defensive framework to minimize the internal argument checks and idiot-proofing required.

User interaction with the framework

• Users interact with rules and activations through Façade classes:
  • RuleEngine façade provides access to an Agenda and it's rule instances
  • ExecutionSchema façade provides access to a Scheduler and through that to the Executor
  • These Façade classes can be retrieved through the static methods of EventDrivenVM or by static create methods (for specific implementations)
• Any object that a user can access through the Façade must have only public methods that do not endanger their engine (e.g. Activation.fire(), but not Activation.setState())
• Any collection that a user can access through the Façade must be immutable to avoid modifications (e.g. getActivations)
• Any object that is provided by the user must be copied if it's later modification can cause internal problems (e.g. life cycle for rules)

Parameters, input checking and logging

• Method parameters cannot be null!
  • This is checked by Preconditions.checkNotNull(ref, msg). Return a meaningful message on null.
  • Use the this.field = checkNotNull(field, msg) form in constructors when possible.
  • Define delegate methods where optional parameters are allowed.
• All logging is done through the IncQueryEngine, if available, use the debug level for detailed report messages and error or warning when encountering a real problem (e.g. IncQueryException)

Default implementations

There are a high number of notification mechanisms and event processing, that must have an interface for extendibility, a good default implementation and a clear way of overriding.

• In RuleInstance notification providers and listeners are created in prepareX methods.
• Default life cycles prepared with unmodifiable static instances.
• Update complete provider implementations (IQBase and EMF transaction).
• Scheduler implementations (update complete and timed).
• Job implementations (stateless with a single match processor, and recording for transactional model modification).

Development ideas

Related issues on BugZilla

• engine lifecycle: https://bugs.eclipse.org/bugs/show_bug.cgi?id=398744
• activation ordering: https://bugs.eclipse.org/bugs/show_bug.cgi?id=403825
• exception handling: https://bugs.eclipse.org/bugs/show_bug.cgi?id=404307

Other future tasks not yet created as an issue

• systematic testing: both unit and component tests on API classes and default implementation

• add identifier to RuleSpecification (also allow finding rule by id in RuleEngine?)

• support main transformation scenarios
  • batch
  • single job appear-disappear
  • semi and fully automated execution
  • ignore initial activations

• support retrieval of filtered activations of a rule with a partial match

• integrate into upstream components
  • query-based features

• ensure that commands stored by RecordingJobs are only kept in Context when necessary or for a short time (e.g. clear at start of scheduled execution?)

• make EVM core EMF and EMF-IncQuery independent (IPatternMatch as Activation context is acceptable, since interface is clear)

• event queueing is problematic (job execution during activation firing can change the state)