Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
VIATRA/Transformation/EventDrivenVM
Contents
The VIATRA Event-driven Virtual Machine
Overview
Nowadays, collaboration and scalability challenges in modeling tools are typically addressed with dedicated problem-specific solutions e.g.:
- On-the-fly constraint evaluation engines (to provide scalability for model validation)
- Incremental model transformation tools (to address performance issues of e.g. model synchronization)
- Incremental model comparison algorithms (to support versioning and model merge in collaborative scenarios)
- Design space exploration tools (to optimize models towards a design goal).
The common recurring task in these applications is to capture and process not only the models, but also their changes as a stream of events (operations that affect models). We generalized this approach to provide a common conceptual framework for an event-driven virtual machine (EVM) architecture.
The EVM is a rule-based system with a special focus on versatile model transformations, with built-in support for reacting to model and query result changes and user interactions. The EVM integrates various execution schemes into a uniform and flexible architecture, to provide a common framework that even allows for combinations of advanced model transformation scenarios, e.g. by the interleaving of various execution strategies (batch, live/triggered and exploratory) within a single transformation program.
Core architecture
The event-driven virtual machine allows the central management of executable actions defined on event sources and can execute these actions automatically with a predefined schedule.
- An activation is wrapper of a pattern match with a corresponding rule instance and state.
- Activation states are: inactive, appeared, fired, updated, disappeared
- A rule instance manages the activations corresponding to a rule specification in a given VIATRA Query engine.
- A rule specification defines the life cycle for changing the activation state in response to events and the possible actions that can be executed on an activation in a given state.
- Events related to a life cycle are: match appears/disappears/updates, activation fires
- State transitions in a life cycle always have a source state, an event and a target state. There can be only one (source state, event) pair in the life cycle, thus the target state is always deterministic.
- Jobs are atomic actions that are performed if an activation is fired when in a state defined by the job.
- An activation is enabled if there is at least one job that is defined for the current state of the activation.
- An agenda is a collection of rule instances with an added responsibility of ordering the enabled activations of all rule instances related to the same VIATRA Query engine
- The agenda keeps track of the activations of the rule instances by an activation notification mechanism. Rule instances notify the agenda if one of their activations changed state in response to an event.
- An executor is responsible for executing enabled activations in the agenda when it is scheduled to do so, and to provide an execution context to store execution results or other data related to execution.
- A scheduler is defined to respond to some kind of global event (e.g. transaction commit, user request, or VIATRA Query Base update callback) by scheduling its executor.
- A rule engine is created for a given VIATRA Query engine and an optional set of rule specifications, and has its own agenda.
- A execution schema is a special rule engine, that also has a scheduler set up.
Example code
We illustrate the two main usage modes of the EVM with UML models. First, we have to define the preconditions with patterns, then define the rule specifications which can be added to a rule engine or execution schema.
The example project is on the repository: UML EVM Example
Precondition pattern definition
The code example below shows the possibleSuperClass and onlyInheritedOperations patterns of that are based on the UML example, the complete query definition can be found in our repository: preconditions.vql
/* Precondition for add generalization rule */ pattern possibleSuperClass(cl : Class, sup : Class) { neg find superClass(cl, _otherSup); neg find superClass(_cl2, sup); } /* Precondition for create owned operation */ pattern onlyInheritedOperations(cl : Class) { find hasOperation(cl, _op); neg find ownsOperation(cl, _ownOp); }
Rule specifications
We define two rule specifications, both encapsulated by a method in UMLexampleForEVM.java.
The first rule specification uses the possibleSuperClass pattern as a precondition and when executed for a given class pair, it creates a new Generalization element to set the class sup as a superclass for cl. The life-cycle is the most simple, where neither the updated, nor the disappeared state is used.
// the job specifies what to do when an activation is fired in the given state Job job = Jobs.newStatelessJob(CRUDActivationStateEnum.APPEARED, new PossibleSuperClassProcessor() { @Override public void process(Class cl, Class sup) { System.out.println("Found cl " + cl + " without superclass"); Generalization generalization = UMLFactory.eINSTANCE.createGeneralization(); generalization.setGeneral(sup); generalization.setSpecific(cl); } }); // the life-cycle determines how events affect the state of activations DefaultActivationLifeCycle lifecycle = DefaultActivationLifeCycle.DEFAULT_NO_UPDATE_AND_DISAPPEAR; // the factory is used to initialize the matcher for the precondition IMatcherFactory<PossibleSuperClassMatcher> factory = PossibleSuperClassMatcher.factory(); // the rule specification is a model-independent definition that can be used to instantiate a rule RuleSpecification spec = Rules.newSimpleMatcherRuleSpecification(factory, lifecycle, Sets.newHashSet(job));
The second rule specification is similar, it uses the onlyInheritedOperations pattern and when executed it creates a new operation with name newOp and adds it to the class which had no own property before.
Job job = Jobs.newStatelessJob(CRUDActivationStateEnum.APPEARED, new OnlyInheritedOperationsProcessor() { @Override public void process(Class cl) { System.out.println("Found class " + cl + " without operation"); Operation operation = UMLFactory.eINSTANCE.createOperation(); operation.setName("newOp"); operation.setClass_(cl); } }); DefaultActivationLifeCycle lifecycle = DefaultActivationLifeCycle.DEFAULT_NO_UPDATE_AND_DISAPPEAR; IMatcherFactory<OnlyInheritedOperationsMatcher> factory = OnlyInheritedOperationsMatcher.factory(); RuleSpecification spec = Rules.newSimpleMatcherRuleSpecification(factory, lifecycle, Sets.newHashSet(job));
Fire activations manually using a rule engine
The first option when using the EVM is creating a rule engine, which manages the set of activations, but will not fire the enabled activations. A rule engine has no context of its own, so the user can create one to use when firing activations. The rule specifications above are returned by the two getter methods, and the addRule method is used on the rule engine to instantiate the rules.
// create rule engine over query engine RuleEngine ruleEngine = RuleEngines.createViatraQueryRuleEngine(engine); // create context for execution Context context = Context.create(); // prepare rule specifications RuleSpecification createGeneralization = getCreateGeneralizationRule(); RuleSpecification createOperation = getCreateOperationRule(); // add rule specifications to engine ruleEngine.addRule(createGeneralization); ruleEngine.addRule(createOperation);
Once a rule specification is added to the rule engine, the existing activations of a given rule can be retrieved from the rule engine and can fire them manually. Alternatively, the next activation as selected by the conflict resolver (which is a simple hash set without ordering) can be retrieved and fired.
// check rule applicability Set<Activation> createClassesActivations = ruleEngine.getActivations(createGeneralization); if (!createClassesActivations.isEmpty()) { // fire activation of a given rule createClassesActivations.iterator().next().fire(context); } // check for any applicable rules while (!ruleEngine.getConflictingActivations().isEmpty()) { // fire next activation as long as possible ruleEngine.getNextActivation().fire(context); }
As long as the rule engine exists, it will keep on managing the activations of added rules. It is possible to remove a single rule, which will remove all its activations from the rule engine, and the rule engine can be disposed when not needed anymore.
// rules that are no longer needed can be removed ruleEngine.removeRule(createGeneralization); // rule engine manages the activations of the added rules until disposed ruleEngine.dispose();
Fire activations automatically with an execution schema
The second option for using the EVM is to create an execution schema that has a scheduler to fire activations after predefined events and an executor that specifies how to fire activations when scheduled. In the example, we use the feature in VIATRA Query Base that allows us to register a callback on model changes. An execution schema is created using a VIATRA Query engine and a scheduler factory, then the rules are added, in the same way as for the rule engine.
// use IQBase update callback for scheduling execution UpdateCompleteBasedSchedulerFactory schedulerFactory = Schedulers.getIQEngineSchedulerFactory(engine); // create execution schema over ViatraQueryEngine ExecutionSchema executionSchema = ExecutionSchemas.createViatraQueryExecutionSchema(engine, schedulerFactory); // prepare rule specifications RuleSpecification createGeneralization = getCreateGeneralizationRule(); RuleSpecification createOperation = getCreateOperationRule(); // add rule specifications to engine executionSchema.addRule(createGeneralization); executionSchema.addRule(createOperation);
In the example we simply modify the model by removing a generalization from a random class. When the model modification is handled by VIATRA Query, the callback notifies the scheduler, which starts the executor, which in turn will fire enabled activations as-long-as-possible. The scheduler in the example uses the model update listener of the engine (https://bugs.eclipse.org/bugs/show_bug.cgi?id=398744) to get callbacks on changes.
// execution schema waits for a scheduling to fire activations // we trigger this by removing one generalization at random SuperClassMatcher.factory().getMatcher(engine).forOneArbitraryMatch(new SuperClassProcessor() { @Override public void process(Class sub, Class sup) { sub.getGeneralizations().remove(0); } });
Similarly to the rule engine, it is possible to remove a rule from the execution schema or to dispose it when not needed any longer. The main difference between the rule engine and the execution schema is, that once a rule has been added, the activations that are enabled will be executed automatically every time the scheduler is notified. This allows us to implement components that can react to changes incrementally, without requiring additional scaffolding.
// rules that are no longer needed can be removed executionSchema.removeRule(createGeneralization); // execution schema manages and fires the activations of the added // rules until disposed executionSchema.dispose();
Impose ordering between activations of different rules
Activations that are enabled are in conflict with each other since firing any of them can cause the other activations to become disabled. The conflict set of a rule engine is the set of enabled activations of all rules, and users can define a conflict resolver that provides an ordering in the conflict set (https://bugs.eclipse.org/bugs/show_bug.cgi?id=403825).
Enabling log messages in EVM
Due to the event-driven nature of EVM, it is often difficult to debug your program, since the control flow will go through EVM internals and activation life-cycle is handled independently of activation firing. In order to see exactly what happens inside EVM, you can set the log level of the Log4J logger of rule engines to display DEBUG or even TRACE level messages. The log will include events, activation state changes, scheduling and executor events, firing and other useful information.
// just set the log level of the engine as needed ruleEngine.getLogger().setLevel(Level.DEBUG);
Usage scenarios
Both the data binding and validation frameworks of VIATRA use the EVM for handling events.
- Data binding: observable match result collections are created in the createRuleSpecification method of ObservableCollectionHelper (org.eclipse.viatra.databinding.runtime.collection package)
- Validation constraints are created in the constructor of ConstraintAdapter (org.eclipse.viatra.validation.runtime package)
Programming against the EVM API
The basic usage of the EVM is to react to match set changes easily.
Efficiently reacting to pattern match set changes
If you want to efficiently react to appearing, changing or disappearing matches, the EVM is a perfect choice. Just define a rule specification with the correct life-cycle and jobs and create an execution schema as described above.
- If you only want to react to appearance events: use DefaultActivationLifeCycle.DEFAULT_NO_UPDATE_AND_DISAPPEAR and a single APPEARED job.
- If you want to react to both appearance and disappearance: use DefaultActivationLifeCycle.DEFAULT_NO_UPDATE and two jobs, one with APPEARED and the other with DISAPPEARED state.
- If you want to react to changes of match parameters (e.g. an attribute value changes, but the match still exists): use DefaultActivationLifeCycle.DEFAULT and add an additional job with UPDATED state.
Roll your own event provider for EVM
The EVM core is independent of EMF and VIATRA Query, see https://bugs.eclipse.org/bugs/show_bug.cgi?id=406558 You can create your own event realm and use the EVM core concepts to execute event-driven rules. You can see a small example in http://git.eclipse.org/c/viatra/org.eclipse.viatra.git/tree/examples/evm-proto
Design decisions and code style
These guidelines are derived from the main decision to create a defensive framework to minimize the internal argument checks and idiot-proofing required.
User interaction with the framework
- Users interact with rules and activations through Façade classes:
- RuleEngine façade provides access to an Agenda and it's rule instances
- ExecutionSchema façade provides access to a Scheduler and through that to the Executor
- These Façade classes can be retrieved through the static methods of EventDrivenVM or by static create methods (for specific implementations)
- Any object that a user can access through the Façade must have only public methods that do not endanger their engine (e.g. Activation.fire(), but not Activation.setState())
- Any collection that a user can access through the Façade must be immutable to avoid modifications (e.g. getActivations)
- Any object that is provided by the user must be copied if it's later modification can cause internal problems (e.g. life cycle for rules)
Parameters, input checking and logging
- Method parameters cannot be null!
- This is checked by Preconditions.checkNotNull(ref, msg). Return a meaningful message on null.
- Use the this.field = checkNotNull(field, msg) form in constructors when possible.
- Define delegate methods where optional parameters are allowed.
- All logging is done through the rule base, use the debug level for detailed report messages and error or warning when encountering a real problem (e.g. ViatraQueryException)
Default implementations
There are a high number of notification mechanisms and event processing, that must have an interface for extendibility, a good default implementation and a clear way of overriding.
- In RuleInstance notification providers and listeners are created in prepareX methods.
- Default life cycles prepared with unmodifiable static instances.
- Update complete provider implementations (IQBase and EMF transaction).
- Scheduler implementations (update complete and timed).
- Job implementations (stateless with a single match processor, and recording for transactional model modification).
- Job implementations can specify their own error handling (https://bugs.eclipse.org/bugs/show_bug.cgi?id=404307)
Development ideas
Other future tasks
- systematic testing: both unit and component tests on API classes and default implementation
- remove log4j dependency
- add identifier to RuleSpecification (also allow finding rule by id in RuleEngine?)
- support main transformation scenarios
- examples:
- batch
- single job appear-disappear
- semi and fully automated execution
- ignore initial activations
- part of this will be done in the viatra.emf.runtime plugin
- examples:
- Completely remove generics from EVM core
- create rule specification builder:
- should this be specific to VIATRA Query or generic?
RuleSpecification.builder(eventSourceSpecification).putJob(job1).putJob(job2).putLifecycle(lifecycle).build() // build creates RuleSpecification Rules.builder(querySpecification).putAppear(appearProcessor).putDisappear(disappearProcessor).putLifecycle(lifecycle).build() Rules.builder(querySpecification).ofAppear(appearProcessor) // only appear job, lifecycle selected based on that unless specified with putLifecycle
- support retrieval of filtered activations of a rule with a partial match
- also make filters that are reusable between rule specifications
- integrate into upstream components
- query-based features
- ensure that commands stored by RecordingJobs are only kept in Context when necessary or for a short time (e.g. clear at start of scheduled execution?)
- event queueing is problematic (job execution during activation firing can change the state)