User Interviews concerning domain administration module

The first interview partner works in the driving safety department (MDM Fahrsicherheit, FaSis). He’s concerned with the technical development and testing of integral vehicle safety and drive assist systems. In the department, in 2008 one of the first to introduce the OpenMDM application, the application is used to plan tests, to save, archive and recover test and measurement data.

The interviewee main task is to assure that the OpenMDM application provides all necessary features the clerks in his department need for accomplishing their work. He gathers the requirements, checks them against the existing components in the 'toolbox' and eventually chooses and integrates the useful components into the running application. If no such component exists he files a request to the component development department.

The department created a catalog of drive maneuvers, each one described in a separate measurement step template. Thus complex sequences of drive scenarios can be composed. Every new test vehicle in the car pool has to accomplish a certain set of those scenarios. Thereby every step is specified in great detail such that test planners can describe and order tests within narrow limits. A problem here is that for each variant a new, similar named template had been created which led to a cluttered interface.

Besides this, endurance tests (german: Dauerlauf) are ordered by the FaSis test planners. This is a rather opposite approach: units under test are sparsely specified and sent to a test area where they collect during operation as much sensor data as possible. Upon return the data is imported into the ASAM ODS system and analysed.

In summary it can be said that the granularity of the measurement step template specification depends on the focus of the measurement. In contrast to the multitude of measurement step templates only few basic catalog components were defined.

About once a week value lists are updated providing new selection options during test planning. (In contrast to this units or physical dimension have not been edited at all yet.)

Sometimes the domain administrator has to recall accidentally archived templates.

Furthermore measurement parameters are added from time to time (when new types of sensor devices are used which collect new sorts of data).

Another common task for users of the domain administrator module is to add attributes to a component and then to create new versions of all the templates in which the component is referenced directly or indirectly. Currently this can become quite tedious especially when the hierarchy of templates enlarges. Furthermore it is prone to mistakes when done manually. Thus the application provides the possibility to create new versions for either all or none of the referencing templates. But in constellations with 100 and more measurement step templates per test template generating new versions for all templates would cause the application to halt. Besides the navigator tree collapses and the user has to recover the node he had been editing. Furthermore the new versions must still be edited manually because the references to the new versions must be set. Afterwards the template must be declared valid before they can be used in higher level templates. If a mistake (e.g. a typo) is spotted half way up the tree, the user has no choice but to start all over again. This produces another set of new versions - some declared valid, some not - leaving the user puzzled which ones to choose. Novice users would be completely lost.

The interviewee wished some better support here: versioning, validity declaration and combination to a new test template shall be realized within a few steps, guided by the application when for example adding an attribute to a referenced component template. Furthermore it should be possible for the user to choose which intermediate templates should be versioned. The interview partner estimated that this can only be realized with a top-down approach: select a test template which shall feature an updated component template and thus for which a new version shall be generated. If instead the component template would be adapted first, then the user wouldn’t know which of the large amount of intermediate templates would need to be versioned and updated.

Because the maintenance is that difficult there is currently a tendency in the department to create one large template which allows all configurations. This way the test planner has all freedom and removes or disables parts he doesn’t need i.e. he doesn’t require measurement results for. This shifts the responsibilities from the domain admin to the test planner role. But the department employees see this as an advantage: in case changes have to be applied any clerk with test planner rights can take care. Otherwise this task would rely on only one or two people possessing the domain admin role rights. A configurable rule-set plugin checks before a test can be commissioned that certain safety conditions are met e.g. to avoid drive maneuver which would certainly harm the test drivers.

The more template versions are added the more confusing becomes the display in the navigation tree exacerbating to find a certain node. A better means to structure and maintain the large amount of templates would be appreciated.

Another difficulty while defining templates comes with the three flags 'Optional', 'Active by default' and 'Variable per series of measurement'.

One problem lies in the fact that invalid combinations like not 'optional' and not 'active by default' at the same time are possible. This configuration would result in a mandatory but invisible section during test planning - the clerk would not know that there are configuration details which need to be specified. This is a source of frustration. Furthermore all component templates referenced in measurement step templates need to commonly have the flag unset if divergent values should be prohibited.

For the calculation of the parameters which are constant throughout the whole test only the first measurement step template is currently considered. A more thorough validation is desired here.

The interview partner stated, that enumerations are rarely administrated, if ever, but tags are - at least in the FaSis department - used for starting complex workflows. A polling mechanism checks recurrently for elements tagged in a specific way and triggers workflows accordingly.

In the following some wishes of the interview partner are listed:

Even though the above wishes were expressed, the interviewee would recommend the application to others. It was also important for him that the rather technical attribute datatypes e.g. DT_STRING or DS_LONG remain visible to the user and are not replaced by terms like Text or Numbers. The department employees are used to them and more important the sensor devices collecting the data use these datatypes. This way easy matching can be accomplished and the communication with sensoring software providers is simplified.

The interview partner got into contact with the OpenMDM application while working in the department whose concern is the development of chassis and high voltage batteries. Furthermore he was responsible for the integration client, an application combining and presenting (almost) all available OpenMDM components. The integration client is the starting point for any system configurator responsible for tailoring the OpenMDM client according to the users needs (see preload.ini file). Concretely he would throw out all components with irrelevant features, a job for which a profound knowledge of the component dependencies is needed.

Thus the interviewee breathed the wish that dependencies of each mdm component - which are currently rather obscure - would be automatically resolved.

Recurring tasks in domain admin module were

In contrast to this whole new component template types were created only now and then e.g. when a new workbench became available (normally each workbench has a different purpose and delivers different measurement parameters).

For most of the tasks deep understanding of the application is needed - also this interviewee estimated that a novice would be totally lost without profound training or help by experienced co-workers.

It has to be noted that in the interviewee’s department no tests are planned with the OpenMDM application. A third party application is used for that. Therefore no measurement step templates are defined in the domain admin module. Nonetheless the element hierarchy (test, measurement step, measurements) is generated during import of the measurement results.

Some features which would improve the OpenMDM application significantly were requested by the interview partner:

Currently the versioning paradigm is only realized for templates, not for components. As a consequence when updating or removing an attribute of a component then a copy has to be generated and adapted. Curiously when adding a new attribute no such copy is needed.

Translations can be defined for both components and templates. It couldn’t be clarified during the interview if the translations for the components are automatically applied on the corresponding templates and if the template translations can be overridden.

The interview partner explained that all data delivered by the workbenches is transformed into the common ATFX data format prior to import into the ASAM ODS system. Inside the external system section the mapping of ATFX data to ASAM ODS attributes is managed.

A backup of the system data can be realized with the ATFX export feature. But it is currently unusable for transferring data e.g. from the test system to the production system as extracted ids won’t match in the target system and won’t be adapted during import. As a consequence all changes made on any of the three systems development, test and production must be mirrored immediately on the others. As those updates are executed manually they are prone to error. In the future domain admins won’t have any longer access to the database or other system internals via the ASAMCommander. Instead the domain admins will have to describe the changes to be made (which packages to be replaced, which scripts to run etc.). Thus a feature to run update macros would be appreciated.

The interviewee sees much room for improvement but nonetheless recommends the OpenMDM application as he does not know of any comparable alternative software.

The third interviewee is concerned with the configuration of OpenMDM systems (client and server setup) and particularly tailors clients out of components according to business needs. With regards to the domain admin module he shapes component and test templates for a number of departments i.e. he provides expert support as part of several maintenance contracts.

The interview partner uses the domain admin module daily, at least once per week for every department assigned to him. During the initial setup of the client for a department the workload to be done in the domain admin module is higher.

A regular task is to update the mapping of external system parameters to internal mdm parameters. The goal is to connect third party systems to the OpenMDM application. A parameter analysis feature would be appreciated which would - based on an ATFX file or a web service interface description - collect and import all parameters automatically. Currently the system admin has to enter all external parameters manually - a very time consuming task. Furthermore it would be valuable to have a validation check at hand listing all unmapped internal and external parameters as well as parameters mapped multiple times. It has to be noted though that huge changes and adaptions are done only during initial setup. There is also no version control of the mapping lists i.e. deletion or manipulation of parameter settings cannot be traced back.

As there can only one action be defined to be executed when the test planner changes the value of an attribute, there has been created a workaround: a proxy plugin which can be configured in the preload.ini file to run multiple different actions e.g. compiling the user input against a regex pattern or writing a value to a dependent input field.

Some configuration details e.g. default templates, data locations, webservice urls or user credentials are defined within the system parameter editor - a separated module. But some of the system parameters only affect the domain admin module and therefore might be shifted to the latter module in the OpenMDM 5 application.

Some wishes concerning the further development of the application were:

Finally the interviewee stated that he also defines tags to be used for triggering business workflows. But they are also used for marking tests or measurement results so that they can be easily retrieved.

It can be summarized that even though the OpenMDM application is a powerful tool for planning, reviewing and administrating tests it also has its weaknesses. The major pain points are