Difference between revisions of "HGraph"

Revision as of 23:45, 15 June 2010

This page describes HGraph, a new component and API being discussed for Higgins 2.0.

Motivation

The initial motivation for HGraph was to address a number of limitations of IdAS related to describing and managing metadata. As will be described, rather than changing IdAS at all, this page describes a new component called HGraph that, when layered over IdAS, not only addresses these limitations but affords some additional capabilities.

Some Higgins-based applications need to able to conveniently describe and manage metadata about regular entities. Examples include the need to cleanly separate provenance metadata entities from the base entities, or the need to be able to associate a set of attributes with a complex-valued (link-valued) attribute. While it is possible, it is far from convenient.

The cleanest way to handle metadata is to first place the base entities in a new context and then to add the desired metadata attributes to the entity representing the new context.

Here is an example of this approach diagrammed below. We have two contexts (represented by c1 and c2) both of which make statements about entity e1. In the left-hand context, for which the RMV is the authority, makes the statement (e1 eye-color "blue") whereas in the right-hand context, for which the State Department is authoritative, makes the statement (e1 eye-color "green"). As you can see, these two authorities disagree about e1's eye color.

The problem lies in the fact that IdAS might be said to be context-centric. What we mean by this is that whereas the authoritative context for a given entityId can be discovered dynamically, there is no convenient and fast way to assemble all of the other (non-authoritative) contexts that may also make statements about a given entity (of the same entityId). In practice this is sufficiently awkward that the general IdAS idiom is to simply not do so, and to only have a single context for information about an entity. In practice a given entityId typically only occurs in a single context--its authoritative context.

The IdAS API consumer has the burden of keeping track of all of the contexts that contain a given entityId and merging together the attributes of each. The reason for the merge is that most IdAS consumers simply want to know the value of some attribute of e1 and doesn't care which context made the statement. In the previous example most IdAS consumers simply want to know the the eye-color of e1. In other words it simply wants to get the values "blue" and "green".

To complicate matters further, in order for the IdAS consumer to have acceptable performance it would need to maintain its own cache of the list of all of the contexts that make statements about e1.

Note that due to the open world assumption neither the consuming code nor IdAS can every be guaranteed to know about all of the contexts that exist on the net that make statements about e1.

Design

HGraph is a new component that layers over IdAS and provides an entity-centric view of the data managed by IdAS below.

The HGraph component exposes compound entities (ICEntity instances) as opposed to the regular IdAS IEntity objects. Given an entityId, e1, HGraph will return a compound entity. A compound entity has 1..N sets of attribute/values where set is a cached copy of the attributes of e1 in each of N contexts:

Architecture

It includes an internal write-through cache that implements the SAIL API. HGraph will use one of the existing open source SAIL storage implementations.

Benefits

HGraph was initially conceived in order to make it easier to manage multi-contextual entities. But it has the following additional benefits:

Cache/index for IdAS

• HGraph provides a write-through cache for data managed by any and all context providers plugged into IdAS. This provides:
• Performance boost for read operations from context providers that access remote backing stores
• Simplifies context provider development: context providers no longer have to implement internal data caches.

SAIL API

• HGraph's cache is an implementation of the openrdf.org SAIL RDF quad store API. This provides the following advantages.
• Provides a new extension point: any SAIL-compatible quad store implmentation can be used. In addition SAIL implementations can be stacked allowing SAIL-compatible inference engines to be incorporated--thus adding inferencing capabilities to Higgins
• The SAIL API provides a new standards-based API to access IdAS data.
• The SAIL API can be accessed by SPARQL engines--thus providing the Higgins PDS with support for this additional protocol