Difference between revisions of "HGraph"

Revision as of 00:17, 15 June 2010

This page describes HGraph, a new component and API being discussed for Higgins 2.0.

Motivation

The initial motivation for HGraph was to address a number of limitations of IdAS 1.1 related to describing and managing metadata.

Higgins-based applications have a requirement to be able to conveniently describe and manage metadata about regular entities. Examples include the need to cleanly separate provenance metadata entities from the base entities, or the need to be able to associate a set of attributes with a complex-valued (link-valued) attribute.

Imagine we have two contexts both of which make statements about entity e1. In the left-hand context, for which the RMV is the authority, we have the statement (e1 eye-color "blue") whereas in the right-hand context, for which the State Department is authoritative we have the statement (e1 eye-color "green"). These two authorities disagree about e1's eye color.

In IdAS this would be represented as follows:

Where c1 and c2 are the entities that represent the contexts themselves.

IdAS 1.1 might be said to be context-centric. What we mean by this is that whereas the authoritative context for a given entityId (aka a UDI) can be discovered dynamically, there is no easy and fast way to assemble all of the other (non-authoritative) contexts that may also make statements about a given entity (of the same entityId). In practice this is sufficiently awkward that the general IdAS idiom is to simply not do so, and to only have a single context for information about an entity. In practice a given entityId typically only occurs in a single context--its authoritative context.

With this design the IdAS client code is responsive for keeping track of all of the contexts that contain a given entityId and essentially merging these representations together. Most of the client code will simply want to know the value of some attribute of e1 and doesn't care which context made the statement. In this case the code would want to know the value(s) of eye-color of e1. It simply wants the answer "blue" and "green". And to have acceptable performance the client code would have to maintain a list of all of the contexts that make statements about e1.

Note that due to the open world aassumption neither the consuming code nor IdAS can every be guaranteed to know about all of the contexts that exist on the net that make statements about e1.

Design

HGraph is a new component that layers over IdAS and provides an entity-centric view of the data managed by IdAS below. It includes an internal write-through cache that implements the SAIL API. HGraph will use one of the existing open source SAIL storage implementations.

Benefits

HGraph was initially conceived in order to make it easier to manage multi-contextual entities. But it has the following additional benefits:

Cache/index for IdAS

• HGraph provides a write-through cache for data managed by any and all context providers plugged into IdAS. This provides:
• Performance boost for read operations from context providers that access remote backing stores
• Simplifies context provider development: context providers no longer have to implement internal data caches.

SAIL API

• HGraph's cache is an implementation of the openrdf.org SAIL RDF quad store API. This provides the following advantages.
• Provides a new extension point: any SAIL-compatible quad store implmentation can be used. In addition SAIL implementations can be stacked allowing SAIL-compatible inference engines to be incorporated--thus adding inferencing capabilities to Higgins
• The SAIL API provides a new standards-based API to access IdAS data.
• The SAIL API can be accessed by SPARQL engines--thus providing the Higgins PDS with support for this additional protocol