Equinox/p2/Proposals/Query Management and Optimization

While p2 was designed to support a number of different repositories, there are a few areas of p2 that could be enhanced to improve the performance of database backed repositories. This (working draft) proposal will help identify these areas and present concrete suggestions on how to address these issues.

There are currently six (6) areas that we have identified that can be improved:

1. Define standard queries as "non-opaque" meaning repository implementors could supply custom implementations using SQL or some other query language
2. Remove the complexities surrounding collectors. Many collectors act as queries, hampering the repositories ability to create custom implementations
3. Add "lazy" or "asynchronous" query results
4. Query result life cycle. Callers of a query will need to specify when / how the results are available, and what happens if the query gets restarted, or enhanced
5. Provide query result Status (likely MultiStatus)
6. Make query results queryable. In many instances, it would be convenient to re-query a set of query results. This may have performance implications with respect to the non-opaque queries.

Non Opaque Queries

To help repository implementors craft custom queries to represent many of the "standard" p2 queries, a set of non-opaque (i.e. transparent) queries should be provided. These queries should have well documented semantics and allow query implementors to access the properties of the query without explicitly depending on the Query itself.

To support alternative implementations of Queries, we propose adding the following two methods to Query.java:

/**
 * Indicates whether or not this Query is Transparent.  The properties of Transparent queries can be
 * accessed via the <link>getProperty</link> method.  Each transparent query should provide a 
 * get<PropertyName>() method for each parameter. In addition to this, each property should be describe as a 
 * constant at the top of the class.  For example, the CapabilityQuery should define:
 * 
 * public static final String REQUIRED_CAPABILITIES = "RequiredCapabilities";
 *
 * public IRequiredCapability[] getRequiredCapabilities();
 *
 * This query should also override isTransparent to return "true".
 *
 */
public boolean isTransparent {
    return false;
}
 
/**
 * Returns a particular property of a given Query.  
 */
public Object getProperty(String property) {
  // Reflectively look up the property and call the getter to get it
  // If it fails, return null;
}

Implementors of IQueryable can use these methods to construct alternative mechanisms of querying their data. For example, a DB backed IQueryable may use these methods to construct an SQL statement.

class MyRepository implements IQueryable {
  public Collector query(Query query, Collector collector, IProgressMonitor monitor) {
    if (query.isTransparent() && query.getClass().getName().equals("CapabilityQuery") {
        Object o = query.getProperty("RequiredCapabilities");
        if ( o != null ) {
           IRequiredCapability[] capabilities = (IRequiredCapability[]);
           SQLStatement statement = constructCapabilityQuery(capabilities);
           executeSQL(statement);
           return results;
        }
    }
    return query.perform(getIterator, collector);
  }
}