Difference between revisions of "Eclipse b3/proposals/Build File Syntax"

Revision as of 09:26, 6 October 2009

Introduction

This document contains ideas for an xtext based syntax describing build meta data. I have taken liberties with the syntax, you have to read it with a "do what I mean" attitude, as this proposal otherwise may be very difficult to handle for a parser.

This text also includes some explanations of the b3 models as this is needed to understand the proposed syntax.

The idea is that a b3 build file is used as input to b3 commands. Commands in b3 are in practice just two basic commands - either "import" (to populate the workspace), or "invoke" (to get the value (or side effect) of a build unit part (e.g. MyProduct#productZip). (Although, there may be other commands that can be issued that b3 handle, like querying, debugging, dumping, logging, etc. - but in principle, there is just "import" and "invoke").

It is possible to do both import and return a result at the same time. The only difference is that the "import" does not require specification of a particular part, and that the typical use of "invoke" is to perform it against what has already been imported to (or preexisted in) the workspace, thus not requiring a resolver specification other than the default (workspace and target platform).

Context

The b3 build file can be thought of as a configuration of the context in which commands to b3 execute. The b3 context contains:

• units (the build units in the context)
• resolutions (the resolution of each required capability)
• advice (the set of advice to apply in the context)
• requests (the input request(s), and requests originating from required capabilities)
• resolver (the network of resolvers to use to resolve requests/requirements).

Each request sets up a new context (it may inherit from the default context), and it is (optionally) configured with a build file.

Requests

Requests, as you have already seen are the original "import statements" and requests made by "required capabilities" in the resolution being performed (as part of an import or an invocation). The requests can be advised, but are otherwise not terribly interesting :). Note that it is possible ro request something in any namespace, but that the resolution will always resolve this into some build unit providing the capability (analogous to how p2 does this).

Resolver

A resolver resolves a request into a resolution. It can handle the entire process of connecting to a repository, reading the information, and translating it to the common build model (i.e. translate it into build units), or it can be a composition of a connector and a meta data translator. Resolvers can be composed into a network of resolvers to facilitate resolution from multiple repositories.

Resolutions

Resolutions are the result of resolving - they tie the resulting build units to the requirements that caused them to appear in the context. Interesting, advanced things can be done with this as shown in the advise section, but mainly this is a mechanism to tie everything together and enable the invocation of actions that traverse the resulting model.

Build File

The build file is a file with the suffix .b3 and has the following syntax:

build: import-statements specs ;

import-statements :
    |
    | import-statements import
    ;
import: "import" package-name ;

general-statements :
    | 
    | general-statements general-statement ;
    ;

general-statement :
    | resolver-statement;
    | advice-statement;
    | synchronized-statement;
    ;

Import Statements

Import statements makes it possible to use short form names in the rest of the build file. Names of resolvers, etc. should be fully qualified to reduce naming collisions. Imports reduces typing, and increases readability. The syntax should be the same as java package imports.

import org.eclipse.b3.resolvers.*; 

Example of import statement

Resolver Statement

A context has a resolver network consisting of resolver nodes. Two special nodes "first" and "best" are used to specify how the search is conducted - resolvers in a "first" list results in the first found resolution, and resolvers in a "best" list results in the resolution with the highest score (compared to the request).

A b3 build file makes it possible to specify the resolution network with a simple syntax.

The specified resolution node itself is a "first" type of node.

import org.eclipse.b3.resolvers.*; // import names of resolvers to reduce typing

resolver {
    TargetResolver;
    WorkspaceResolver;
    }

This specifies that an attempt is made to resolve against the target platform, and if that fails against the Workspace. No options are specified for the resolvers.

What if I want to resolver first from Target platform, but not for certain tool bundles that, if I have them in the workspace, I would like to get those instead of the installed. This could look something like this:

resolver {
    WorkspaceResolver {
        namePattern="org.myorg.mytool.*";
        };
    TargetResolver;
    WorkspaceResolver;
    }

Here is an example using best:

resolver {
    TargetPlatformResolver;
    WorkspaceResolver;
    Best {
        P2Resolver {
            location = "http://www.someplace.good";
            }
        SVNResolver {
            location = "svn+ssh://someplace.else";
            ...;
            };
        }
    }

There are several settable properties in the different resolvers. (Compare to Buckminster's Readers in the RMAP).

TODO: Short form - using URI schemes

Additional features (TBD): Here are some additional things I would like to be able to do:

• name top level resolution specifications
• reuse a named resolution in another resolution
• import resolutions from another build file, and reuse a named resolution in the imported build file
• use of properties

Possibly also:

• specify an import with a translator to interpret the file in a particular location

Advice

// advice sets values in the model // similar to CSS // An XPath is a query, it matches a set of nodes, where a value can be set // advice {

   xpath = value;
   ...
   xpath {
       rel-path = value;
       ...
   }

The xpath is relative to the context - which is analogous to the "document" in XML/HTML. At the root of the context there are:

• units (the build units in the context)
• resolutions (the resolution of each required capability)
• advice (the set of advice to apply in the context)
• requests (the input request(s), and requests originating from required capabilities)
• resolver (the network of resolvers to use to resolve requests/requirements).

It is unclear if it should be allowed to apply advice on all of these (i.e. advice on advice). The idea is to primarily advice on requests, and resolutions, and in some cases on units.

So here are some examples:

/requests[name=org.myorg.projx.*]/options/source = true;

All requests to org.myorg.projx.* should be requests for source

/requests[name=org.myorg.projx.*]/options/mutable = true;

All requests to org.myorg.projx.* should be requests for mutable

/requests[name=org.myorg.projx.*]/options {
    source = true;
    mutable = true;
    }

shorthand for the two pieces of advice above

/requests[name=org.myorg.projx.*, namespace="eclipse.feature"]/options/ {
    source = true;
    mutable = true;
    }

if I only need mutable source for the eclipse.features

Options include: source, mutable, branchTagPath, timestamp, revision, resolverFilter, filterGroups, overlayPath, includeParts, excludeParts, and prune.

Advice on resolutions

Here are some examples

// Place all of my organization's "core" components in the target platform // (the rest goes to workspace by default).

/resolutions[name=org.myorg.core.*] /options {

   materializer = org.eclipse.b3.targetPlatformMaterializer;
   location = /where/it/should/be/stored;
   conflictResolution = update;

}

It is possible to make very advanced advice:

/resolutions/[name=org.myorg.*]/subResolutions/resolution[namespace="org.myorg.annotations"]/options {

   materializer = org.eclipse.b3.workspaceMaterialzer;
   location = /where/they/are/stored;

}

This would store all build units that provide a capability in the namespace org.myorg.annotations in a particular location on disk and link them to the workspace.

Advice on units The advice on units is applied as units are resolved. This means that the advice has an effect during the contexts lifetime, but as soon as the operation is over, the live model will reflect the original meta data translation.

The same advice can be applied again in some other context if needed. Sometimes all that is required is to override some faulty meta data (that the user wants to correct) to make resolution possible. As an example, we want to override a components request for a particular required capability. Once the components has been resolved it will probably not build, the problem is corrected by the user and the unit checked in. The next time, the overriding advice is not needed.

/units[name="org.myorg.projx.broken"]/requiredCapabilities[name="org.myorg.x"]/versionRange = [1.0.0, 3.0.0];

Creation of complex types. Most of the time values are simply strings, or numbers, or there is a simple translation from string form to the datatype required for a settable attribute. Sometimes this is not possible, and a "new instance operator" is required. This could look something like

   path-expr = new RequiredCapability {
       name = ...;
       nameSpace = ...;
       ...
   };

Synthetic nodes It must be possible to address synthetic nodes in the document such as the "before first" and "after last" positions in a list. This is done by suffixing the xpath expression iwth ":synthetic node name"

e.g. /units[name=org.myorg.foo]/requiredCapabilities:first = new RequiredCapability { ... }.

Lists It must be possible to reference the list itself. Clear the list. Remove an entry in the list, add an element first or last, and before or after an element. (Have to look more at XPath to see how this is typically done to give examples).

UNITS As the advice mechanism is very powerful it is actually possible to create build units directly in the b3 build file. This is useful in some cases where a virtual/configuration unit is required to define the order of dependencies. In many cases, creating an additional bundle or feature that is only used for some special aspect of building/testing can be an overkill - so it is quite convenient to declare it directly in the b3 build file.

Apart from this, the build units are mostly the result of an import, and they may be advised (as described earlier).