Tycho/Ideas/Deferring Dependency Resolution

Tycho does a lot of things in the afterProjectsRead callback right at the beginning of the build. In that way, Tycho is different to normal Maven plug-ins/extensions. It has been suggested on the list, that Tycho defers the target platform and dependency resolution to a normal build phase of the respective module. This page elaborates how this could be implemented.

Implementation Idea

Build order

Currently, Tycho does a full dependency resolution for the build order. However, this is not absolutely necessary. It should be sufficient to only "scan for reactor project dependencies, mostly by matching Require-Bundle or Import-Package to the list of Bundle-SymbolicName or Export-Package" [1]. In the rare case of indirect dependencies between reactor modules via a remote artifact (as pointed out in [2]), the build order may not be correct. However this could be detected by the dependency resolution of a module (by checking that the resolved dependencies do not contain old versions of artifacts which will be built later in the reactor), and reported as error. The order then would need to be corrected through hints in the POM; it should even be enough to list the modules in the right order in the modules section.

It should also be possible to extend the build order computation to accomodate maven-bundle-plugin and Tycho packaging types in the same reactor: We only expect Tycho modules to reference maven-bundle-plugin artifacts (and not the other way round), so we could just always make Maven build maven-bundle-plugin modules first.

Target platform

Target file resolution can be bound to the lifecycle of a new packaging type, e.g. "eclipse-target". This would naturally prevent multiple resolutions of the same target file. Obviously, the build ordering needs to make sure that eclipse-target modules used in the target-platform-configuration of a module are built before that module. The resolution as normal module build would also ease extensions: the target file resolution could be done by other mojos, e.g. one which reused the PDE implementation and hence also supports the other, non-portable location types.

The difficult part: What to do if the eclipse-target module is not built locally but resolved from remote? It would be natural if the target resolution result would be the artifact of an eclipse-target module. We probably don't want to include all artifacts, so the storage format could be a simple metadata repository with the resolved metadata plus a composite artifact repository pointing to the artifact repositories listed in the target file. (This won't work for the non-portable location types, but IMHO it doesn't have to.)

The target platform for each module is computed early in the lifecycle of the module. It is aggregated according to the target-platform-configuration from resolved target files, repositories with layout p2 and POM dependencies. Also, all modules which have been built earlier in the reactor are part of the target platform.

The target platform should not automatically download all p2 artifacts, but only offer an interface method to obtain artifacts which downloads on demand. In this way, it is no longer necessary to create an "implicit target platform" – a subset of the content of "layout p2 repositories" filtered by the project dependencies (!) – which blurs the distinction between target platform and resolved dependencies. The conceptionally clean implementation is to treat the entire p2 repository content as target platform when the repository is specified via layout p2.

Dependency resolution

Dependency resolution would be implemented in a mojo and enabled for modules which need it. This is in particular the case for modules which are compiled (eclipse-plugin, eclipse-test-plugin), because for determining the compile class path, the artifacts need to be physically available – something that can't be done for the target platform (see above).

Dependency resolution should be done with the p2 planner, starting with the published metadata of the module. IIRC publishing neither requires the artifact to be packed nor to have binary content, so this can be done before compilation. The steps required before publishing the real metadata, like qualifier replacement and hardening of inclusions, can be done before dependency resolution without problems. In fact, 0.0.0 replacement in inclusions has to be done only based on the target platform; it must not take the resolution result into account (see bug 352081). However since publishing requires inclusion replacement which requires the target platform which requires the publisher results of other modules (see above), publishing and target platform computation have to be in the same phase.

While dependency resolution ensures that the artifact being built can be installed later, it could be made optional for a feature build (see enhancement request 353399).

A p2 planner needs to be used when an eclipse-repository is built with includeAllDependencies=true. Although this could be done through dependency resolution, doing this directly during p2 repository aggregation would allow certain extensions over what is possible today: dependency inclusion rules could be defined on a seed unit level, and it would be possible to combine content which can't be installed at once.

Motivation/Benefits

• Performance: Target platform resolution would only happen if necessary, i.e. if the normal lifecycle is executed. A mvn clean on a Tycho project would probably only be slightly slower than in non-Tycho projects.
• Understandable Log: If there is a target or dependency resolution problem, it would be reported in the module having the problem.
• Correctness: When the dependencies of a module are resolved, the dependencies in the local reactor would already be published. It would no longer be necessary to imitate things that happen later in the build during dependency resolution. Issues related to this are:
  • Replacement of 0.0.0 for plug-in inclusions in features (bug 352081). With eager resolution, this has to be done twice, once for resolution and once during the build.
  • Qualifier replacement (TYCHO-606): The root cause was that replaced qualifiers (with a leading 'v') were compared to literal "qualifier" strings.
• Extensibility: It would be significantly easier to extend/customize Tycho – extenders could combine Tycho goals with their own goals in new packaging types. (Open question: how to make build order computation extensible?) Customization in POMs will be possible but limited as long as it is not (yet) possible squeeze in custom goals between lifecycle goals.