Jump to: navigation, search

Difference between revisions of "Minerva"

(Repositories (p2))
Line 232: Line 232:
 
p2 Repositories simply reference the parent pom and have a packaging attribute of '''eclipse-repository'''.  The project must also include a ''category.xml'' if you want to publish features or a ''myProduct.product'' if you want to publish a product.
 
p2 Repositories simply reference the parent pom and have a packaging attribute of '''eclipse-repository'''.  The project must also include a ''category.xml'' if you want to publish features or a ''myProduct.product'' if you want to publish a product.
  
I didn't get a chance to adapt my orion experiment to minerva yet, but here is the [http://git.eclipse.org/c/e4/org.eclipse.orion.server.git/tree/releng/org.eclipse.orion.server.repository/pom.xml?h=mavenExperiment pom.xml] and [http://git.eclipse.org/c/e4/org.eclipse.orion.server.git/tree/releng/org.eclipse.orion.server.repository/category.xml?h=mavenExperiment category.xml].
+
I didn't get a chance to adapt my orion experiment to minerva yet, but here is the [http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/tree/releng/org.eclipse.orion.server.repository/pom.xml?h=mavenExperiment pom.xml] and [http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/tree/releng/org.eclipse.orion.server.repository/category.xml?h=mavenExperiment category.xml].
  
 
<source lang="xml">
 
<source lang="xml">

Revision as of 10:27, 17 February 2012

Minerva is the Maven version of Athena.

At the moment, the code is hosted at GitHub but the plan is to have it hosted at eclipse.org...

git clone git://github.com/caniszczyk/minerva.git

Building

Note: Please have the latest version of Maven 3 installed.

To build the project from the command line after checking the code out, simply run

mvn -Dskip-ui-tests=true clean install

In Maven, the parent pom.xml serves as the central point on adding things to the build. It's also generally the most complicated piece of the build as it contains information that is shared by children pom.xml files. The first part of the parent pom.xml we'll look at contains identifying information:

...
<groupId>org.aniszczyk.minerva</groupId>
<artifactId>minerva-parent</artifactId>
<version>1.0.0-SNAPSHOT</version>
<packaging>pom</packaging>
 
<name>Minvera Parent</name>
...

The second part contains profile information and where to get dependencies.

  <properties>
    <tycho-version>0.10.0</tycho-version>
    <platform-version-name>helios</platform-version-name>
    <eclipse-site>http://download.eclipse.org/releases/${platform-version-name}</eclipse-site>
    <wikitext-site>http://download.eclipse.org/tools/mylyn/update/weekly</wikitext-site>
    <swtbot-site>http://download.eclipse.org/technology/swtbot/${platform-version-name}/dev-build/update-site</swtbot-site>
  </properties>
 
  <profiles>
    <profile>
      <id>platform-helios</id>
      <activation>
        <property>
          <name>platform-version-name</name>
          <value>helios</value>
        </property>
      </activation>
      <properties>
        <eclipse-site>http://download.eclipse.org/releases/helios</eclipse-site>
        <platform-version>[3.6,3.7)</platform-version>
        <swtbot-site>http://download.eclipse.org/technology/swtbot/helios/dev-build/update-site</swtbot-site>
      </properties>
    </profile>
    <profile>
      <id>platform-indigo</id>
      <activation>
        <property>
          <name>platform-version-name</name>
          <value>indigo</value>
        </property>
      </activation>
      <properties>
        <eclipse-site>http://download.eclipse.org/releases/indigo</eclipse-site>
        <platform-version>[3.7,3.8)</platform-version>
        <swtbot-site>http://download.eclipse.org/technology/swtbot/indigo/dev-build/update-site</swtbot-site>
      </properties>
    </profile>
  </profiles>
...
  <repositories>
    <repository>
      <id>helios</id>
      <layout>p2</layout>
      <url>${eclipse-site}</url>
    </repository>
    <repository>
      <id>swtbot</id>
      <layout>p2</layout>
      <url>${swtbot-site}</url>
    </repository>
    <repository>
      <id>wikitext</id>
      <layout>p2</layout>
      <url>${wikitext-site}</url>
    </repository>
  </repositories>

The third part lists the modules (e.g., features, plug-ins) that are part of the build:

  <modules>
    <module>org.aniszczyk.minerva.core</module>
    <module>org.aniszczyk.minerva.ui</module>
 
    <module>org.aniszczyk.minerva-feature</module>
    <module>org.aniszczyk.minerva.source-feature</module>
 
    <module>org.aniszczyk.minerva-updatesite</module>
 
    <module>org.aniszczyk.minerva.tests.core</module>
    <module>org.aniszczyk.minerva.tests.ui</module>
   </modules>

You can also configure your qualifier in your parent pom.xml, but this might need >0.11.0 to work correctly

   <plugin>
      <groupId>org.eclipse.tycho</groupId>
      <artifactId>tycho-packaging-plugin</artifactId>
      <version>${tycho-version}</version>
      <configuration>
         <format>'v'yyyyMMdd-HHmm</format>
      </configuration>
   </plugin>


Features

Features simply reference the parent pom and have a packaging attribute of eclipse-feature.

Here's a pom.xml snippet from the minerva feature:

  <parent>
    <groupId>org.aniszczyk.minerva</groupId>
    <artifactId>minerva-parent</artifactId>
    <version>1.0.0-SNAPSHOT</version>
  </parent>
 
  <artifactId>org.aniszczyk.minerva-feature</artifactId>
  <packaging>eclipse-feature</packaging>
 
  <name>Minerva Feature (Incubation)</name>

Plug-ins

Plug-ins simply reference the parent pom and have a packaging attribute of eclipse-plugin.

Here's a pom.xml snippet from the minerva core plug-in:

  <parent>
    <groupId>org.aniszczyk.minerva</groupId>
    <artifactId>minerva-parent</artifactId>
    <version>1.0.0-SNAPSHOT</version>
  </parent>
 
  <artifactId>org.aniszczyk.minerva.core</artifactId>
  <packaging>eclipse-plugin</packaging>
 
  <name>Minerva Core Plug-in</name>

Source

To package source with Tycho, you need first need a feature that contains the source plug-ins.

In Minerva, this is in org.aniszczyk.minerva.source-feature. The pom.xml is just like any feature with Tycho:

  <parent>
    <groupId>org.aniszczyk.minerva</groupId>
    <artifactId>minerva-parent</artifactId>
    <version>1.0.0-SNAPSHOT</version>
  </parent>
 
  <artifactId>org.aniszczyk.minerva.source-feature</artifactId>
  <packaging>eclipse-feature</packaging>
 
  <name>Minerva Sources Feature</name>

The important part is that your feature.xml references the plug-ins you want source for:

   <plugin
         id="org.aniszczyk.minerva.core.source"
         download-size="0"
         install-size="0"
         version="0.0.0"
         unpack="false"/>
 
   <plugin
         id="org.aniszczyk.minerva.ui.source"
         download-size="0"
         install-size="0"
         version="0.0.0"
         unpack="false"/>

Then in any plug-ins that you reference, you need to ensure their respective pom.xml contains a reference to the tycho-source-plugin maven plug-in.

 <build>
    <plugins>
      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>tycho-source-plugin</artifactId>
      </plugin>
     ...
    </plugins>
  </build>

Repositories (Update Sites)

Update Site repositories simply reference the parent pom and have a packaging attribute of eclipse-update-site.

Here's a pom.xml snippet from the minerva site:

  <parent>
    <groupId>org.aniszczyk.minerva</groupId>
    <artifactId>minerva-parent</artifactId>
    <version>1.0.0-SNAPSHOT</version>
  </parent>
 
  <artifactId>org.aniszczyk.minerva-updatesite</artifactId>
  <packaging>eclipse-update-site</packaging>

Repositories (p2)

p2 Repositories simply reference the parent pom and have a packaging attribute of eclipse-repository. The project must also include a category.xml if you want to publish features or a myProduct.product if you want to publish a product.

I didn't get a chance to adapt my orion experiment to minerva yet, but here is the pom.xml and category.xml.

  <parent>
        <relativePath>../org.eclipse.orion.server.parent/pom.xml</relativePath>
        <groupId>org.eclipse.orion</groupId>
        <artifactId>org.eclipse.orion.server.parent</artifactId>
        <version>0.2.0-SNAPSHOT</version>
  </parent>
  <groupId>org.eclipse.orion</groupId>
  <artifactId>org.eclipse.orion.server.repository</artifactId>
  <packaging>eclipse-repository</packaging>

The category.xml must include the features to be built into the repo:

   ...
   <feature url="features/org.eclipse.orion.base.feature_0.2.0.qualifier.jar" 
         id="org.eclipse.orion.base.feature" version="0.2.0.qualifier">
      <category name="org.eclipse.orion.server.category"/>
   </feature>
   <category-def name="org.eclipse.orion.server.category" label="Orion Server Category">
      <description>Orion Server Category</description>
   </category-def>

The category.xml can also include p2 query syntax:

<category-def name="all" label="All Repository Bundles"/>
<iu>
<category name="all"/>
<query><expression type="match">providedCapabilities.exists(p | p.namespace == 'osgi.bundle')</expression></query>
</iu>

Tests

Tests are an important part of any software project. Tycho both supports headless and UI tests (with SWTBot).

Headless Tests

Headless tests simply reference the parent pom and have a packaging attribute of eclipse-test-plugin.

Here's a pom.xml snippet from the minerva core tests:

  <parent>
    <groupId>org.aniszczyk.minerva</groupId>
    <artifactId>minerva-parent</artifactId>
    <version>1.0.0-SNAPSHOT</version>
  </parent>
 
  <artifactId>org.aniszczyk.minerva.core.tests</artifactId>
  <packaging>eclipse-test-plugin</packaging>
 
  <name>Minerva Core Test Plug-in</name>
 
  <build>
    <plugins>
      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>tycho-surefire-plugin</artifactId>
        <version>${tycho-version}</version>
        <configuration>
          <excludes>
            <!-- test mojo matches TestProject be default and treats it as PojoTest -->
            <exclude>**/Test*.class</exclude>
          </excludes>
          <useUIHarness>false</useUIHarness>
          <useUIThread>false</useUIThread>
        </configuration>
      </plugin>
     </plugins>
    </build>

Tycho does all the hard work and finds the tests to run as part of the build.

UI Tests

UI tests simply reference the parent pom and have a packaging attribute of eclipse-test-plugin.

Here's a pom.xml snippet from the minerva core tests:

  <parent>
    <groupId>org.aniszczyk.minerva</groupId>
    <artifactId>minerva-parent</artifactId>
    <version>1.0.0-SNAPSHOT</version>
  </parent>
 
  <artifactId>org.aniszczyk.minerva.ui.tests</artifactId>
  <packaging>eclipse-test-plugin</packaging>
 
  <name>Minerva UI Test Plug-in (Incubation)</name>
 
  <properties>
    <local-p2-site>file:/${basedir}/../org.aniszczyk.minerva-updatesite/target/site</local-p2-site>
    <ui.test.vmargs>-Xmx512m -XX:MaxPermSize=256m</ui.test.vmargs>
  </properties>
 
  <repositories>
    <repository>
      <id>local-p2</id>
      <layout>p2</layout>
      <url>${local-p2-site}</url>
    </repository>
  </repositories>
 
  <profiles>
    <profile>
      <id>skip-ui-tests</id>
      <activation>
        <property>
          <name>skip-ui-tests</name>
        </property>
      </activation>
      <properties>
        <maven.test.skip>true</maven.test.skip>
      </properties>
    </profile>
  </profiles>
 
  <build>
    <plugins>
      <plugin>
        <groupId>org.sonatype.tycho</groupId>
        <artifactId>tycho-surefire-plugin</artifactId>
        <version>${tycho-version}</version>
        <configuration>
          <testSuite>org.aniszczyk.minerva.tests.ui</testSuite>
          <testClass>org.aniszczyk.minerva.tests.ui.AllTests</testClass>
          <useUIHarness>true</useUIHarness>
          <!-- Set UIThread to true for UI Tests that do not use SWTBot -->
          <useUIThread>false</useUIThread>
          <product>org.eclipse.sdk.ide</product>
          <argLine>${ui.test.vmargs}</argLine>
          <application>org.eclipse.ui.ide.workbench</application>
          <dependencies>
            <dependency>
              <type>p2-installable-unit</type>
              <artifactId>org.eclipse.pde.feature.group</artifactId>
              <version>${platform-version}</version>
            </dependency>
            <dependency>
              <type>p2-installable-unit</type>
              <artifactId>org.aniszczyk.minerva.feature.group</artifactId>
              <version>[1.0.0,2.0.0)</version>
            </dependency>
            <dependency>
              <type>p2-installable-unit</type>
              <artifactId>org.eclipse.cvs.feature.group</artifactId>
              <version>[1.1.2,2.0.0)</version>
            </dependency>
           </dependencies>
        </configuration>
      </plugin>
     </plugins>
    </build>

In this case, we use the built in support in Tycho to launch an Eclipse to test the UI.

We also ensure any features are installed that we need (like our minerva feature).

Under the covers, the UI tests use SWTBot via @RunWith(SWTBotJunit4ClassRunner.class)

Documentation

WikiText

We will use Mylyn Wikitext to generate our documentation from the Eclipse wiki (Eclipsepedia)

TODO

Javadoc

TODO: GMF Tooling Example

Static Code Analysis

Manually

You can use code coverage by adding the proper maven plug-in dependencies in the parent pom.xml.

 <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>findbugs-maven-plugin</artifactId>
          <version>2.3.2-SNAPSHOT</version>
          <configuration>
            <findbugsXmlOutput>true</findbugsXmlOutput>
            <failOnError>false</failOnError>
          </configuration>
          <executions>
            <execution>
              <goals>
                <goal>check</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-pmd-plugin</artifactId>
          <version>2.5</version>
          <configuration>
            <sourceEncoding>utf-8</sourceEncoding>
            <minimumTokens>100</minimumTokens>
            <targetJdk>1.5</targetJdk>
            <format>xml</format>
            <failOnViolation>false</failOnViolation>
          </configuration>
          <executions>
            <execution>
              <goals>
                <goal>cpd-check</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </pluginManagement>

In this example, we're using the findbugs-maven-plugin and maven-pmd-plugin maven plug-ins.

Then, to enable code coverage for each plug-in you have to update their respective pom.xml.

 <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>findbugs-maven-plugin</artifactId>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-pmd-plugin</artifactId>
      </plugin>
    </plugins>
  </build>

Using Sonar

If you have a Sonar instance available, You simply have to add these properties to your parent pom.xml

<properties>
    <!-- ... other properties ... -->
    <sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
</properties>

and then you can simply run a ***mvn sonar:sonar*** after your build to push reports to Sonar.

Code Coverage with Jacoco

Jacoco is the future of the Emma and EclEmma projects and allows to produce code coverage simply by setting a java.agent VM argument and without need to provide an instrumentated version of compile code. Then it does not require additional work, and a Jacoco Maven plugin is available to make configuration easier.

In the parent pom.xml of your tests, set the following stuff (see file in tree):

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>org.eclipse.gmf-tooling</groupId>
	<artifactId>tests</artifactId>
	<packaging>pom</packaging>
	<version>2.4.1-SNAPSHOT</version>
 
	<parent>
		<groupId>org.eclipse.gmf-tooling</groupId>
		<artifactId>parent</artifactId>
		<version>2.4.1-SNAPSHOT</version>
		<relativePath>../</relativePath>
	</parent>
 
	<modules>
		<module>org.eclipse.gmf.tests</module>
		<module>org.eclipse.gmf.tests.lite</module>
		<module>org.eclipse.gmf.tests.xpand</module>
		<module>org.eclipse.gmf.tests.xpand.migration</module>
	</modules>
 
	<profiles>
		<profile>
			<id>jacoco</id>
			<activation>
				<activeByDefault>true</activeByDefault>
			</activation>
			<properties>
				<!-- Properties to enable jacoco code coverage analysis in Sonar -->
				<sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
				<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
				<!-- When running in children tests/* modules, all reports will be in tests/target/jacoco.exec -->
				<sonar.jacoco.itReportPath>../target/jacoco.exec</sonar.jacoco.itReportPath>
			</properties>
			<build>
				<plugins>
					<!-- Enabling use of jacoco -->
					<plugin>
						<groupId>org.jacoco</groupId>
						<artifactId>jacoco-maven-plugin</artifactId>
						<version>0.5.3.201107060350</version>
						<executions>
							<execution>
								<goals>
									<goal>prepare-agent</goal>
								</goals>
								<configuration>
									<!-- Where to put jacoco coverage report -->
									<destFile>${sonar.jacoco.itReportPath}</destFile>
									<includes>*.gmf.*</includes>
									<!-- Append allows all reports from all executions to be stored in a single file -->
									<append>true</append>
								</configuration>
							</execution>
						</executions>
					</plugin>
				</plugins>
			</build>
		</profile>
	</profiles>
</project>

Then all your children tests modules will inherit from this extension and they'll automatically have test coverage enabled.

WARNING: If your tests are configured with a <argLine...> element, then put into this argLine element ${tycho.testArgLine} to avoid overriding Jacoco plugin configuration:

<!-- ... -->
	<plugins>
		<plugin>
			<groupId>org.eclipse.tycho</groupId>
			<artifactId>tycho-surefire-plugin</artifactId>
			<version>${tycho-version}</version>
			<configuration>
				<testSuite>org.eclipse.gmf.tests</testSuite>
				<testClass>org.eclipse.gmf.tests.AllTests</testClass>
				<useUIHarness>true</useUIHarness>
        			<useUIThread>true</useUIThread>
				<argLine>${tycho.testArgLine} -Xmx512m -XX:MaxPermSize=128m</argLine>
				<!-- ... -->

Signing

If you release your code at eclipse.org, you should get it signed.

There's a /usr/bin/sign binary on build.eclipse.org that you feed a zip file containing your binaries.

Signing is a bit complicated on hudson.eclipse.org and if your using tycho requires the usage of a special maven plugin.

The process the maven plugin goes through is comprised of 4 mojo's and the leveraging of a bit of ant with the ant-maven-plugin.

  1. pack - run the pack operation from the embedded eclipse equinox packing tool which itself is a wrapper over the pack tooling in the jdk.
  2. sign - copies the output from the pack over to the signer directory (should be configured for your project) and then executes the signer script. Once the signer script has finished processing the mojo will copy the signed work back to the target directory of the executing project
  3. repack - once more packs the project
  4. fixCheckSums - currently have to manually clean up the artifact.xml files for the new checksums of the signed and packed artifacts
  5. deploy site somewhere

For jetty we deploy the site to a static development repository under the typical p2 repository setup and when we want to release it public we copy the development directory to a named directory for the version and then update the p2 aggregate artifact and component xml files.

Example Usage:

  <profiles>
    <profile>
      <id>build-server</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.eclipse.dash.m4e</groupId>
            <artifactId>eclipse-signing-maven-plugin</artifactId>
            <version>1.0.4</version>
            <executions>
              <!-- 
              Pack the p2 repository.
               -->
              <execution>
                <id>pack</id>
                <phase>package</phase>
                <goals>
                  <goal>pack</goal>
                </goals>
              </execution>
              <!-- 
              Sign the p2 repository
              -->
              <execution>
                <id>sign</id>
                <configuration>
                  <signerInputDirectory>/home/data/httpd/download-staging.priv/rt/PROJECT</signerInputDirectory>
                </configuration>
                <phase>package</phase>
                <goals>
                  <goal>sign</goal>
                </goals>
              </execution>
              <!-- 
              Repack the p2 repository
              -->
              <execution>
                <id>repack</id>
                <configuration>
                  <inputFile>${project.build.directory}/signed/site_assembly.zip</inputFile> <!-- this is output from signer mojo -->
                </configuration>
                <phase>package</phase>
                <goals>
                  <goal>pack</goal>
                </goals>
              </execution>
              <!-- 
              Signing and packing alters checksums so fix them
              -->
              <execution>
                <id>fixCheckSums</id>
                <phase>package</phase>
                <goals>
                  <goal>fixCheckSums</goal>
                </goals>
              </execution>
            </executions>
          </plugin>
          <!--
          This is what I use to deploy a p2 repository someplace to test from before manually making active.        
          -->
          <plugin>
              <artifactId>maven-antrun-plugin</artifactId>
              <executions>
                <execution>
                  <id>deploy</id>
                  <phase>install</phase>
                  <goals>
                    <goal>run</goal>
                  </goals>
                  <configuration>
                    <tasks>
                      <delete includeemptydirs="false">
                        <fileset
                          dir="/home/data/httpd/download.eclipse.org/jetty/updates/jetty-wtp/development">
                          <include name="**" />
                        </fileset>
                      </delete>
                      <copy includeemptydirs="false"
                        todir="/home/data/httpd/download.eclipse.org/jetty/updates/jetty-wtp/development">
                        <fileset dir="target/checksumFix">
                          <include name="**" />
                        </fileset>
                      </copy>
                    </tasks>
                  </configuration>
                </execution>
              </executions>
            </plugin>
          </plugins>
      </build>
    </profile>
  </profiles>

Hudson (Jenkins)

There's a hudson instance that's available for eclipse.org committers.

To run your Tycho build in Hudson, it's very easy.

The first step is to select the correct Git repository to fetch from.

Minervahudson1.png

The next step is to setup the right Maven goals to run.

Minervahudson2.png

The final step is to ensure we publish our repository artifacts when done.

Minervahudson3.png

That's it, you should see the latest successful artifacts after the build is complete.

Publishing

Publishing is a bit tricky at eclipse.org, what some projects do for nightly builds is run a cron job on build.eclipse.org

For example, the EGit project has a cron entry like this:

* */3 * * * sh /home/data/httpd/download.eclipse.org/egit/pubegit.sh

The script essentially grabs the latest succeessful build from hudson and publishes it to a directory.

BUILD_LOC="/home/data/httpd/download.eclipse.org/egit"
 
# where you will have your promotion logs
PROMO_LOGS_DIR=""
 
# this script log
logFile="publish.log"
 
rm -f $logFile
echo "[`date +%Y/%m/%d\ %H:%M`]: getting last successful build" >> $logFile
mkdir -p $BUILD_LOC
rm -f $BUILD_LOC/site.zip
rm -rf $BUILD_LOC/build
cd $BUILD_LOC
wget --no-check-certificate
"https://hudson.eclipse.org/hudson/job/egit/lastSuccessfulBuild/artifact/org.eclipse.egit-updatesite/target/site/*zip*/site.zip"
-o $logFile
if [ ! -f site.zip ]; then echo "ERROR:build.zip (from Hudson) not
found" >> $logFile; exit -2; fi
unzip site.zip >> $logFile
rm -Rf updates-nightly
mkdir updates-nightly
mv -f site/* updates-nightly/
echo "[`date +%Y/%m/%d\ %H:%M`]: publishing nightly build ..." >> $logFile