Jump to: navigation, search

Difference between revisions of "CBI/p2repoAnalyzers/Repo Reports"

< CBI
(Common Software Repository Reports)
(With Maven)
 
(20 intermediate revisions by 5 users not shown)
Line 1: Line 1:
==Common Software Repository Reports==
+
=p2 Repository Analysis=
  
This page roughly describes collection of automated checks and reports to run against p2 repositories or directories of jars. Eventually it will become more of a "how to" and FAQ oriented page.
+
== Goals ==
  
The reports are currently ran against the latest contents of the "staging" repository and placed in the [http://build.eclipse.org/juno/simrel shared space on the build machine] where they can be viewed with a web browser.  
+
The '''p2RepoAnalyzers''' are a collection of automated quality (legal, version rules...) checks and reports to run against p2 repositories or directories of jars. They're meant to be used by Eclipse projects to guarantee conformance to typical Eclipse.org rules.
  
These are very temporary reports, and will be removed the next time new reports are created. Which is normally when
+
'''The main goal is that all projects can perform these tests themselves, early in development cycle, every time they build a p2 repository.'''
staging repository is updated (Once or twice or three times a day, when busy).  
+
  
While many projects have their own, similar tests (which have one way or another provided the starting point for all these tests)
+
So far, the main user of these reports is SimRel Build: the reports for the simultaneous release repository are simply a final sanity check. They are ran against the latest successful [https://hudson.eclipse.org/simrel/job/simrel.oxygen.runaggregator.BUILD__CLEAN/ Simultaneous Release CLEAN BUILD on Hudson], so can always be viewed there. There is a link on the Hudson build instance [https://hudson.eclipse.org/simrel/ main page] that links to the [https://hudson.eclipse.org/simrel/job/simrel.oxygen.runaggregator.BUILD__CLEAN/lastSuccessfulBuild/artifact/aggregation/final/buildInfo/reporeports/index.html latest report].  In that case, the report is also copied, along with the repository, so it ends up on staging area too, so it can be viewed under a URL such as for the [http://download.eclipse.org/staging/neon/buildInfo/reporeports/ staging repository].
it is worth some effort effort to collect some common tests in a common place to encourage reuse and improvements.  
+
  
For now, the tests and scripts are in Git repo. To load into workspace, and appropriate URL would be similar to
+
While several projects have their own, similar tests (which have one way or another have provided the starting point for all these tests) it is worth some effort to collect some common tests in a common place to encourage reuse and improvements. So if you've set up some similar checks for your project, you should highly consider contributing them to this common place.
  
  <code><nowiki>ssh://<userid>@git.eclipse.org/gitroot/simrel/org.eclipse.simrel.tests.git</nowiki></code>
+
The reports can be ran locally from your workbench or adopted for your own production builds.
  
Or, for casual browsing, see {{Git|simrel|org.eclipse.simrel.tests.git}}.
+
== Description of tests and reports ==
  
The reports can be [[SimRel/Simultaneous_Release_Reports_Running_Locally| ran locally]] or adopted for your own builds.
+
There are some tests, that look at jars specifically,
 +
that require the jars to be on local file system and essentially use plain 'ol Java file IO and regex-type checks on
 +
the contents.
  
Put another way, the ultimate goal is that the reports for the simultaneous release common repository are simply a final sanity
+
Another class of tests, read the content.jar/xml meta-data and reports on the data or relationships in
check and that all projects can perform these tests themselves, early in development cycle. But ... until then, the reports will
+
that meta data.  
be ran against, at least, the common staging repositories. Most of the tests can be used, with some copy/paste/hacking
+
directly from an IDEs workbench, against a local repository on your file system. Start small. :)
+
  
If/when others have fixes, abstractions, or new tests, please open a but in the cross-project component, so these
+
Yet another, small class of tests, verify that the jars are signed.
tests and reports can be improved over time by community effort.  
+
These tests are not really "Java" or "workspace" related, but use Java's "exec" method
 +
to invoke "jarsigner -verify" on multiple threads, on a directory of jars. (There is actually a faster heuristic in the code that simply looks for the presence of the Eclipse signature file, but that is a heuristic and might not always be accurate, so it not used by default).
  
The code and scripts, as of right now, are oriented towards simply "producing reports". But if you browse the code,  
+
The code and scripts, as of right now, are oriented towards simply '''producing reports'''. But if you browse the code, you'll see some commented-out code in places that can cause <span style="color:red">failed</span> flags to be set, which would be appropriate for most projects and, eventually, even the Simultaneous Release repository.. But more recently Dennis Huebner has contributed the framework to run as JUnit tests but that still needs to be documented. See also the -useNewApi flag which produces reports based on <span style="color:green">passed</span>, <span style="color:red">failed</span>, or <span style="color:orange">warning</span> (which is a variant of passed more than a variant of failed). Follow {{bug|487409}} for updating the documentation.
you'll see some commented-out code in places that can cause "failed" flags to be set, which would be appropriate for some
+
projects ... and, long term, maybe even the common repository. Similarly, most, now, are oriented towards being a simple "ant task".
+
It might be possible, with moderate effort, to convert most to "unit tests" for more finely tuned testing and error reporting,
+
if there was ever an advantage to that.  
+
  
One big problem right now, are the "signing checking scripts". These are literally shell scripts, that call 'jarsigner -verify', one jar at a time, so it takes forever. Several hours. Where as all the other tests combined finish in about 10 minutes, total. It'd be much better to do the jar verification with Java more directly, so processes and executables do not have to start and stop 5000 times. Volunteers? :)
+
== Running reports for your project ==
  
 +
The project is built and delivered as both an Eclipse "product" and as a feature installable from a p2 repository. As of this writing, no "permanent" locations for those (watch {{bug|506037}}) but in the meantime can be downloaded or installed from the results of a Hudson build. The Hudson builds and functional testing is done under https://hudson.eclipse.org/cbi/view/p2RepoRelated/ and the p2 repository where the product and feature are currently published is https://hudson.eclipse.org/cbi/job/cbi.p2repo.analyzers.build/lastSuccessfulBuild/artifact/output/p2repo/ .
  
[[Category:Kepler| ]] [[Category:Juno| ]] [[Category:Coordinated]]
+
The '''Eclipse Application is named <tt>org.eclipse.cbi.p2repo.analyzers.repoReport</tt>'''. It is parameterized by two mandatory System Properties; one to specify
 +
where you want the output to go, and another to specify where the repository-to-analyze is on the file system. An optional third parameter names a repository to use as reference for the "version check" reports. For example:
 +
<ul>
 +
<li>-DreportOutputDir=/home/shared/eclipse/repoReport</li>
 +
<li>-DreportRepoDir=/home/www/html/downloads/eclipse/updates/4.6-M-builds/M20161013-0730/</li>
 +
<li>-DreferenceRepo=/home/www/html/downloads/eclipse/updates/4.6/R-4.6.1-201609071200/</li>
 +
</ul>
 +
There is another parameter, <tt>-DuseNewApi=true</tt> which is not yet documented ({{bug|487409}}) but runs the code in such a way that tests <span style="color:green">pass</span>, <span style="color:red">fail</span>, or give a <span style="color:orange">warning</span>, and produces [http://download.eclipse.org/eclipse/downloads/drops4/R-4.6.1-201609071200/buildlogs/errors-and-moderate_warnings.html compact, color coded table of results], to link to an experimental example.
 +
 
 +
=== In Eclipse IDE ===
 +
 
 +
Get source project  named 'org.eclipse.cbi.p2repo.analyzers' and is currently in Git in repository named 'cbi/org.eclipse.cbi.p2repo.analyzers.git'. See {{Git|CBI|org.eclipse.cbi.p2repo.analyzers.git}}. Once you load that project into your workspace, it will include one "launch configuration" that can be used as a starting example,
 +
edited and used to launch the application from your workspace.
 +
 
 +
=== With Eclipse command-line ===
 +
 
 +
For one example of a bash script that takes advantage of a "product build" to run the report application, see the example in the Eclipse Platform Git repository named [http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/plain/production/createReports.sh createReports.sh].
 +
 
 +
=== With Ant ===
 +
 
 +
Running the reports are, as of right now, oriented towards being a simple "Ant task".
 +
 
 +
For an example of installing as a feature from a p2 repository see the [http://git.eclipse.org/c/simrel/org.eclipse.simrel.tools.git/tree/build.xml#n110 'installTestsFromRepo' target] in the SimRel build.xml file. And then, see the actual running of the tests in the [http://git.eclipse.org/c/simrel/org.eclipse.simrel.tools.git/tree/runTests.xml#n97 'runReports' target] in the SimRel runTests.xml file.
 +
 
 +
=== With Maven ===
 +
 
 +
There is a bug open to convert to Maven tasks (see {{bug|487468}}).
 +
 
 +
However, the application can already be used in a Tycho build, typically as a <tt>verify</tt> step when building an <tt>eclipse-repository</tt>. See example:
 +
<source lang="xml">
 +
<!-- ... -->
 +
<build>
 +
  <plugins>
 +
  <plugin>
 +
  <groupId>org.eclipse.tycho.extras</groupId>
 +
  <artifactId>tycho-eclipserun-plugin</artifactId>
 +
  <version>${tycho-version}</version>
 +
  <executions>
 +
  <execution>
 +
  <phase>verify</phase>
 +
  <goals>
 +
  <goal>eclipse-run</goal>
 +
  </goals>
 +
  <configuration>
 +
  <applicationsArgs>
 +
  <arg>-application</arg>
 +
  <arg>org.eclipse.cbi.p2repo.analyzers.repoReport</arg>
 +
  </applicationsArgs>
 +
  <jvmArgs>
 +
  <arg>-DreportRepoDir=${project.build.directory}/repository</arg>
 +
  <arg>-DreportOutputDir=${project.build.directory}/repository/buildInfo</arg>
 +
<arg>-DreferenceRepo=/home/data/httpd/download.eclipse.org/staging/neon/</arg>
 +
  </jvmArgs>
 +
  <executionEnvironment>JavaSE-1.8</executionEnvironment>
 +
  <dependencies>
 +
<dependency>
 +
<artifactId>org.eclipse.cbi.p2repo.analyzers</artifactId>
 +
<type>eclipse-plugin</type>
 +
</dependency>
 +
<dependency>
 +
<artifactId>org.eclipse.equinox.p2.core.feature</artifactId>
 +
<type>eclipse-feature</type>
 +
</dependency>
 +
<dependency>
 +
<artifactId>org.eclipse.e4.rcp</artifactId>
 +
<type>eclipse-feature</type>
 +
</dependency>
 +
</dependencies>
 +
  <repositories>
 +
  <repository>
 +
  <id>cbi-reports</id>
 +
  <url>https://hudson.eclipse.org/cbi/job/cbi.p2repo.analyzers.build/lastSuccessfulBuild/artifact/output/p2repo/</url>
 +
  <layout>p2</layout>
 +
  </repository>
 +
  <repository>
 +
  <id>eclipse-4.6.1</id>
 +
  <url>http://download.eclipse.org/eclipse/updates/4.6/R-4.6.1-201609071200/</url>
 +
  <layout>p2</layout>
 +
  </repository>
 +
  </repositories>
 +
  </configuration>
 +
  </execution>
 +
  </executions>
 +
  </plugin>
 +
  </plugins>
 +
  </build>
 +
<!--... -->
 +
</source>
 +
 
 +
== Notes ==
 +
 
 +
On a large repository, it take 5 or 10 minutes to complete, and then you just look at the
 +
results (usually) with a web browser starting with the 'index.html' at the location you specified in 'reportOutputDir' property.
 +
 
 +
== Get support and Contribute ==
 +
 
 +
You can ask questions on [mailto:cbi-dev@eclipse.org cbi-dev] list if the tests do not work as expected. Or, even better, you can make improvements/fixes directly on this wiki, if the instructions can be better. Feel free to supply patches in the [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=CBI&component=p2%20Repository%20Analyzers p2 Repository Analyzers] component in Bugzilla.
 +
 
 +
For now, the tests and scripts are in Git repo. To load into workspace, an appropriate URL would be similar to <tt>ssh://<userid>@git.eclipse.org:29418/cbi/org.eclipse.cbi.p2repo.analyzers</tt>, or for casual browsing, see {{Git|cbi|org.eclipse.cbi.p2repo.analyzers.git}}.
 +
 
 +
If/when others have fixes, abstractions, or new tests, please open a bug in the [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=CBI&component=p2%20Repository%20Analyzers p2 Repository Analyzers] component in Bugzilla, so these tests and reports can be improved over time by community effort.
 +
 
 +
 
 +
[[Category:CBI| ]] [[Category:Oxygen| ]] [[Category:Neon| ]] [[Category:Mars| ]] [[Category:Luna| ]]

Latest revision as of 09:04, 26 October 2016

p2 Repository Analysis

Goals

The p2RepoAnalyzers are a collection of automated quality (legal, version rules...) checks and reports to run against p2 repositories or directories of jars. They're meant to be used by Eclipse projects to guarantee conformance to typical Eclipse.org rules.

The main goal is that all projects can perform these tests themselves, early in development cycle, every time they build a p2 repository.

So far, the main user of these reports is SimRel Build: the reports for the simultaneous release repository are simply a final sanity check. They are ran against the latest successful Simultaneous Release CLEAN BUILD on Hudson, so can always be viewed there. There is a link on the Hudson build instance main page that links to the latest report. In that case, the report is also copied, along with the repository, so it ends up on staging area too, so it can be viewed under a URL such as for the staging repository.

While several projects have their own, similar tests (which have one way or another have provided the starting point for all these tests) it is worth some effort to collect some common tests in a common place to encourage reuse and improvements. So if you've set up some similar checks for your project, you should highly consider contributing them to this common place.

The reports can be ran locally from your workbench or adopted for your own production builds.

Description of tests and reports

There are some tests, that look at jars specifically, that require the jars to be on local file system and essentially use plain 'ol Java file IO and regex-type checks on the contents.

Another class of tests, read the content.jar/xml meta-data and reports on the data or relationships in that meta data.

Yet another, small class of tests, verify that the jars are signed. These tests are not really "Java" or "workspace" related, but use Java's "exec" method to invoke "jarsigner -verify" on multiple threads, on a directory of jars. (There is actually a faster heuristic in the code that simply looks for the presence of the Eclipse signature file, but that is a heuristic and might not always be accurate, so it not used by default).

The code and scripts, as of right now, are oriented towards simply producing reports. But if you browse the code, you'll see some commented-out code in places that can cause failed flags to be set, which would be appropriate for most projects and, eventually, even the Simultaneous Release repository.. But more recently Dennis Huebner has contributed the framework to run as JUnit tests but that still needs to be documented. See also the -useNewApi flag which produces reports based on passed, failed, or warning (which is a variant of passed more than a variant of failed). Follow bug 487409 for updating the documentation.

Running reports for your project

The project is built and delivered as both an Eclipse "product" and as a feature installable from a p2 repository. As of this writing, no "permanent" locations for those (watch bug 506037) but in the meantime can be downloaded or installed from the results of a Hudson build. The Hudson builds and functional testing is done under https://hudson.eclipse.org/cbi/view/p2RepoRelated/ and the p2 repository where the product and feature are currently published is https://hudson.eclipse.org/cbi/job/cbi.p2repo.analyzers.build/lastSuccessfulBuild/artifact/output/p2repo/ .

The Eclipse Application is named org.eclipse.cbi.p2repo.analyzers.repoReport. It is parameterized by two mandatory System Properties; one to specify where you want the output to go, and another to specify where the repository-to-analyze is on the file system. An optional third parameter names a repository to use as reference for the "version check" reports. For example:

  • -DreportOutputDir=/home/shared/eclipse/repoReport
  • -DreportRepoDir=/home/www/html/downloads/eclipse/updates/4.6-M-builds/M20161013-0730/
  • -DreferenceRepo=/home/www/html/downloads/eclipse/updates/4.6/R-4.6.1-201609071200/

There is another parameter, -DuseNewApi=true which is not yet documented (bug 487409) but runs the code in such a way that tests pass, fail, or give a warning, and produces compact, color coded table of results, to link to an experimental example.

In Eclipse IDE

Get source project named 'org.eclipse.cbi.p2repo.analyzers' and is currently in Git in repository named 'cbi/org.eclipse.cbi.p2repo.analyzers.git'. See org.eclipse.cbi.p2repo.analyzers.git (browse, stats, fork on OrionHub) . Once you load that project into your workspace, it will include one "launch configuration" that can be used as a starting example, edited and used to launch the application from your workspace.

With Eclipse command-line

For one example of a bash script that takes advantage of a "product build" to run the report application, see the example in the Eclipse Platform Git repository named createReports.sh.

With Ant

Running the reports are, as of right now, oriented towards being a simple "Ant task".

For an example of installing as a feature from a p2 repository see the 'installTestsFromRepo' target in the SimRel build.xml file. And then, see the actual running of the tests in the 'runReports' target in the SimRel runTests.xml file.

With Maven

There is a bug open to convert to Maven tasks (see bug 487468).

However, the application can already be used in a Tycho build, typically as a verify step when building an eclipse-repository. See example:

<!-- ... -->
<build>
  	<plugins>
  		<plugin>
  			<groupId>org.eclipse.tycho.extras</groupId>
  			<artifactId>tycho-eclipserun-plugin</artifactId>
  			<version>${tycho-version}</version>
  			<executions>
  				<execution>
  					<phase>verify</phase>
  					<goals>
  						<goal>eclipse-run</goal>
  					</goals>
  					<configuration>
  						<applicationsArgs>
  							<arg>-application</arg>
  							<arg>org.eclipse.cbi.p2repo.analyzers.repoReport</arg>
  						</applicationsArgs>
  						<jvmArgs>
  							<arg>-DreportRepoDir=${project.build.directory}/repository</arg>
  							<arg>-DreportOutputDir=${project.build.directory}/repository/buildInfo</arg>
							<arg>-DreferenceRepo=/home/data/httpd/download.eclipse.org/staging/neon/</arg>
  						</jvmArgs>
  						<executionEnvironment>JavaSE-1.8</executionEnvironment>
  						<dependencies>
							<dependency>
								<artifactId>org.eclipse.cbi.p2repo.analyzers</artifactId>
								<type>eclipse-plugin</type>
							</dependency>
							<dependency>
								<artifactId>org.eclipse.equinox.p2.core.feature</artifactId>
								<type>eclipse-feature</type>
							</dependency>
							<dependency>
								<artifactId>org.eclipse.e4.rcp</artifactId>
								<type>eclipse-feature</type>
							</dependency>
						</dependencies>
  						<repositories>
  							<repository>
  								<id>cbi-reports</id>
  								<url>https://hudson.eclipse.org/cbi/job/cbi.p2repo.analyzers.build/lastSuccessfulBuild/artifact/output/p2repo/</url>
  								<layout>p2</layout>
  							</repository>
  							<repository>
  								<id>eclipse-4.6.1</id>
  								<url>http://download.eclipse.org/eclipse/updates/4.6/R-4.6.1-201609071200/</url>
  								<layout>p2</layout>
  							</repository>
  						</repositories>
  					</configuration>
  				</execution>
  			</executions>
  		</plugin>
  	</plugins>
  </build>
<!--... -->

Notes

On a large repository, it take 5 or 10 minutes to complete, and then you just look at the results (usually) with a web browser starting with the 'index.html' at the location you specified in 'reportOutputDir' property.

Get support and Contribute

You can ask questions on cbi-dev list if the tests do not work as expected. Or, even better, you can make improvements/fixes directly on this wiki, if the instructions can be better. Feel free to supply patches in the p2 Repository Analyzers component in Bugzilla.

For now, the tests and scripts are in Git repo. To load into workspace, an appropriate URL would be similar to ssh://<userid>@git.eclipse.org:29418/cbi/org.eclipse.cbi.p2repo.analyzers, or for casual browsing, see org.eclipse.cbi.p2repo.analyzers.git (browse, stats, fork on OrionHub) .

If/when others have fixes, abstractions, or new tests, please open a bug in the p2 Repository Analyzers component in Bugzilla, so these tests and reports can be improved over time by community effort.