Jump to: navigation, search

Difference between revisions of "NoLongerUsed.ContentsmovedToAnotherPage"

(migrate)
m (David williams.acm.org moved page CBI/p2repoAnalyzers/Repo Reports Running Locally to NoLongerUsed.ContentsmovedToAnotherPage: noLongerUsedCombinedWithAnotherPage)
 
(12 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
<h1>Running your own Repository Reports</h1>
 
<h1>Running your own Repository Reports</h1>
 +
 +
<!--
 +
    Note: This document is duplicated in the wiki and the Git repository:
 +
    https://wiki.eclipse.org/CBI/p2repoAnalyzers/Repo_Reports_Running_Locally
 +
    org.eclipse.cbi.p2repo.analyzers-bundle/documentation/how-to.html
 +
    They should be kept in sync (or, one removed, if not needed).
 +
-->
 +
 
<h2>Introduction</h2>
 
<h2>Introduction</h2>
<p>This document gives a brief outline of how to run the "repo reports" locally,  
+
<p>This document gives a brief outline of how to run the "repo reports" locally,
 
against a repository on your local file system. </p>
 
against a repository on your local file system. </p>
  
<p>There are some tests, that look at jars specifically,  
+
<p>There are some tests, that look at jars specifically,
that require the jars to be on local file system and essentially use plain 'ol Java file IO and regex-type checks on  
+
that require the jars to be on local file system and essentially use plain 'ol Java file IO and regex-type checks on
the contents.  
+
the contents.</p>
  
<p>Another class of tests, read the content.jar/xml meta-data and reports on the data or relationships in  
+
<p>Another class of tests, read the content.jar/xml meta-data and reports on the data or relationships in
that meta data. <p>
+
that meta data. </p>
  
<p>Yet another, small class of tests, verify the jars are signed. These tests are not really "Java" or "workspace" related,  
+
<p>Yet another, small class of tests, verify the jars are signed.  
but instead, use shell scripts to invoke "jarsigner -verify" repeatedly
+
These tests are not really "Java" or "workspace" related, but use Java's "exec" method
on a directory of jars. These shell scripts are not very efficient, but ... do get the basic signature verification checks done
+
to invoke "jarsigner -verify" on multiple threads, on a directory of jars. (There is actually a faster heuristic in the code that simply looks for the presence of the Eclipse signature file, but that is a heuristic and might not always be accurate, so it not used by default).</p>
in the most neutral, flexible way possible. They make
+
sure jars are signed and that pack.gz files can be unpacked, and then the signature of the resulting jar verified.
+
You can change the
+
VM level used since there are known issues with each level of VM; some jars can not be unpacked
+
with java 5, say, or those with nested jars can not be unpacked with Java 7. Which VM level to use depends on your goals and
+
end-user and adopter needs. (We use Java 6 during the report generation on for Simultaneous Release.)</p>
+
  
 
<h2>Running the repo tests</h2>
 
<h2>Running the repo tests</h2>
  
These instructions are focus for "running from your workspace", but could also easily be done "from the command line" if you know how  
+
<p>These instructions are focus for "running from your workspace", but could also easily be done "from the command line" if you know how
to run Eclipse Applications from the command line.  
+
to run Eclipse Applications from the command line.</p>
  
 
<h3>The Basics</h3>
 
<h3>The Basics</h3>
  
The Eclipse Application is named 'org.eclipse.simrel.tests.repoReport'. It takes two "system properties"; one to specify  
+
<p>The Eclipse Application is named 'org.eclipse.cbi.p2repo.analyzers.repoReport'. It takes two "system properties"; one to specify
where you want the output to go, and another to specify where the repo is on the file system. For example:
+
where you want the output to go, and another to specify where the repository-to-analyze is on the file system. An optional third parameter names a repository to use as reference for the "version check" reports. For example:</p>
 
<ul>
 
<ul>
<li>-DreportOutputDir=/home/shared/eclipse/repoReport</li>  
+
<li>-DreportOutputDir=/home/shared/eclipse/repoReport</li>
<li>-DreportRepoDir=/home/www/html/downloads/eclipse/updates/4.2-I-builds/I20120531-1500/</li>  
+
<li>-DreportRepoDir=/home/www/html/downloads/eclipse/updates/4.6-M-builds/M20161013-0730/</li>
 +
<li>-DreferenceRepo=/home/www/html/downloads/eclipse/updates/4.6/R-4.6.1-201609071200/</li>
 
</ul>
 
</ul>
 +
<p>There is another parameter, ''-DuseNewApi=true'' which is not yet documented ({{bug|487409}}) but runs the code in such a way that tests pass, fail, or give a warning, and produces [http://download.eclipse.org/eclipse/downloads/drops4/R-4.6.1-201609071200/buildlogs/errors-and-moderate_warnings.html compact, color coded table of results], to link to an experimental example.</p>
  
 
<h3>The Details</h3>
 
<h3>The Details</h3>
  
<p>The source project is named 'org.eclipse.simrel.tests' and is currently in Git in repository named 'simrel/org.eclipse.simrel.tests.git'. See {{Git|simrel|org.eclipse.simrel.tests.git}}.</p>
+
<p>The source project is named 'org.eclipse.cbi.p2repo.analyzers' and is currently in Git in repository named 'cbi/org.eclipse.cbi.p2repo.analyzers.git'. See {{Git|CBI|org.eclipse.cbi.p2repo.analyzers.git}}.</p>
  
<p>Once you load that project into your workspace, it will include one "launch configuration" that can be used as a starting example,  
+
<p>Once you load that project into your workspace, it will include one "launch configuration" that can be used as a starting example,
edited and used to launch  
+
edited and used to launch
the application from your workspace.<p>
+
the application from your workspace.</p>
  
<p>On a large repository, it take 5 or 10 minutes to complete, and then you just to look at the
+
<p>On a large repository, it take 5 or 10 minutes to complete, and then you just look at the
results (usually) with a web browser starting with the 'index.html' at the location you specified in 'reportOutputDir' property.<p>
+
results (usually) with a web browser starting with the 'index.html' at the location you specified in 'reportOutputDir' property.</p>
  
<h3>Signing verification</h3>
+
<p>For one example of a bash script that takes advantage of a "product build" to run the report application, see the example in the Eclipse Platform Git repository named [http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/plain/production/createReports.sh createReports.sh].</p>
  
There are two scripts in the 'org.eclipse.juno.tests' that work together to check signatures: verify.sh and verifydir.sh. Written for Linux, but could be re-written for Windows, I'd guess. I normally put these on my path, such as in my ~/bin directory. The run time, I navigate to the "top" of the repository, and execute ./verifydir.sh which uses the current directory (if none specified) and iterates through all the files looking for "*.jar" and '*pack.gz' files and eventually calls 'verify.sh' on a specific file. You will need to edit verify.sh and define the Java directory on your local system where "jarsigner" can be found. Once ran, the scripts create report files in ~/verifyoutputdir (in your home folder, erasing what ever was there before) and you can check those report files for "errors" or "unsigned" items.
+
<p>For an example of installing as a feature from a p2 repository see the [http://git.eclipse.org/c/simrel/org.eclipse.simrel.tools.git/tree/build.xml#n110 'installTestsFromRepo' target] in the SimRel build.xml file. And then, see the actual running of the tests in the [http://git.eclipse.org/c/simrel/org.eclipse.simrel.tools.git/tree/runTests.xml#n97 'runReports' target] in the SimRel runTests.xml file.</p>
  
 
<h2>Getting help or making contributions</h2>
 
<h2>Getting help or making contributions</h2>
  
You can ask questions on cross-project list if the tests do not work as expected. Or, even better, you can make improvements/fixes directly on this wiki, if the instructions can be better. Feel free to supply patches on the cross-project component in bugzilla.
+
<p>You can ask questions on cross-project list if the tests do not work as expected. Or, even better, you can make improvements/fixes directly on this wiki, if the instructions can be better. Feel free to supply patches on the cross-project component in bugzilla.</p>
 
+
 
+
[[Category:Kepler| ]]  [[Category:Juno| ]] [[Category:Coordinated]]
+

Latest revision as of 14:34, 17 October 2016

Running your own Repository Reports


Introduction

This document gives a brief outline of how to run the "repo reports" locally, against a repository on your local file system.

There are some tests, that look at jars specifically, that require the jars to be on local file system and essentially use plain 'ol Java file IO and regex-type checks on the contents.

Another class of tests, read the content.jar/xml meta-data and reports on the data or relationships in that meta data.

Yet another, small class of tests, verify the jars are signed. These tests are not really "Java" or "workspace" related, but use Java's "exec" method to invoke "jarsigner -verify" on multiple threads, on a directory of jars. (There is actually a faster heuristic in the code that simply looks for the presence of the Eclipse signature file, but that is a heuristic and might not always be accurate, so it not used by default).

Running the repo tests

These instructions are focus for "running from your workspace", but could also easily be done "from the command line" if you know how to run Eclipse Applications from the command line.

The Basics

The Eclipse Application is named 'org.eclipse.cbi.p2repo.analyzers.repoReport'. It takes two "system properties"; one to specify where you want the output to go, and another to specify where the repository-to-analyze is on the file system. An optional third parameter names a repository to use as reference for the "version check" reports. For example:

  • -DreportOutputDir=/home/shared/eclipse/repoReport
  • -DreportRepoDir=/home/www/html/downloads/eclipse/updates/4.6-M-builds/M20161013-0730/
  • -DreferenceRepo=/home/www/html/downloads/eclipse/updates/4.6/R-4.6.1-201609071200/

There is another parameter, -DuseNewApi=true which is not yet documented (bug 487409) but runs the code in such a way that tests pass, fail, or give a warning, and produces compact, color coded table of results, to link to an experimental example.

The Details

The source project is named 'org.eclipse.cbi.p2repo.analyzers' and is currently in Git in repository named 'cbi/org.eclipse.cbi.p2repo.analyzers.git'. See org.eclipse.cbi.p2repo.analyzers.git (browse, stats, fork on OrionHub) .

Once you load that project into your workspace, it will include one "launch configuration" that can be used as a starting example, edited and used to launch the application from your workspace.

On a large repository, it take 5 or 10 minutes to complete, and then you just look at the results (usually) with a web browser starting with the 'index.html' at the location you specified in 'reportOutputDir' property.

For one example of a bash script that takes advantage of a "product build" to run the report application, see the example in the Eclipse Platform Git repository named createReports.sh.

For an example of installing as a feature from a p2 repository see the 'installTestsFromRepo' target in the SimRel build.xml file. And then, see the actual running of the tests in the 'runReports' target in the SimRel runTests.xml file.

Getting help or making contributions

You can ask questions on cross-project list if the tests do not work as expected. Or, even better, you can make improvements/fixes directly on this wiki, if the instructions can be better. Feel free to supply patches on the cross-project component in bugzilla.