Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Orion/Releng Builds"

(Remove PDE sections. PDE build is dead (**balloons fall from ceiling**))
(Tycho build)
Line 105: Line 105:
  
 
; [https://hudson.eclipse.org/orion/job/orion-client/ orion-client]
 
; [https://hudson.eclipse.org/orion/job/orion-client/ orion-client]
: Builds Orion's client-side JavaScript code and performs JS minification.
+
: Builds Orion's client-side JavaScript code and performs JS minification. Unit tests may also be executed if the <tt>orion-test</tt> profile is enabled.
  
 
; [https://hudson.eclipse.org/orion/job/orion-server/ orion-server]
 
; [https://hudson.eclipse.org/orion/job/orion-server/ orion-server]

Revision as of 16:54, 22 May 2014

Orion currently has 3 kinds of builds:

How to contribute

This section applies to all builds.

Contributing changes to the build

Contributing to a build is easy: just commit your code to the master stream of the Orion repository and it will be included in the next integration build.

Setting up your workspace

  1. See Orion/Getting the source for instructions on how to set up your workspace to work with Orion source code.

Contributing your changes

Once you have made code changes, you can contribute them to the Integration build as follows:

  • Commit your changes to the local repository and push them to the master
git commit -a -m "fix lots of bugs"                   #commits all changes
git push                                              #push local commits to the master

Minification

The Orion build performs optimization of client pages and plugins using RequireJS. Note the distinction between concatenation (inlining a module's AMD dependencies to reduce HTTP overhead) and minification (shrinking a module's JavaScript code size by removing whitespace). The build minifies all JavaScript code automatically, but getting your module concatenated requires an extra step. This section has instructions on how to add new content for concatenation in the build.

Adding a new module to the optimization

To add a new page or plugin for concatenation, you must update this file: releng/org.eclipse.orion.client.releng/builder/scripts/orion.build.js Find the property modules, and add an entry for your page/plugin.

Every entry needs a name property giving the module's web path. For example, here is the Git Commit page's entry:

    { name: "git/git-commit" },

The Tycho and Node builds all share orion.build.js, so your new module will be correctly built everywhere.

Adding a new client bundle to the JS build

  1. Add an entry to the bundles section of orion.build.js. (Ant-style property replacement is supported within the bundles entry, but ${orionClient} is the only property you should use there.)
  2. If your bundle contains any JSDocs, add an entry to the jsdocs section of orion.build.js.
  3. For each of the bundle's modules you want to optimize, add an entry to the modules section of orion.build.js. (see previous section).
  4. Add a pom.xml file in the root folder of your bundle.
  5. Add an entry to the <modules> section of the client repo's top-level pom.xml. This ensures that the Maven builder finds your bundle.
  6. Update the selfHostingRules.js file by adding an entry for the new bundle. This is required for your bundle's resources to be accessible when self-hosting Orion.

Building stand-alone features

Orion pages and plugins are handled by a multi-module invocation of r.js that optimizes all of them in one shot. However, stand-alone features need different build config options, which means r.js must be invoked separately for each of these features.

To build a stand-alone feature, you must add logic to the requirejs task in the Ant script orion.mini.xml. You must invoke r.js, passing the build file for your feature. Here is an example showing how a file used for the standalone Orion editor is minified:

  1.  <!-- Clean widget output directory -->
  2.  <target name="widgetClean" depends="widget.check" if="widget.run">
  3.     <delete dir="${widgetOutDirectory}" />
  4.     <mkdir dir="${widgetOutDirectory}" />
  5.  
  6.     <copy file="${builder}/scripts/editor.build-js.js" tofile="${staging}/editor.build-js.js" overwrite="true"/>
  7.     <!-- ... -->
  8.  </target>
  9.  
  10.  <target name="requirejs" depends="checkDirs, cleanStaging, cleanOptimized">
  11.      <!-- ... -->
  12.  
  13.      <!-- build standalone editor (unless skipEditor is set) -->
  14.      <antcall target="editorOptimize">
  15.          <param name="staging" value="${staging}" />
  16.          <param name="out" value="${widgetOutDirectory}" />
  17.      </antcall>
  18.  
  19.      <!-- ... -->
  20.  </target>
  21.  
  22.  <target name="editorOptimize" unless="skipEditor">
  23.      <sequential>
  24.          <!-- Invoke r.js -->
  25.          <widgetOptimizeFile
  26.             out="${out}/built-editor-amd.min.js"
  27.             buildfile="${staging}/editor.build-js.js"
  28.             optimize="closure"
  29.             staging="${staging}" />
  30.  
  31.          <!-- Additional calls to r.js can be placed here, if needed -->
  32.      </sequential>
  33.  </target>

Deploying builds to orion.eclipse.org or orionhub.org

Builds are deployed using the script deploy.sh. This script should be copied to your home directory and run from there. Your home directory will contain a symlink to the downloads directory, so you can perform a deploy directly from the downloads area. The script takes a single argument which is the location of the zip containing the download. For our servers we want the Linux 64-bit build. It is useful to log the output of this script so it can be reviewed later. Example:

./deploy.sh -archive downloads/orion/drops/I201201102230/eclipse-orion-I20120110-2230-linux.gtk.x86_64.zip >> deploy.log

This script simply copies the build onto the deployment server, and invokes an upgrade script on that server - upgrade.sh. The upgrade script shuts down the old server, moves it, unzips and configures the new build, and finally starts it.

The entire deploy/upgrade process takes about 5 seconds when it runs smoothly. Occasionally there will be a communication error copying the new build onto the target machine. In this case simply re-running the script usually succeeds.

Server configuration

The server configuration file on orion.eclipse.org and orionhub.org is found in ~admin/current/orion.conf. This file gets copied into the server by the deployment script. So, any change made to this configuration file will take effect only on the next deployment.


Tycho build

The Tycho builds use the Eclipse Tycho infrastructure (Category:Tycho), centered around Maven. Orion committers can managed Tycho builds using the Orion Hudson instance. The build comprises 2 jobs:

orion-client
Builds Orion's client-side JavaScript code and performs JS minification. Unit tests may also be executed if the orion-test profile is enabled.
orion-server
Downstream project of orion-client. Builds Orion's Java server and runs the tests.

As of this writing, Tycho builds only run Orion's Java server tests; Orion's JavaScript tests are not yet supported.

Node build

The Node build is a Grunt script that produces a self-contained, Node.js-based Orion server with optimized client-side JavaScript code.

The script runs automatically before Orion is published to the npm repository, but it can be run manually as well. The Orion dev team publishes builds on a regular basis, at least once per milestone.

Running the Node build script

First, you should also install grunt globally:

npm install -g grunt

Then ensure you have the dev dependencies the Orion Node server needs:

$ cd modules/orionode
$ npm install

To run the build, simply run grunt from the orionode folder:

$ cd modules/orionode
$ grunt

< lots of output >

Done, without errors.
$ 

When the script finishes, modules/orionode contains a complete optimized server suitable for publishing to npm. Running node server.js as normal will launch the optimized server.

To produce a standalone optimized tarball of Orion, use the npm pack command:

$ npm pack

> orion@0.0.27 prepublish .
> grunt

< lots of output >

Done, without errors.
orion-0.0.27.tgz
$

How it works

The script copies all the client code from the top-level bundles/ folder of the Orion client repo into modules/orionode/orion.client/bundles. The code is then "staged" (i.e. the runtime web-path structure is recreated on disk) to a temp directory, and RequireJS optimization is performed. After optimization, minified code and source maps are copied back into the bundle folders that they originated from. The end result is a self-contained directory.

To build a customized Orion server, edit Gruntfile.js. You can supply an alternative path to the Orion client code, or a different RequireJS build configuration (take care to maintain the Orion "shape", however: for example your build config must have a bundles property).

Publishing to npm (for package maintainers)

There's a publish.sh script that performs a build and then publishes to the npm package repository if the build succeeded. You must be a maintainer of the orion package to publish, contact @kwalker or @mamacdon if you need access.

$ ./publish.sh
<lots of grunt output>
<lots of npm output>

Back to the top