View source for Making Builds Reproducible

===Making Builds Reproducible===

==Background==

In order to build non-SDK projects offline -- a requirement of our buildsystem
for security purposes -- we must do the following:

# Generate a source drop
## check out releng plugin
## add a fetch target
## run the fetch target with the Eclipse antrunner
## tar up the results
# Run the standard PDE build (ie. java -cp $SDK/startup.jar org.eclipse.core.launcher.Main

==Common Pain Points==

The last step -- the actual building with PDE -- is often different between
projects.  Some common variable differences:

* javacFailOnError - why isn't this always set?
* mapsLocal
* pde.build.scripts
* skipFetch
* buildLabel
* buildAlias
* baseExists
* eclipse.builder.fetch
* baseLocation vs. base.location
* pde.builder.path
* mapVersionTag
* buildDirectory

* a build.xml that is essentially a shell for PDE's build.xml - why not just use PDE build.xml directly?
* ant build files that are shells around the "standard" (essentially empty) build.xml that set obscure variables that are necessary but non-standard
* build scripts (bash, DOS) that set obscure variables that are necessary but non-standard
* build scripts that assume relative/absolute paths
* customTargets.xml attempts to fetch dependencies with no flags to override this behaviour
** ex. getBaseBuilder is in postFetch unconditionally - we have to patch this out
* build scripts that assume certain usernames/passwords -- as mentioned in [[Europa_Build_Workshop_Report#Eclipse_Build_Best_Practices|Eclipse Build Best Practices]], a common cause of this is the lack of separation between fetching, building, and publishing

==Solutions to these common gotchas==
* separate fetching of your project's source code from fetching basebuilder/SDK and dependencies
* wrap fetching of basebuilder/SDK and dependencies with flags that can be overridden (ie. dontFetch)
* use consistent flags/variables
** derive values from other variables (ie. pde.build.scripts/path from baseLocation
* do not assume use of basebuilder as it is ''almost always'' perfectly fine to use the SDK -- especially for I- or M-buildsand releases -- and we do this :)
* do not hard-code usernames, paths, etc. that aren't either universal or at the very least overridable

==An Ideal Future==

In an ideal world, all it would take to build a project on top of the SDK would
be something like this:

# assume SDK is available and have a standard way of passing its path during the build
# download source drop from project -- no generated files (including build.xmls), no binary code
# run the PDE build.xml with consistent options
## I envision us using:  baseLocation, dontUnzip, dontFetch -- everything else should be derived from these values