Jump to: navigation, search

Difference between revisions of "Platform-releng/Releng Plan for CBI adoption"

(Towards M6)
(Week 2: (3/15 - 3/29))
Line 134: Line 134:
 
=== Week 2: (3/15 - 3/29) ===
 
=== Week 2: (3/15 - 3/29) ===
  
: Equinox drop site
+
: Create Equinox drop site
 
: finish delta pack
 
: finish delta pack
 +
: fix or ensure all warnings/compile errors are shown in summary
 +
:: figure out how to not "fail on error"?
 +
: Add post-processing step (using p2.process.artifacts)
 +
:: add traditional comparator step for sanity check (and fringe cases)
 +
:: add md5 checksums
 +
:: move pack.gz step to the end of production builds
 
: improve build error checking
 
: improve build error checking
: automate build failure notification
+
:: automate build failure notification
: ?include POM Updates in build notifications?
+
:: ?include POM Updates in build notifications?
: begin building our "build/publishing" separately (so we'd not rely on basebuilder, per se).  
+
: begin building our "build/publishing" separately  
 +
:: so we would not rely on basebuilder, per se.  
 +
:: could more easily fix or improve "publishing" issues
  
 
=== Week 3: (4/01 - 4/05) ===
 
=== Week 3: (4/01 - 4/05) ===

Revision as of 09:23, 20 March 2013

Purpose

This document is the plan, and working notes for moving to CBI-based builds. There is still lots to do, and many changes to make, so want to be sure as much is communicated as best as possible.

We want to be aggressive in adopting CBI based builds for Kepler, but at the same time, responsible. That is, to make sure it is correct -- or, correct enough -- that adopters (or, ourselves!) are not broken by any changes. And, that means "correct enough" from one I-build to the next, not "just by the time we release". This is a new area for most of us so will take a lot of focus.

The following are the major criteria to judge if/when we are ready to adopt as our primary build and/or release. Any others?

Main Criteria for moving to CBI based build for Kepler

  1. Have same output deliverables as current build: zips, repository, etc. (For Equinox and Eclipse)
  2. (Be able to reproduce exact same build, given a tag (or tags) to start the build with.)
  3. Same warnings (and compile errors, if any) as PDE based build.
  4. Be able to run our JUnit tests, with same results, as PDE based build.
  5. Produces the same Java Doc and Help documentation.
  6. Run a binary comparator against the bundles to ensure that they are the same binary content as the regular bundles.
    1. [Not sure how to do this since "qualifier algorithm" is different].
  7. Qualifiers not change (except for branding bundles) if the content has not changed.
  8. All bundles signed.
  9. Final I-build type repository have same content and metadata as PDE based build (nothing missing, nothing extra).
  10. Be easy to fit in to current workflow of automated builds and tests of "nightlies", I-builds, milestones (i.e. committers have to know what to do to "release" something for a build, how to "freeze" changes at a certain point).
  11. Don't create maintenance burden by duplicating information (bug 387802)

Bug queries and reference documents

  • URL for CBI based builds (temporary, just until we "switch"):
http://download.eclipse.org/eclipse/downloads/cbibasedindex.html

Significant builds and repositories

The last PDE based build

  • I20130219-1600 is the final "PDE Based build". It is recommended for those "building on" Eclipse to move up to that level now, so once they try a "CBI based build", they will have a good comparison if "all is ok" or if something surprising changed, or broke (if so, open a bug, if one doesn't exist; if one exists, add comments so we can better understand impact and priorities).
   http://download.eclipse.org/eclipse/downloads/drops4/I20130219-1600/

after M6 this was moved to archives site, but you'll need to know this exact URL to "see" it (not displayed on archives summary page). We will leave it there for remainder of Kepler, in case needs to be consulted for testing, etc., but then remove it.

   http://archive.eclipse.org/eclipse/downloads/drops4/I20130219-1600/
  • Repository. Since we are still evaluating version numbering (to make sure they always increase compared to previous I-builds) the repo site at http://download.eclipse.org/eclipse/updates/4.3-I-builds/ may not always give the result from the most recent build (for we hope only a short time). Therefore, the repo that corresponds to that "last PDE build", in non-composite form, is documented here:
   http://download.eclipse.org/eclipse/updates/4.3-I-builds/I20130219-1600/

after M6 this was moved to archives site, but you'll need to know this exact URL to "see" it (not displayed on archives summary page). We will leave it there for remainder of Kepler, in case needs to be consulted for testing, etc., but then remove it.

   http://archive.eclipse.org/eclipse/downloads/drops4/I20130219-1600/repository/

The first Tycho/Maven based Milestone build

  • S-4.3M6-201303141330 is the "first" Tycho/Maven based milestone (stable) build. Its distributions such as "SDK" and "Platform", tests framework, etc., all appear to be perfectly usable, though there remains some issues. The following limitations are known and being worked on next:
Some of the "zipped repositories" are not complete.
No Equinox drops
   http://download.eclipse.org/eclipse/downloads/drops4/S-4.3M6-201303141330/
  • The p2 repository for above site was published at the normal milestones sites.
     Composite: http://download.eclipse.org/eclipse/updates/4.3milestones/

   Simple site: http://download.eclipse.org/eclipse/updates/4.3milestones/S-4.3M6-201303141330/

Rollout and deadlines

Weeks below are numbered in "milestone weeks", beginning with "towards M6" ... there are only 6 weeks for M6.

Towards M6

Week 2: (2/11 - 15)

Begin to produce daily automated CBI based builds and run unit tests in parallel with PDE-based builds.
I'll schedule these to start at 8:11 PM, same time as PDE-based Nightlies and same time as weekly I-builds.
Initially, all CBI based builds (even "nightlies") will be "I-builds" to better assess versioning, and more work is need to implement "nightly vs. integration" builds.
I builds will build against master, controlled by repositories.txt
(Unit tests on Linux only, until those work as expected, to save Windows and Mac machine resources).
During this week "announce" the CBI based builds available, get community/adopter help in testing adequacy.
Committers should begin to use SDK's produced, and evaluate for correctness. In particular:
Check that version numbers change (or don't change) as expected. (I recommend you make some change, even trivial, to make sure it "gets in to" a build, if not otherwise planning changes this week).
Those with "binary bundles" check that they are correct ones.
Help evaluate failing JUnit suites. Some may be "releng" issues, but some may be project pom issues?
Initial Limitations: (meaning of priorities: high, need before making primary; medium, need before release; low, probably need at some point)
there will be no "autotagging" of aggregator or repositories, until we are more confident. (medium priority)
there will be no "equinox drops". (low/medium priority)
there will be no cbibased "I-build repo" (high priority)
"zipped repos" are known not to be quite right yet (low prioriy)
no ecj.jar produced (in hands of JDT Team? To tweak their "custom script"?) (low priority)
no "auto update" of pom's, based on manifest ... such as committers have to update version number in both, make "build.properties" changes in both. (low priority)

Week 3: (2/18 - 22)

Cut over to CBI based builds as primary (unless someone raising a true, blocking objection). [done. 2/20]
Begin to tag aggregator upon each build, (with "commits" of latest submodules added to aggregator). [done. 2/20]
(This allows "local build" to duplicate our build following directions given at Platform Build wiki)
TODO: someone (Thanh?) should confirm it gives same "target" results?)

Week 4: (2/25 - 3/1)

Begin to produce "4.3-I-build" repositories at .../eclipse/updates bug 393927.
Will make "massive" coordinated change to update all pom groupIds bug 397850 and the eclipse parent pom renamed bug 394831.
Some projects may need to "touch" their projects to get qualifiers "in order" (there is a difference in heuristics used for qualifiers so some in CBI build might be "lower than" those produced by PDE, in theory).
Will "back port" some changes to 422+ branches, since even though we will not be be releasing anything after 4.2.2, one of the reasons we committed to doing this was to support the LTS efforts, which needs the changes too.

Week 5: (3/4 - 8)

Possibly this week start to have true "CBI nightlies" vs. "Integration".

Week 6: (3/11 - 15) <== 3/15 is M6+0 day.

Towards M7

Week 1: (3/18 - 3/22)

triage/planning
initial 'nightlies'
who needs "master" vs. "integration" branches?

Week 2: (3/15 - 3/29)

Create Equinox drop site
finish delta pack
fix or ensure all warnings/compile errors are shown in summary
figure out how to not "fail on error"?
Add post-processing step (using p2.process.artifacts)
add traditional comparator step for sanity check (and fringe cases)
add md5 checksums
move pack.gz step to the end of production builds
improve build error checking
automate build failure notification
 ?include POM Updates in build notifications?
begin building our "build/publishing" separately
so we would not rely on basebuilder, per se.
could more easily fix or improve "publishing" issues

Week 3: (4/01 - 4/05)

sliced, zipped repos
Do we really need all those zipped repos? See bug 403811
make accurate. And remove pack.gz files (bug 403805

Week 4: (4/08 - 4/12)

Week 5: (4/15 - 4/19)

Week 6: (4/22 - 4/26)

Week 7: (4/29 - 5/03) <== 5/03 is M7 +0 day

Milestone stabilization week

Post Kepler

Over time, there may be efforts to make the build more "pure maven" (it still has some ant tasks, bash scripts, and still not ran on Hudson itself, for example), be less monolithic (its still pretty much "all or nothing", from a clean state), etc. BUT this will only be done with (continued) community help and involvement. There are no specific plans or resources to apply in this area, and this is just mentioned here for those from the community who read our "transition plan", wondering what the long term plan is.