Difference between revisions of "Hudson-ci/community meetings/Mar052012"

From Eclipsepedia

Jump to: navigation, search
(Notes)
 
(One intermediate revision by one user not shown)
Line 1: Line 1:
 
{{hudson|pageTitle=Community Meeting 05-March-2012}}  
 
{{hudson|pageTitle=Community Meeting 05-March-2012}}  
  
= Agenda  =
+
= Notes =
  
 
Add or comment on agenda items here, just prefix the item with your name so we can identify you on the call:  
 
Add or comment on agenda items here, just prefix the item with your name so we can identify you on the call:  
  
*2.x release?
+
*2.2.1
 +
There have been some P3 bugs fixed. No other core issues/mails request sent.
 +
There are 2 requests for github plugin that we need to fix as blockers. Henrik has seen this on Hudson on Hudson too
 +
Steven Christou is interested in looking at Github
  
 
*3.0.0  
 
*3.0.0  
**M2
+
Groovy has now been removed from core
**M3
+
Layer of scripting support created - for Groovy or other scripting tool
**RC
+
There is a hardcode dependency on Stapler - needs forking and hardcode dependency removed
 +
Stapler fork (at 1.1.55) and scripting layer need approval from Eclipse then can release M2.
 +
Should be no problem with provenance of that code
  
* [henrik] I got two bugs submitted with patches,(https://bugs.eclipse.org/bugs/show_bug.cgi?id=371312  & https://bugs.eclipse.org/bugs/show_bug.cgi?id=371313) but I haven't had any reaction on them. Am I missing some step in the submission process or are people just busy with other tasks?
+
*M2 - push back release to 16th April (can be edited if needed)
 +
to allow for QA cycle of above
 +
to check how long Eclipse approval may take for Groovy/Stapler
 +
Duncan to ping Eclipse with heads-up
  
* [henrik] we should clarify system requirements
+
*Eclipse Migration
** Java version ?
+
**Groovy - provenance issue, hence it's not approved in Eclipse (now removed from Hudson core)
** Capabilities of container ? (Java EE 5 web container?)
+
**Antlr - cannot get approval for current version, looker to move to Antler 3 (before RC). Winston also looking into removing it completely
 +
**Jaxen - rejected previously because of provenance, Duncan talking to author and may be able to solve. Some functionality might now be in JDK if we need to remove it. Not used directly by any code, brought in by other library
 +
**SVNKit - Externalise SVN plugin in github and provide a UI to install the bundled 'recommended' plugins first
  
* [henrik] performance work
 
** I have seen high load causing lock congestion in java.text.MessageFormat / java.util.Concurrency (see https://bugs.eclipse.org/bugs/show_bug.cgi?id=372679) I have tried to look at the code but I cannot seem to find a nice way around it. first of it is stapler features, secondly the messageformat class seems to always instantiate the sub formats, so no way to cache them. Any ideas?
 
  
** When hudson is stressed it gets really slow at serving files e.g. downloading a artifact is 5-10 times slower than loading in apache/scp. How do we best debug this, so that core developers have something to go on, and how can we improve the situation ? (can we siwtch to nio for master/slave communication? or use better streaming format?)
+
*M3 new feature - Install 'recommended' plugins either
** I suggest that many of the resources which does not change often, is written to disc and served from there instead of being processed/calculated on every request. I suggest we create a "component" (service) where plugins can register a url to match,a file to serve and a expiration time. what do you think of this idea.
+
**Swing UI (will not work if war deployed in a container)
 +
**Browser UI
 +
***Decision to go the Web UI route in M3
  
*Infrastructure Machines
+
*M4 new feature
** [henrik] Room for sonar install (we really need this)?
+
**Re-written update center
  
*[Duncan]Eclipse Migration
+
*Machine Infrastructure move
**SVN-Plugin decision. With the last discussions on this we had two options on the table.
+
**new machines setup and running
#Bundle the SVN plugin and dynamically load SVNKit on first run
+
**dedicated server box with 5 VMs
#Externalize this into GitHub rather than Eclipse managed core (Manfred was in favour of doing this with all the Source Control plugins)
+
**additional disk space for archiving etc will be available
**Groovy externalization
+
**some tuning on website side and little downtime at weekend for changes (will have litle/no impact - 10 minutes)
**Antlr upgrade 2.7 -> 3.n
+
**some library version issues on Hudson on Hudson - being upgraded for Git
**Jaxen - possible replacement?
+
**still using Confluence 3.0 at present - database is slightly corrupted, so export for migration to new version needs data cleanup. Duncan working on this.
 +
**Henrik would like to see Sonar installed - Duncan to check if there is a LGPL license issue on Oracle infrastructure with this.
 +
 
 +
*[henrik] I got two bugs submitted with patches,(https://bugs.eclipse.org/bugs/show_bug.cgi?id=371312  & https://bugs.eclipse.org/bugs/show_bug.cgi?id=371313) but I haven't had any reaction on them. Am I missing some step in the submission process or are people just busy with other tasks?
 +
**Winston will look at these (he has been working on the Eclipse Foundation performance issues).
 +
 
 +
*[henrik] we should clarify system requirements
 +
**Java version ?
 +
**Capabilities of container ? (Java EE 5 web container?)
 +
***Wiki page now clarifies Java 6
 +
 
 +
 
 +
*Eclipse problems
 +
**Winston did memory dumps over days
 +
**large requests on the webserver led to sluggish web pages
 +
**someone had tried to download very large file through Winstone (Hudson servlet) - very bad practice
 +
**they have moved from Winstone server to JD server and is now much faster
 +
***We need to edit core so that large downloads are not enabled in Winstone. As best practice should always have proxy with Apache and a way to set caching instance (in proxy some other way). Need to document this
 +
***Duncan to experiment on Hudson on Hudson and see if mod_cache improves this
 +
**Using NFS that was very slow - Eclipse changed their NFS system and had good results
 +
**Memory issue - Hudson creates huge trace kept in memory - over period of days (90% memory) becomes extremely slow - for large enterprise cannot keep all in memory.
 +
***Need to implement MRU (most recent used) type solution - Steven Christou has agreed to look into this
 +
***with lightweight metadata - not full build information
 +
***need to ensure that plugin developers understand this and do not write plugins that hurt Hudson performance
 +
 
 +
*[henrik] performance work
 +
**I have seen high load causing lock congestion in java.text.MessageFormat / java.util.Concurrency (see https://bugs.eclipse.org/bugs/show_bug.cgi?id=372679) I have tried to look at the code but I cannot seem to find a nice way around it. first of it is stapler features, secondly the messageformat class seems to always instantiate the sub formats, so no way to cache them. Any ideas?
 +
***Winston and Henrik to discuss this further over email
 +
 
 +
**When Hudson is stressed it gets really slow at serving files e.g. downloading a artifact is 5-10 times slower than loading in apache/scp. How do we best debug this, so that core developers have something to go on, and how can we improve the situation ? (can we siwtch to nio for master/slave communication? or use better streaming format?)
 +
**I suggest that many of the resources which does not change often, is written to disc and served from there instead of being processed/calculated on every request. I suggest we create a "component" (service) where plugins can register a url to match,a file to serve and a expiration time. what do you think of this idea.
 +
***Winston and Henrik to discuss this further over email
 +
 
 +
*Winston thanked Steven for all the work they've done and are committing to for the future!
 +
 
 +
Attendees:
 +
Henrik Lynggard Hansen
 +
Paul Hartenstine
 +
Duncan Mills
 +
Winston Prakash
 +
Susan Duncan
 +
Manfred Moser
 +
Steven Christou (and team)

Latest revision as of 14:11, 5 March 2012

Hudson Continuous Integration Server
Website
Download
Community
Mailing ListForumsIRC
Bugzilla
Open
Help Wanted
Bug Day
Contribute
Browse Source
Hudson-bust.png Community Meeting 05-March-2012











[edit] Notes

Add or comment on agenda items here, just prefix the item with your name so we can identify you on the call:

  • 2.2.1

There have been some P3 bugs fixed. No other core issues/mails request sent. There are 2 requests for github plugin that we need to fix as blockers. Henrik has seen this on Hudson on Hudson too Steven Christou is interested in looking at Github

  • 3.0.0

Groovy has now been removed from core Layer of scripting support created - for Groovy or other scripting tool There is a hardcode dependency on Stapler - needs forking and hardcode dependency removed Stapler fork (at 1.1.55) and scripting layer need approval from Eclipse then can release M2. Should be no problem with provenance of that code

  • M2 - push back release to 16th April (can be edited if needed)

to allow for QA cycle of above to check how long Eclipse approval may take for Groovy/Stapler Duncan to ping Eclipse with heads-up

  • Eclipse Migration
    • Groovy - provenance issue, hence it's not approved in Eclipse (now removed from Hudson core)
    • Antlr - cannot get approval for current version, looker to move to Antler 3 (before RC). Winston also looking into removing it completely
    • Jaxen - rejected previously because of provenance, Duncan talking to author and may be able to solve. Some functionality might now be in JDK if we need to remove it. Not used directly by any code, brought in by other library
    • SVNKit - Externalise SVN plugin in github and provide a UI to install the bundled 'recommended' plugins first


  • M3 new feature - Install 'recommended' plugins either
    • Swing UI (will not work if war deployed in a container)
    • Browser UI
      • Decision to go the Web UI route in M3
  • M4 new feature
    • Re-written update center
  • Machine Infrastructure move
    • new machines setup and running
    • dedicated server box with 5 VMs
    • additional disk space for archiving etc will be available
    • some tuning on website side and little downtime at weekend for changes (will have litle/no impact - 10 minutes)
    • some library version issues on Hudson on Hudson - being upgraded for Git
    • still using Confluence 3.0 at present - database is slightly corrupted, so export for migration to new version needs data cleanup. Duncan working on this.
    • Henrik would like to see Sonar installed - Duncan to check if there is a LGPL license issue on Oracle infrastructure with this.
  • [henrik] we should clarify system requirements
    • Java version ?
    • Capabilities of container ? (Java EE 5 web container?)
      • Wiki page now clarifies Java 6


  • Eclipse problems
    • Winston did memory dumps over days
    • large requests on the webserver led to sluggish web pages
    • someone had tried to download very large file through Winstone (Hudson servlet) - very bad practice
    • they have moved from Winstone server to JD server and is now much faster
      • We need to edit core so that large downloads are not enabled in Winstone. As best practice should always have proxy with Apache and a way to set caching instance (in proxy some other way). Need to document this
      • Duncan to experiment on Hudson on Hudson and see if mod_cache improves this
    • Using NFS that was very slow - Eclipse changed their NFS system and had good results
    • Memory issue - Hudson creates huge trace kept in memory - over period of days (90% memory) becomes extremely slow - for large enterprise cannot keep all in memory.
      • Need to implement MRU (most recent used) type solution - Steven Christou has agreed to look into this
      • with lightweight metadata - not full build information
      • need to ensure that plugin developers understand this and do not write plugins that hurt Hudson performance
  • [henrik] performance work
    • I have seen high load causing lock congestion in java.text.MessageFormat / java.util.Concurrency (see https://bugs.eclipse.org/bugs/show_bug.cgi?id=372679) I have tried to look at the code but I cannot seem to find a nice way around it. first of it is stapler features, secondly the messageformat class seems to always instantiate the sub formats, so no way to cache them. Any ideas?
      • Winston and Henrik to discuss this further over email
    • When Hudson is stressed it gets really slow at serving files e.g. downloading a artifact is 5-10 times slower than loading in apache/scp. How do we best debug this, so that core developers have something to go on, and how can we improve the situation ? (can we siwtch to nio for master/slave communication? or use better streaming format?)
    • I suggest that many of the resources which does not change often, is written to disc and served from there instead of being processed/calculated on every request. I suggest we create a "component" (service) where plugins can register a url to match,a file to serve and a expiration time. what do you think of this idea.
      • Winston and Henrik to discuss this further over email
  • Winston thanked Steven for all the work they've done and are committing to for the future!

Attendees: Henrik Lynggard Hansen Paul Hartenstine Duncan Mills Winston Prakash Susan Duncan Manfred Moser Steven Christou (and team)