Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Non Java projects Proposal"

(Proposal)
(Script Actor ( under discussion ))
 
(9 intermediate revisions by the same user not shown)
Line 32: Line 32:
 
This page is a proposal to extends Buckminster in order to bring users from the non Java world a better experience.
 
This page is a proposal to extends Buckminster in order to bring users from the non Java world a better experience.
  
=== Executor Actor ===
+
=== Executor Actor (now available)===
  
 
This actor would add the possibility to execute programs directly from the CSpec without writing Ant Scripts.
 
This actor would add the possibility to execute programs directly from the CSpec without writing Ant Scripts.
Line 71: Line 71:
 
* '''newenvironment''' : if set to true, the execution will start from a blank environment variables.
 
* '''newenvironment''' : if set to true, the execution will start from a blank environment variables.
 
** Note in early design development the opposite paradigm was chosen and one has to use '''inheritEnvVar''' to start from the system environment variables, now it's the default behavior.
 
** Note in early design development the opposite paradigm was chosen and one has to use '''inheritEnvVar''' to start from the system environment variables, now it's the default behavior.
 +
* '''failonerror''' : if unset or set to true, the action will fail if execution returns non 0 return value. Default is true.
 
* '''exec''' : the program to execute
 
* '''exec''' : the program to execute
 
* '''execDir''' : the program execution directory (if not set : the component directory)
 
* '''execDir''' : the program execution directory (if not set : the component directory)
Line 80: Line 81:
 
Those variables are set by surrounding properties or named path with '''${}'''. See examples [[Non_Java_projects_Proposal#Example|above]].
 
Those variables are set by surrounding properties or named path with '''${}'''. See examples [[Non_Java_projects_Proposal#Example|above]].
  
=== Fetcher Actor ===
+
=== Fetcher Actor (now available)===
  
 
This actor is used to materialize resources into the component. I'm still not sure this behavior should be provided by an actor or through the materialization process. It seems the actor can be used at will so it's more flexible than the materialization but this point can be discussed.
 
This actor is used to materialize resources into the component. I'm still not sure this behavior should be provided by an actor or through the materialization process. It seems the actor can be used at will so it's more flexible than the materialization but this point can be discussed.
Line 133: Line 134:
 
</source>
 
</source>
  
=== Script Actor ===
+
=== Script Actor ( [http://www.eclipse.org/newsportal/article.php?id=360&group=eclipse.tools.buckminster-dev#360 dismissed actually] )===
  
 
Builds a platform dependent shell script baking a set of environment variables. It is used to executes actions independently from Buckminster.
 
Builds a platform dependent shell script baking a set of environment variables. It is used to executes actions independently from Buckminster.
Line 143: Line 144:
 
* allow appending lines to the script
 
* allow appending lines to the script
 
* allow filtering of specific lines
 
* allow filtering of specific lines
* unset specific variables
+
* allow unsetting of specific variables
  
 
==== Example ====
 
==== Example ====
Line 191: Line 192:
  
 
* '''append.*''' : appends a line after the export section.
 
* '''append.*''' : appends a line after the export section.
** * is a number which is used to order the lines. ''We have no choice here has properties have unique keys.''
+
** * is a number which is used to order the lines. ''We have no choice here as properties have unique keys.''
 
** a line might be filtered by a filter.* property.
 
** a line might be filtered by a filter.* property.
 
* '''filter.*''' : filters the line * according to the [http://tools.ietf.org/html/rfc4515 LDAP filter specification].
 
* '''filter.*''' : filters the line * according to the [http://tools.ietf.org/html/rfc4515 LDAP filter specification].
Line 201: Line 202:
  
  
'''Unix scripts will be chmoded to a+x to unable execution.'''
+
'''Unix scripts will be chmoded to a+x to allow execution.'''
 +
 
 +
=== Echo Actor ( [http://www.eclipse.org/newsportal/article.php?id=360&group=eclipse.tools.buckminster-dev#360 under discussion] )===
 +
 
 +
Ouputs data to a file.
 +
 
 +
==== Features ====
 +
* write or append to file
 +
* input can come from
 +
** file on disk
 +
** property value
 +
** inner elements with CDATA sections
 +
 
 +
==== Example ====
 +
 
 +
<source lang="xml">
 +
<cs:public name="my.script" actor="echo">
 +
  <cs:actorProperties>
 +
      <cs:property key="append" value="true"/>
 +
      <cs:property key="input" value="text"/>
 +
      <cs:property key="some.key">these</cs:property>
 +
      <cs:property key="text"><![CDATA[
 +
#!/bin/bash
 +
# Here you can write virtually anything. Constructs like
 +
# ${some.key} would of course be expanded.
 +
export BOOST_INCLUDE = ${BOOST_INCLUDE}
 +
export BOOST_LIBS = ${BOOST_LIBS}
 +
${BJAM} $*
 +
]]></cs:property>
 +
  </cs:actorProperties>
 +
  <cs:prerequisites>
 +
      <cs:attribute name="bjam" alias="BJAM" component="boost-1.36.0"/>
 +
      <cs:attribute name="include" alias="BOOST_INCLUDE" component="boost-1.36.0"/>
 +
      <cs:attribute name="libs" alias="BOOST_LIBS" component="boost-1.36.0"/>
 +
  </cs:prerequisites>
 +
  <cs:products base="${buckminster.home}/script">
 +
      <cs:path path="script.sh"/>
 +
  </cs:products>
 +
</cs:public>
 +
</source>
  
 
[[Category:Buckminster]]
 
[[Category:Buckminster]]
 
[[Category:Buckminster Technology]]
 
[[Category:Buckminster Technology]]
 
[[Category:Buckminster Technology Ant]]
 
[[Category:Buckminster Technology Ant]]

Latest revision as of 11:52, 13 December 2008

< To: Buckminster Project
< To: Helping_Out_(Buckminster)
< To: Buckminster component meta-data language 1.0 (Reference)

Purpose

While trying to use Buckminster for both my Java and C++ projects, I realized that Java projects within Eclipse were pretty straightforward but C/C++ projects were much more difficult to manage.

Why is it so hard to use Buckminster with non Java projects ?

Component materialization is much harder in non Java environments because :

  • there are no standard way to describe C/C++ projects so you have to write the CSpec file anyway ( contrary to Eclipse Java projects which provides a way to auto generate the CSpec )
  • compiling, testing, your code requires to execute programs from outside ( shell scripts, compiler, unit tests and so forth ).
  • some of the resources you need to materialize are libraries that are only available as zipped url resources.
    • which implies you have to download and unzip them : Buckminster cannot do that for the moment despites its exactly what you expect it to do. So to overcome this issue you have to write ant tasks.


Work flow currently used

The toolchain I created to manage C/C++ projects dependencies is as follows :

  • Create Spec components to describe dependencies
  • Specify actions like build / clean / rebuild
    • those actions have prerequisites pointing to other components (eg: the path to the libTiff include folder)
  • Write ant scripts called from buckminster to
    • retrieve libraries from urls or retrieve files from archives pointed by url
    • execute commands with specific environment variables (eg. calling the compiler with paths to libraries)
  • Create the script to compile the code ( Makefile or Boost Jamfile or SCons file )


Proposal

This page is a proposal to extends Buckminster in order to bring users from the non Java world a better experience.

Executor Actor (now available)

This actor would add the possibility to execute programs directly from the CSpec without writing Ant Scripts.

Features

  • Executes programs or shell scripts
  • Sets arguments
    • arguments can refer to prerequisites
  • Sets environment variables for the execution
    • environments variables values can refer to prerequisites
  • Sets execution directory

Example

<cs:public name="build" actor="executor">
      <cs:actorProperties>
            <cs:property key="newenvironment" value="true"/>
            <cs:property key="env" value="TIFF_LIB=${tiff.lib};TIFF_INCLUDE=${tiff.include};
                                          VERSION=${VERSION};OSGI_VERSION=${buckminster.version}"/>
            <cs:property key="shell" value="${compiler} -j2 debug"/>
      </cs:actorProperties>
      <cs:properties>
            <cs:property key="VERSION" value="1.0.0"/>
      </cs:properties>
      <cs:prerequisites>
            <cs:attribute component="mingw-5.1.4" name="compiler" alias="compiler" />
            <cs:attribute component="libtiff-3.6.1" name="include" alias="tiff.include" />
            <cs:attribute component="libtiff-3.6.1" name="lib" alias="tiff.lib" />
      </cs:prerequisites>
</cs:public>

Valid properties

This syntax integrates into Buckminster's CSpec Actions.

  • env : a list of semi-colon separated environment variable
  • newenvironment : if set to true, the execution will start from a blank environment variables.
    • Note in early design development the opposite paradigm was chosen and one has to use inheritEnvVar to start from the system environment variables, now it's the default behavior.
  • failonerror : if unset or set to true, the action will fail if execution returns non 0 return value. Default is true.
  • exec : the program to execute
  • execDir : the program execution directory (if not set : the component directory)
  • shell : a batch file to execute

Note that you have to choose between exec or shell keyword.

All those properties can contain variables.

Those variables are set by surrounding properties or named path with ${}. See examples above.

Fetcher Actor (now available)

This actor is used to materialize resources into the component. I'm still not sure this behavior should be provided by an actor or through the materialization process. It seems the actor can be used at will so it's more flexible than the materialization but this point can be discussed.

Features

  • fetch single (for the moment)
  • log to https sites
  • unzip, untar, untargz, untarbz2 files (add 7zip support ?)
    • possibility to specify destination directory
    • possibility to filter the archive content
    • keep hierarchy or flatten folders

Example

<cs:public name="download" actor="fetcher">
   <cs:actorProperties>
      <cs:property key="dir" value="data"/>
      <cs:property key="options" value="uncompress;flatten;include=*.html;include=*.c"/>
      <cs:property key="url" value="http://download.osgeo.org/libtiff/tiff-3.8.2.tar.gz"/>
   </cs:actorProperties>
</cs:public>

Valid Properties

  • url : the resource to fetch
  • dir : the directory where the file(s) should be placed
    • if not set dir points to component's location


If the resource you are accessing is protected with basic authentication, you can also specify :

  • login : the login to use with basic authentication
  • pass : the password to use with basic authentication

Note that both properties have to be set to enable authentication.


  • options : a set of options to customize the fetch operation, valid options are :
    • uncompress : specify that the resource pointed by the url is an archive and that it should be uncompressed
    • flatten : valid only with uncompress. If specified the folder hierarchy is flatten
    • include : a regular expression to include specific files only. Note that you can specify multiple include directives in the options line. Regular expressions beginning with the '-' (minus sign) will be considered as an exclusion. See examples below :
Options samples

full options sample, uncompress the archive and flattens the folder hierarchy, including only files in the binaries directory that are not html files

uncompress;flatten;include=binaries/*;include=-*.html

flatten and include keywords will NOT work until the uncompress keyword is set, the following options line will simply copy the url resource locally.

flatten;include=binaries/*;include=-*.html

Script Actor ( dismissed actually )

Builds a platform dependent shell script baking a set of environment variables. It is used to executes actions independently from Buckminster. Sure, you will loose the dynamic aspect of Buckminster's actions but you'll get faster executions.

Features

  • sets the environment variable according to the platform
  • action's prerequisites are automatically set in the script
  • allow appending lines to the script
  • allow filtering of specific lines
  • allow unsetting of specific variables

Example

<cs:public name="my.script" actor="script">
   <cs:actorProperties>
      <cs:property key="append.0" value="echo Will be displayed anyway"/>
      <cs:property key="append.1" value="echo Will be displayed on windows"/>
      <cs:property key="append.2" value="echo Will be displayed on linux"/>
      <cs:property key="append.3" value="echo BJAM=${BJAM}"/>
      <cs:property key="filter.1" value="(target.os=win32)"/>
      <cs:property key="filter.2" value="(target.os=linux)"/>
      <cs:property key="script.dir" value="data"/>
      <cs:property key="script.linux.shell" value="/bin/bash"/>
      <cs:property key="script.name" value="script"/>
      <cs:property key="unset" value="BJAM"/>
   </cs:actorProperties>
   <cs:prerequisites>
      <cs:attribute name="bjam" alias="BJAM" component="boost-1.36.0"/>
      <cs:attribute name="include" alias="BOOST_INCLUDE" component="boost-1.36.0"/>
      <cs:attribute name="libs" alias="BOOST_LIBS" component="boost-1.36.0"/>
   </cs:prerequisites>
</cs:public>
  • On windows, would produce the file data/script.bat
set BOOST_INCLUDE="C:\PATH\TO\BOOST_INCLUDE"
set BOOST_LIBS="C:\PATH\TO\BOOST_LIBS"
echo Will be displayed anyway
echo Will be displayed on windows
echo BJAM="C:\PATH\TO\BJAM"
  • On linux, would produce the file data/script.sh
#!/bin/bash
export BOOST_INCLUDE="/PATH/TO/BOOST_INCLUDE"
export BOOST_LIBS="/PATH/TO/BOOST_LIBS"
echo Will be displayed anyway
echo Will be displayed on linux
echo BJAM="/PATH/TO/BJAM"

Valid Properties

  • append.* : appends a line after the export section.
    • * is a number which is used to order the lines. We have no choice here as properties have unique keys.
    • a line might be filtered by a filter.* property.
  • filter.* : filters the line * according to the LDAP filter specification.
    • the line is appended only if the filter returns true.
  • script.dir : the directory where the script should be placed
  • script.linux.shell : used on linux to set the shebang.
  • script.name : sets the script filename (extension will be appended automatically).
  • unset : a comma separated list of alias not to set.


Unix scripts will be chmoded to a+x to allow execution.

Echo Actor ( under discussion )

Ouputs data to a file.

Features

  • write or append to file
  • input can come from
    • file on disk
    • property value
    • inner elements with CDATA sections

Example

<cs:public name="my.script" actor="echo">
   <cs:actorProperties>
      <cs:property key="append" value="true"/>
      <cs:property key="input" value="text"/>
      <cs:property key="some.key">these</cs:property>
      <cs:property key="text"><![CDATA[
#!/bin/bash
# Here you can write virtually anything. Constructs like
# ${some.key} would of course be expanded.
export BOOST_INCLUDE = ${BOOST_INCLUDE}
export BOOST_LIBS = ${BOOST_LIBS}
${BJAM} $*
]]></cs:property> 
   </cs:actorProperties>
   <cs:prerequisites>
      <cs:attribute name="bjam" alias="BJAM" component="boost-1.36.0"/>
      <cs:attribute name="include" alias="BOOST_INCLUDE" component="boost-1.36.0"/>
      <cs:attribute name="libs" alias="BOOST_LIBS" component="boost-1.36.0"/>
   </cs:prerequisites>
   <cs:products base="${buckminster.home}/script">
      <cs:path path="script.sh"/>
   </cs:products>
</cs:public>

Back to the top