CDT/Obsolete/ScannerDiscovery61/Stories

The Stories and The Alpha Version of Scanner Discovery for CDT 8.0

Disclaimer

I posted patch Work In Progress Alpha version of Scanner Discovery for CDT 8.0 to 290631. This is generally working version but may be rough on the edges. The code is far from clean and not suitable for commit.

There is a lot of moving parts and connection points to wire to CDT. I may have missed some - if you notice let me know. So far my biggest concern was to connect the moving parts and keep it from falling apart. Big thanks to git and its rebase feature BTW. UI is pretty much experimental and needs to be reworked from interaction design standpoint and consolidated with existing one. I shall go again along the stories posted trying to streamline the flow from user POV. If your scenario is missing, please let me know.

So far there was not much of feedback and I'd be happy to get any at the Summit. I won't attend, so please post thoughts in 290631. If feedback is positive, my plan is to add this to CDT 8.0 peacemeal. I would split it to smaller pieces and work on each one under a separate bugzilla.

Update: The project is hosted currently on GitHub. Please, refer there https://github.com/angvoz/SD80/ for the most recent API and code. Use branch [sd80] for bleeding edge code with more advanced UI and [sd80.Patch#.CDT] to take a look at clean patches intended for CDT submission. If you want to try it, the functionality is still experimental and disabled by default. To enable:

• first select "Display Providers" tab in Preferences->C/C++->Property Pages Settings,
• then in project properties C/C++ General->Paths and Symbols you'll get "Providers" tab. Click "I want to try..." check box and use "Configure" button to add providers to the configuration.

To define your own provider refer to Providers_Defined_via_Extension_Point and Dynamic_Providers.

Legacy Projects

Special care is being taken to keep the choice of using legacy Scanner Discovery mechanism. That choice could be the default until the new functionality is stable enough. There is a checkbox in New Project Wizard to enable/disable it on project creation (see #Regular Use). The new stuff is enabled by default in demo version though - solely for the demo purpose.

Regular Use

For the regular user using project types and toolchains supported by CDT, no any special setup is necessary.

Scanner discovery for a project is defined when the project is created by New Project Wizard. It is possible to enable the new "alpha" version or keep using legacy method.

If user is using legacy discovery it is still possible to use project properties to use new scanner discovery providers (see [LanguageSettingsProviders]).

Boost Story

Here is an example how you would setup Boost libraries project in eclipse. This does not require any special settings related to Scanner Discovery, just setting it up to compile and bjam option to provoke compiler command in output.

1. Download latest boost library from http://sourceforge.net/projects/boost/files/boost/1.44.0/
2. Download also bjam from there
3. Unzip the source somewhere outside the workspace and copy bjam.exe into the source root.
4. Using wizard "Makefile Project with Existing Code" create a new Makefile project with appropriate toolchain (MinGW in this example) referring to the folder with boost sources.

   

5. In project properties change settings to be able to compile:
   • In Tool Chain Editor select "Gnu Make Builder"
   • In C/C++ Build/Builder Settings: Use External Builder
   • Uncheck "Generate Makefiles automatically"
   • Uncheck "Use default build command"
   • Set "Build Command" to "bjam -d+2 --toolset=gcc --with-python". This is to compile python library only. Option "-d+2" is needed to get compiler command printed in the output. If you add "-na" options it will print the output but not compile.
   • In Behaviour Tab remove "all" from "Build (Incremental build)" target.
   • Save settings.
6. Build the project.

The project may not compile successfully everything but it is not the point here. It generates enough build output to demonstrate how scanner discovery picks compilation options and assigns them to proper resources in the project.

The settings appear in each file properties->Paths and Symbols->All Language Settings tab. You can see which language settings providers are used and which settings the providers retrieved.

Integrator Story

This is not really a story but rather some pointers about settings under the hood. I think the integrators could use that short description.

Similarly to ErrorParsers there is a list of available language settings providers. They provide the language settings for indexer use (corresponding to Include paths -I compiler options, predefined preprocessor macros -D options and others). There are several providers available, each one for a separate purpose. An integrator can pick and configure providers on per-project-configuration basis. The providers are defined as extensions in plugin.xml.

• There are 2 special providers, one is User provider which supports adding/editing user entries and MBS provider which represents setting entries supplied by MBS.

• GCC Build Output Parser is responsible for parsing build output and assigning individual language settings for source files (see #Boost Story).

• GCC Builtin Settings Detectors fetch built-in compiler settings. That kind of provider would be normally set as global instance in workspace and run only once. The language settings it retrieved would be shared between different projects.

• The initial list of providers is defined by a Toolchain and Configuration as an additional attribute in buildDefinitions extension point and the providers are assigned to each configuration during initial creation of CDT project.

Compiler Upgrade Story

One of the painful stories related to scanner discovery is the story of compiler upgrade. Even if we finally provided UI to clean up discovered entries in 7.0.1/8.0 (bug 206372), it is hard for user to discover it or figure out how to use it.

The UI in the new proposed version is arguably not ideal at that point, but the story itself is a good example to illustrate another feature the new design.

The compiler language settings are kept by corresponding compiler Builtin Settings Detector. Normally that kind of language settings provider is defined on workspace level and the settings retrieved only once for different projects to reuse. To re-run the detector select it in Preferences->C/C++ Build->Discovery tab and use "Clear" button. After the settings cleared, you could do "Run Now" or it would be run automatically with the first build.

Providers Defined via Extension Point

Often projects define a set of include paths and predefined preprocessor macros to apply as -I and -D options for every file during compilation. It makes sense to dedicate a special language settings provider as extension point in plugin.xml. Take a look at sample provider defined in org.eclipse.cdt.core.tests:

Here is how the language settings will appear in Paths and Symbols (after you add it to the project in C/C++ Build Settings Discovery tab):

After a language settings provider is defined it can become part of definition of a Toolchain and Configuration in MBS in buildDefinitions extension point.

"Dynamic" Providers

#Providers Defined via Extension Point demonstrates how to add "static" providers where language settings entries never change. For more complex scenarios one may have a need for "dynamic" entries which are calculated depending on some external conditions. To do that one would need to define a class implementing ILanguageSettingsProvider (or extending one of a few specialized abstract classes like AbstractBuildCommandParser or AbstractBuiltinSpecsDetector). Then add it to plugin.xml as LanguageSettingsProvider extension point. The interface is defined as following:

public interface ILanguageSettingsProvider {
	public String getId();
	public String getName();
	public List<ICLanguageSettingEntry> getSettingEntries(ICConfigurationDescription cfgDescription, IResource rc, String languageId);
}

One would implement method getSettingEntries(cfgDescription, rc, languageId) to return dynamically calculated entries.

Pkg-config Story

There is an old request to be able to import pkg-config provided build options, see bug 290631. For example, following output:

$ pkg-config --cflags gtk+-2.0

-I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12  

The possible approach to that enhancement would be as follows. There are 2 abstract classes defined in build plugin, AbstractBuildCommandParser and AbstractBuiltinSpecsDetector. AbstractBuildCommandParser is intended for providers parsing regular build output line by line and AbstractBuiltinSpecsDetector is for providers run once during each build (they could define a command to run). It looks like AbstractBuiltinSpecsDetector is more suitable in this case. So, it goes this way:

1. Define the provider extending AbstractBuiltinSpecsDetector in plugin.xml with org.eclipse.cdt.core.LanguageSettingsProvider extension point. There is "parameter" attribute. Enter there "pkg-config --cflags gtk+-2.0". If necessary, use "language-scope" to restrict languages where it is applicable.
2. In the new class, implement processLine(String line) to parse the line, create appropriate ICLanguageSettingEntry and populate with it List<ICLanguageSettingEntry> super.detectedSettingEntries. AbstractBuiltinSpecsDetector will take care of the rest.
3. Optionally add the new provider to appropriate toolchain in buildDefinitions extension point (languageSettingsProviders attribute).

After the provider has been defined and added to the project, build it and doublecheck the entries in Paths and Symbols:

Similar approach could've been taken with AbstractBuildCommandParser should output of pkg-config be a part of regular build output.

XLC story

This story is to be written yet. The new Scanner Discovery (i.e. language settings providers) is a part of CDT core. It associates language id with language settings providers. However, it looks like XLC Toolchain is relying on input types which is managed build concept and is not available in CDT core. Since current UI is pretty much MBS-centric it gets away with it now. I plan to look at it more closely, perhaps the best way to get benefits of the new stuff for XLC toolchain is to wire it to the languages provided by the core.

Support for defining settings by SDK/Library (export?)

See http://wiki.eclipse.org/CDT/ScannerDiscovery61/LeosAddlThoughts and https://bugs.eclipse.org/bugs/show_bug.cgi?id=290631#c14

Different built-in settings per source file

This story was brought up on CDT Summit.

User-edited makefiles in Standard Makefile project can contain any set of commands to compile the project. There is a category of projects like that when different files are compiled with different compilers:

/usr/bin/gcc-4.5 -c -oA.o A.c

/usr/bin/gcc-3.4 -c -oB.o B.c

• I am making up this example, if somebody sends excerpt from a real life one, it is appreciated.

In this case built-in compiler settings need to correspond to the compiler which is actually being used. This is working in legacy scanner discovery for per-file type of profile, see bug 73607, although not without scalability issues.

More stories

If your story is not here yet, you are more than welcome to add it here. If we know your story we could make it flow smooth during development. If we don't know your story it may become difficult to adapt after development has finished and API is set in stone.