Jump to: navigation, search

CDT/ScannerDiscovery61/ExistingArch

Document the existing architecture from it's main client, the parser, to the individual providers.

Doug's Discoveries

I'm going about this bottom up, from the parser up to the points where the built-ins and the build output parsing is done.

  • At the core of the Scanner Discovery profile is the IScannerInfo interface. It has two methods, getIncludePaths, and getDefinedSymbols. It is used by a number of clients to determine the compiler options to help with parsing.
    • Note that for gcc, it's more than just -I and -D, the -m args for example can change the built-in defines. IScannerInfo is supposed to hide all that from the parser. Wonder if it should be hidden?
  • There is also an IExtendedScannerInfo that adds is additional methods for other command line options (mainly from gcc) that can affect the compile, e.g. macro files, include files, and local include paths.
  • The IScannerInfo type hierarchy is very interesting. e.g. there are two ScannerInfo classes.
  • IScannerInfoProvider gets a IScannerInfo for a particular IResource.
  • For Makefile project, CCorePlugin.getScannerInfoProvider(project) returns a ScannerInfoProviderProxy. The proxy farms out operations to a DescriptionScannerInfoProvider (why is it a proxy?). In turn, the DescriptionScannerInfoProvider gets the scanner info from the project description (i.e. .cproject file). It returns an ExtendedScannerInfo.
  • The ExtendedScannerInfo objects seem to be created when the project description is loaded. Also created when CModel is built and when index is rebuilt. But cached after those operations.
  • ExtendedScannerInfo properties set from PathEntry properties in DescriptionScannerInfoProvider createProjectScannerInfo(), createScannerInfo()
  • PathEntry objects (ICLanguageSettingPathEntry) are created by ProfileInfoProvider from info found in PathInfo objects
  • PathInfo objects managed by CfgDiscoveredPathManager derived from IDiscoveredPathInfo.
  • IDiscoveredPathInfo can be per file or per project.
  • IDiscoveredPathInfo populated by ScannerInfoCollectors (e.g. DefaultGCCScannerInfoCollector).

Andrew's notes to himself

I've started another bottom-up, from UI DiscoveryTab. I'll try to untangle the code to the point when I can understand what is going on. Those would be my notes about details I found easy to forget. Hope to meet Doug somewhere up.

  • Random roaming
    • Discovery profile is defined as extension point org.eclipse.cdt.make.core.ScannerConfigurationDiscoveryProfile Org.eclipse.cdt.make.core.ScannerConfigurationDiscoveryProfile2.png
    • Scanner Discovery is (at least) 3-fold:
      • 1. buildOutputProvider (BOP) is connected to CommonBuilder (and possibly other builders) and parses its build output line by line. The collected entries (options -D, -I etc.) are kept temporarily by scannerInfoCollector
      • 2. scannerInfoProvider (SI) is implemented as a separate project builder which can be seen in project Properties->Builders->"Scanner Configuration Builder". It collects built-in compiler specs (also -D -I) running special command like "g++ -E -P -v -dD "${plugin_state_location}/specs.cpp". Those are also kept in scannerInfoCollector - momentarily. "Scanner Configuration Builder" then proceeds without delay with 3)
      • 3. scannerInfoCollector entries collected on 2 previous steps are persisted by "Scanner Configuration Builder" in project model and file .metadata/.plugins/org.eclipse.cdt.make.core/${ProjName}.sc
    • In MBS buildDefinitions extension point the profiles can be connected to toolchain or input type
    • One can see scanner discovery traces setting TraceUtil.SCANNER_CONFIG to true
    • It is easy to see that there are 2 types of profiles, those with scope per "project" and others per "file". Does anybody knows exactly what these are supposed to mean? Looks like Toolchain.isRcTypeBasedDiscovery is not related to "per file" profile. Debugging shows "per project" profile being applied for perRCTypeDiscovery in CommonBuilder.createBuildOutputSniffer().
    • "Discovery profiles scope". Project properties DiscoveryTab controls isRcTypeBasedDiscovery via "Discovery profiles scope". Switching scope there changes the way profiles are applied, i.e. sets Toolchain.isRcTypeBasedDiscovery.
      • If isRcTypeBasedDiscovery=true, createBuildOutputParser-s created for each combination resource(IResourceInfo)+input_type(IInputType). All of them are listening to build output provided rc+tool+input_type+cfg as a context. However, only default configuration is considered. Also, only worthy resources are considered (as determined by cfg.getResourceInfos()) - so far I've seen only root folder of the project in the list.
      • If isRcTypeBasedDiscovery=false, only one createBuildOutputParser is listening to build output provided default configuration as a context.
    • Class PreferenceInitializer does initialization of Discovery Options defaults for Makefile Project (Preferences->New CDT Project Wizard->Makefile Project)
    • Class SCProfileInstance instantiates classes defined in extension point ScannerConfigurationDiscoveryProfile
    • GCCSpecsRunSIProvider to get compiler specs is being called for every tool. However in long chain of calls that distinction is lost (in IDiscoveredPathManager.IDiscoveredPathInfo.loadPathInfo()). Only configuration is available there. (not 100% sure)
      • In CommonBuilder.contributeToConsoleParserList() InfoContext instance is formed where its ID contains semicolon separated list of IDs: "cfg.id;resource.id;tool.id;input.type.id"
    • CfgScannerConfigUtil.getDefaultProfileId() assigns "selected" profileId. Some logic there.
  • Where it is hooked up
    • The first time scanner discovery is run is on creating project description deep inside CProjectDescriptionManager.setProjectDescription(). CfgDiscoveredPathManager.loadPathInfo() calls ScannerConfigBuilder.build(). There seems to be intention to cache but setCachedPathInfo() calls are commented out.
    • One place where IExternalScannerInfoProvider providers are invoked is CfgSCJobsUtil.getProviderScannerInfo(). The providers are taken from IScannerConfigBuilderInfo2 buildInfo.getProviderIdList().
    • CommonBuilder.createBuildOutputSniffer() iterates over all input types for all tools and calls contributeToConsoleParserList() where IScannerInfoConsoleParser is created from profile and added to the list.
    • DiscoveredPathManager.updateDiscoveredInfo() would save collected info (in .metadata/.plugins/org.eclipse.cdt.make.core/Project.sc file) and (false lead?) fire event INFO_CHANGED.
    • CfgDiscoveredPathManager.updateCoreSettings() oops i got lost again...
  • Scanner Discovery and legacy new project model.
    • There are 3 ways of storing ICLanguageSettingEntry:
      • USER_ENTRIES_LEVEL. This one is easy to use: lang.setSettingEntries(ICLanguageSettingEntry.INCLUDE_PATH, new ICLanguageSettingEntry[] {entry}); Those are editable/removable in UI. They are converted to MBS Option and kept in .cproject. However BUILTIN flag is not supported.
      • ENV_ENTRIES_LEVEL. ManagedBuildManager.getEnvironmentVariableProvider() is used.
      • DISCOVERY_ENTRIES_LEVEL. ProfileInfoProvider.getEntryValues() would "getDiscoveredInfo()" and create ICLanguageSettingEntry[] using preset level's flags RESOLVED|READONLY|BUILTIN. Serialization is done in DiscoveredScannerInfoStore.getDocument() in .metadata/.plugins/org.eclipse.cdt.make.core/Project.sc
    • <storageModule moduleId="scannerConfiguration"> elements are liberally stored in .cproject by ConfigurationDataProvider.applyConfiguration() with CfgScannerConfigInfoFactory2.save() call.
    • CConfigurationDescriptionCache.applyData() dispatches storing action for .cproject.
    • As far as loading, the data are loaded to numerous *Cache classes and taken from there on request.
      • Language settings are cached naturally in CLanguageSettingCache. It would access it via core classes CConfigurationDescriptionCache->CFolderDescriptionCache->CLanguageSettingCache->EntryStore->KindBasedStore. KindBasedStore categorizes entries by ICSettingEntry flags, i.e. include paths, #defined macros, etc.
      • EntryStore.getEntries() is a useful hook for debugging to see the chains of calls.