The RTSC Flow

RTSC overlays new capabilities for specifying, delivering, and assembling real-time software components atop the familiar steps of compiling and linking C libraries and programs. This resulting RTSC flow — spanning the full component life-cycle, from production to consumption — comprises five major phases described in detail below:

     Specification
     Implementation
     Packaging
     Configuration
     Analysis

Specification

Acknowledging the limitations of C, RTSC introduces a special-purpose language for specifying programmatic contracts in a more expressive and robust manner than otherwise possible with .h header files. Designated XDCspec in the figure above, RTSC's specification language has its roots in the IDLs (interface definition languages) first introduced with COM and CORBA in conjunction with C++ several decades ago. As you will eventually learn, XDCspec source files serve as a higher-level, more evolved form of the familiar C header; and underlying tools will automatically translate XDCspec sources into corresponding .h files. The XDCspec language also supports special doxygen-like comments and markup tags used to automatically generate client documentation in a variety of formats.

You might find it interesting that contemporary languages like Java and C# favor a single source-file approach, suggesting a single notation can cover specification as well as implementation of software components.

But XDCspec sources are more than "cleaned-up" header files that define and document APIs in terms of client-visible constants, types, and functions. The underlying language also elevates three fundamental constructs to keyword status, enabling better support in standard C for what others have termed programming-in-the-large:

module: a cohesive collection of constants, types, and functions; akin to a class in object-oriented languages like C++ or Java, a RTSC module maintains a crisp boundary between a public, client-visible spec (expressed in XDCspec) and a private, supplier-proprietary body (typically expressed in C); modules serve as primary unit of implementation within RTSC.

interface: an abstract module; mirroring similarly-named constructs found in early IDLs as well as contemporary programming languages, a RTSC interface is effectively a module spec minus the module body; unlike modules, however, an interface is open in that others can extend it (XDCspec definitions can inherit from one another) and ultimately implement it through a concrete (and closed) module.

package: a namespace containing module and interface specs, each in turn defining a scope populated with named constants, types, and functions; a RTSC package caps our two-tier programming-in-the-large hierarchy, enabling us to aggregate 1000s of functions into a more tractable number of named programmatic entities; packages also serve as the elemental unit of delivery between suppliers and clients.

Implementation

Unlike other component frameworks which (practically speaking) demand use of an object-oriented programming language, we recognize that C remains the language of choice for embedded programming and have tailored RTSC accordingly. Having said that, you can implement RTSC modules using C++ (which meshes well with C) as well as using assembly code that respects the underlying C compiler's runtime conventions; and even the use of C can range from a highly-portable implementation to one that leverages compiler-specific extensions. In general, though, RTSC promotes a very clean, crisp, and portable style of C programming that you will find both easy-to-read and fun-to-write — a style that emphasizes correctness and robustness as the first order of business, and yet can yield highly optimized implementations using modern compiler technology.

But looking back at the figure, you may have also noticed a companion implementation language labeled XDCscript. In general, all RTSC modules maintain implementations in two complementary programming domains:

a target-domain, where C-based functions are eventually bound into application programs executing on specific hardware platforms; target-functions will typically respect the time/space/power constraints imposed by most embedded systems; and

a meta-domain, where corresponding XDCscript meta-functions play an active role in the design-time configuration as well as the run-time analysis of target programs; meta-functions execute on host computers with virtually unlimited resources, enabling module suppliers to factor-out critical functionality from more resource-constrained target implementations.

To fulfill the role of the XDCscript meta-language, RTSC leverages Mozilla's Rhino — an openly-available implementation of the ECMA-262 language standard, popularly known as JavaScript. Easy-to-learn because of its inherent simplicity plus its syntactic resemblance to C, Rhino/JavaScript already serves as the textual configuration language for the widely-deployed DSP/BIOS kernel; inclusion in RTSC as a general-purpose meta-language ensures continuity here.

Once again, were we working in Java or C# (which support on-the-fly reflection and introspection of executing code) the implementation language and the meta-language effectively become one in the same.

Packaging

RTSC packages play a pivotal role in managing software content throughout its lifespan, from initial component production through final component consumption. More than a programmatic namespace for modules/interfaces declared in XDCspec sources, each RTSC package has a concrete manifestation as a directory that adheres to a prescribed set of organizational conventions: packages are physical as well as a logical containers. But above all, RTSC packages (logically and physically) remain indivisible entities from start to finish: packages are built as a unit; packages are released as a unit; and packages are deployed as a unit.

Building Packages. From the supplier's perspective, a RTSC package directory will house source files in multiple languages that specify and implement the package's constituent modules/interfaces. Guided by a specialized build script — itself expressed in the XDCscript meta-language — underlying tools first translate XDCspec module/interface specs into corresponding .h files and then compile C-based module implementations into object-code libraries. In general, package build scripts may prescribe that a common set of (portable) C source-files be compiled multiple times:

using the same underlying compiler but with different options; or
using a different compiler for each generated library.

To help manage this flow, each library designated in the build script must be associated with a RTSC target — a formal XDCscript recipe for how to invoke a particular C compiler tool-chain. By separating "what to build" from "how to build it", package build scripts written in XDCscript achieve the same levels of clarity and portability that RTSC encourages with target-domain C code.

Releasing Packages. Supplier-side build scripts can also prescribe the contents of a RTSC package in its released form(s), each being distinctly named compressed directory files ready for consumption by downstream clients:

one release of the package may contain buildable sources;
another release may only ship binaries;
one release of the package may contain libraries for all targets;
another release may only support a subset;
and so on, as an exercise for the package supplier.

Automatically-generated manifests delivered with each released package record dependencies of constituent modules/interfaces on other packaged content. The package manifest also records compiler versions used to build target libraries, important for downstream validation during program configuration. Finally, suppliers can explicitly tag their packages with a compatibility key that reflects whether a given package's content is binary-compatible, source-compatible, or incompatible with a previously released version.

Deploying Packages. Other than de-compressing RTSC packages delivered (say) as .tar or .zip files, clients never alter the structure or contents of the original package directory; even the name of this directory is fixed, reflecting the original logical name of the package in an obvious way. Clients do, however, control, the ultimate deployment of each RTSC package within their own file-system:

clients create and manage arbitrary directories termed package repositories into which (de-compressed) RTSC packages are installed; and

clients define and manage their package path — an ordered list of repository directories — used to locate individual files named relative to their containing package.

Both command-line and GUI tools exist to streamline administration of package repositories as well as composition of the package path. These tools also validate the completeness and consistency of the current set of deployed packages, using the manifest and compatibility key associated with each.

Though new to many C programmers, RTSC's three-tier deployment hierarchy — package directory, package repository, package path — is virtually identical to a physical design first popularized by Java a decade ago for locating class files. Most contemporary programming and scripting languages now employ some variant of this scheme, which has amply proven itself to scale across millions of content suppliers. When in doubt, steal shamelessly from the success of others.

Configuration

As illustrated in the figure, configuration plays a central role in meshing a set of deployed packages containing C headers and libraries with the traditional flow of compiling/linking embedded programs that execute on a particular hardware platform. Orchestrated as a "pre-build" step using the client's own build environment (an IDE, make, scons, ...), RTSC configuration starts with a meta-program — an XDCscript fragment that ultimately guides the integration of target content residing in a designated set of deployed packages. Leveraging the full resources of the client's host computer, RTSC configuration provides an integration-time staging-area in which we can robustly validate the specific combination of target-domain elements prescribed in the XDCscript meta-program as well as optimally generate downstream C compiler/linker inputs tailored for the application at hand.

Prescribing Program Needs. Besides explicitly identifying all modules required by the target-program, the corresponding XDCscript meta-program input to the process can selectively assign configuration parameters ("configs" for short) declared in the module's original XDCspec source file. One of RTSC's most distinctive features, configs are essentially integration-time variables in the meta-domain that eventually become run-time constants in the target-domain. Knowledge of these config values prior to downstream compiling/linking is also key to enabling new levels of application-specific optimization for embedded C programs.

Syntactically similar to a C variable definition, each config belongs to a specified type and optionally has a default value of that type. And like C, a config's type can range from built-in elementary types (Int, Bool, String) to supplier-defined aggregate types involving arrays, structures, pointers, and even references to other modules. RTSC's XDCscript meta-language extends the underlying Rhino/JavaScript language here by automatically checking for type-compatibility whenever scripts assign new values to a config.

Once the client's XDCscript meta-program completes, RTSC configuration systematically invokes a series of special XDCscript meta-functions implemented by each module required by the client's target-program. One of these meta-functions enables individual modules to themselves designate other required modules as well as assign other configs — not unlike the top-level meta-program. This recursive process eventually converges upon a rich XDCscript dataset that serves as a program model — a blueprint for how independent modules from multiple suppliers will eventually co-exist within the target application program.

Validating Program Needs. With a top-to-bottom program model now in place, RTSC configuration enables each constituent module to validate the model's contents from its own vantage, by implementing a special XDCscript meta-function that can raise errors or emit warnings when it detects inconsistencies or incompatibilities. Automatic validation of overall package-to-package compatibility using accompanying manifests produced upon package release happens here as well.

The decision to move ahead and enable downstream compiling/linking at this stage in the configuration process must be unanimous across all participating packages and their modules. The net result becomes early detection of serious problems that (at best) may yield obscure downstream compiler/linker errors or else (at worst) may trigger run-time behavior with fatal consequences.

Generating Program Artifacts. Once validated, the program model's content drives generation of several files consumed by downstream tooling. And once again, RTSC configuration invites each constituent module to actively participate by implementing special XDCscript meta-functions. One such file contains linker commands that enumerate the correct set of target module libraries for this program configuration, as well as overlay the program's logical code/data sections onto the platform's physical memory map.

Just as we've already employed formal XDCscript recipes to abstract the use of different compilers when building different target libraries, RTSC platforms likewise help unify the process of linking/loading programs for diverse embedded hardware. Effectively another input to RTSC configuration (together with a compatible RTSC target used for downstream compilation), the platform's description becomes accessible to any XDCscript meta-function involved in creating or validating the program model. Applying a file-generation template coordinated between the prospective program's target and platform, RTSC configuration yields a linker command-file tailored to the specific requirements of the application at hand.

Last but not least, RTSC configuration generates a C source file comprising:

statically initialized constants reflecting each config's final value in the program model;

internal state-structures for each module, initialized with help from special XDCscript meta-functions implemented by the module supplier;

small code trampolines that bind client-side calls of spec'd module functions to the supplier's corresponding internal implementation (with additional run-time code for parameter validation, event logging, or statistics accumulation inserted under the control of special per-module configs); and

miscellaneous fragments of C code/data synthesized through optional module-specific templates which otherwise enjoy full access to the XDCscript program model.

Optimizing Program Performance. Automatic compilation of this generated C file followed by linking against the appropriate module libraries then occurs, guided by the RTSC target associated with this configuration. With a growing number of contemporary C compilers now supporting whole-program optimization — enabling the compiler to look across multiple source files when generating object code — the initialized constants and small code fragments in our generated C file will typically "melt-away" in the final executable image by applying classic optimizations such as constant folding and function inlining across independent module boundaries.

Supported in all TI compilers as well as the latest GCC and Microsoft offerings, whole-program optimization plays an important role in enabling a higher-level breed of C programming without compromising overall system performance.

Automating Program Integration. As a final point, we can't stress enough that the RTSC configuration process is an automated one, supplanting past practices in which customers were expected to hand-edit files of linker commands or C code/data in an ad hoc manner. Put another way, RTSC configuration effectively automates many of the manual steps otherwise described in (unread!) README files accompanying legacy C libraries.

RTSC configuration gives component consumers the ability to prescribe their intent in a higher-level format — an XDCscript meta-program — without sweating the low-level (and potentially error-prone) details of consolidating back-end artifacts. Component producers also benefit here, since each module they supply can encapsulate knowledge of optimal parameter settings and code selection through accompanying XDCscript meta-functions invoked in a standardized manner throughout the configuration process.

Analysis

Each RTSC module is effectively a black-box that publicizes a spec to its clients while hiding the implementation choices of its supplier — a "good thing" for clients architecting richly layered software that is resiliant to change, but potentially a bad thing for clients when something goes awry during execution. To help with program debug in the lab, with system test in the factory, or even with product diagnostics in the field, each RTSC module has the opportunity to support several distinct modes of run-time analysis that again leverage a skillful partitioning between a resource-constrained target-domain and a host-based, resource-rich host-domain.

Analyzing Program/Module State. Just as modules can actively participate in the design-time synthesis of application programs through RTSC configuration, XDCscript meta-functions flanking a module's target-domain C implementation can play an vital role in the run-time analysis of these programs once execution has commenced (and perhaps even terminated). Given the ability to read the program's data memory — via a host-based debugger, an embedded program monitor, or even a snapshot of program state saved to disk — RTSC analysis first decodes target-domain module state-structures into equivalent meta-domain objects and then calls special XDCscript meta-functions on a per-module basis to optionally transform these objects into a more meaningful view of the underlying state.

While code involved in formulating the client's ultimate view of module state may have migrated from the resource-constrained target-domain to the richer world of host-based XDCscript programming, internal implementation details (e.g., the organization of state structures) continue to remain hidden from a module's client; the integrity of the module supplier's contract with the client remains intact.

Analyzing Program/Module Events. Complementing this state-oriented perspective, another dimension of RTSC analysis involves merging, filtering, and interpreting streams of time-stamped events sourced during the course of program execution. The events themselves could be retained in target-side memory or else transmitted to another processor at run-time using an appropriate output device on the underlying hardware platform.

Whereas state-oriented analysis involves no additional run-time code (except, perhaps, when using a real-time monitor to read module state-structures), event-oriented analysis presumes target-side C code is peppered with what amounts to a very lightweight form of printf — one that can efficiently initialize and queue a time-stamped data packet in fractions of a microsecond. Decode of this packet into something more meaningful happens elsewhere.

Using special Trace functions called at run-time, module suppliers can emit meaningful events anywhere in their C code. More important, special configuration parameters exist on a per-module basis to enable/disable different levels of event logging either permanently or else through run-time flags. Here again, configuration-time flexibility coupled with whole-program optimization enables module suppliers to field one implementation that downstream clients can effectively tailor for a particular set of application requirements or platform constraints.

— Bob Frankel, 16 Nov 2007