From RTSC Central

Jump to: navigation, search
[offline version]

Rhetorical RTSC

Initial Questions About Real-Time Software Components

  Make no mistake about it—the material you're about to read introduces a discontinuity within conventional thinking about the C programming language, one that some may regard as natural and inevitable but that others may find a little challenging at first. So please, just keep an open mind as you enter the world of RTSC.

Contents

I'm an embedded C programmer, so why should I care about RTSC?

Because components have become a pillar of modern software engineering practices.... Starting with COM and CORBA in the 1990s—used with C++ in desktop and enterprise applications—contemporary programming languages like Java or C# innately support the design, delivery, and deployment of software components. Thanks to component frameworks aligned with these languages, present-day applications are more modular, more scalable, and more flexible than ever before; software components also promote greater levels of re-use from one application to the next.

While overwhelmingly successful in the mainstream, component technology has had minimal impact on the development of embedded applications where C (still!) remains the programming language of choice. Given the constraints of time, space, and power under which most embedded systems must operate, the run-time overhead typically incurred by mainstream component frameworks is simply too high a price to pay. And yet, given the rising complexity of embedded applications coupled with a desire to field portable (and re-usable) program elements across a range of embedded processors and platforms, we C programmers continually struggle with trade-offs between developing flexible, generic—yet inefficient—building-blocks versus more specialized software optimized for one particular application domain or hardware platform.

With RTSC (pronounced rit-see), however, you can "have your cake and eat it too": requiring no more than ANSI-standard C, we effectively take the language to new heights through component tooling and infrastructure aligned with modern software practices, and yet responsive to the unique challenges of embedded development. Simply a better way to program in C, you can enjoy higher-level programming and higher-levels of performance with RTSC.

What can you tell me about components in general, outside of RTSC?

Whether we're talking about software, hardware, cars, or home-entertainment, the rationale for engineering these systems using a component-oriented approach remains the same—managing change. And while re-using existing components in novel ways is certainly an upside benefit, our ability to simply swap-out component S for an alternate (and perhaps new-and-improved) component S′ with minimal impact on the overall system remains fundamental. This level of extreme modularity becomes critical throughout the system life-cycle, from when requirements change during initial development to when products must evolve once in the customer's hands.

To achieve this highly-desirable property of interchangeability, the experts teach us that any component-oriented methodology requires three fundamental capabilities: 1) the ability to specify components; 2) the ability to deliver components; and 3) the ability to assemble components.

Specifying Components.  Simply put, you can't replace S with S′ if you don't know what S is. Through an appropriate mixture of formal language and informal commentary, we must explicitly specify the boundaries of each component—its inputs, outputs, and operating characteristics captured as a contract between a component's supplier and client.

Delivering Components.  Component-based systems encourage highly distributed modes of development, with suppliers and clients often separated by time and distance. To efficiently reach their clients, suppliers must rely on some form of standardized container into which they can deposit and deliver components. From the client's perspective, receiving components from different suppliers that are uniformly "boxed-and-labeled" streamlines inventory management and system integration.

Assembling Components.  At the end of the day, components exist to be composed with one another. We simply expect components to easily "snap-together" using only basic tools, whether replacing elements of an existing system or else assembling a new system using existing components. Here too, standardization plays a key role in streamlining the overall process.

How does RTSC map these concepts onto embedded C programming?

The greatest challenge here lies with the C programming language—it simply lacks the essential expressive power of contemporary (component-friendly) languages like Java and C#, which significantly complicates the job of specifying, delivering, and assembling software components using C alone. At the same time, C has emerged over the past few decades as the ideal "medium-level" language for implementing embedded software, blending portable structured programming with efficient access to underlying hardware resources; replacing C with a more modern alternative is just not viable today, given the performance constraints faced by most embedded programmers.

RTSC addresses this dilemma by introducing two special-purpose languages that both supplement and complement standard C, granting embedded programmers entitlement to new features needed to produce and consume software components without compromising the vital qualities of C. Termed eXpanDed  C, the net result becomes a synergistic alliance of programming capabilities which, working hand-in-hand with standard C, elevates the language to an entirely new level:

Briefly, XDCspec serves as RTSC's specification language—used to formally define programmatic contracts (often termed APIs) between producers and consumers of software components in a higher-level manner than otherwise possible using C alone. XDCspec has its roots in the C-like IDLs (interface definition languages) first introduced with COM and CORBA in conjunction with C++ several decades ago, and embodies one of the fundamental principles of modern software engineering—clean separation of public, client-visible specifications from internal, supplier-proprietary implementations.

Along the implementation axis, we also mitigate some limitations of embedded C programming by introducing XDCscript as RTSC's meta-language—used by component producers and consumers alike to streamline delivery and assembly of real-time software components through (meta-)functions that execute on a resource-rich, general-purpose host computer. Complementing the more resource-constrained target-domain of embedded hardware executing embedded applications, RTSC's meta-domain opens up a world of possibilities for selectively migrating C-based target content to a higher-level language that operates in an implementation environment with virtually unlimited resources. XDCscript itself leverages industry-standard JavaScript for its foundation, extending this popular language through tighter coupling back to XDCspec and indirectly through to C itself.

To best appreciate the strategic role fulfilled by XDCspec and XDCscript within RTSC, consider the five-phase flow illustrated in the following figure which overlays new capabilities for specifying, delivering, and assembling real-time software components atop the familiar steps of compiling and linking C libraries and programs:

  What follows is a rather condensed summary of each phase of the RTSC flow, introducing concepts and terminology used further on. For a more thorough treatment covering the "why, what, and how" of the RTSC flow, you should dive into the material found here, either now or else at a later time. But if you already have an intuitive sense of what RTSC's all about—and wish to skip the technical details—feel free to just skim the rest of this article.

1  Specification

The flow ultimately begins with XDCspec, a C-like specification language used for defining programmatic contracts in a more expressive and robust manner than possible with C header .h files; in fact, RTSC automatically generates the latter files from XDCspec sources. The XDCspec language also elevates three new programming constructs to keyword status, enabling better support for "programming-in-the-large" in standard C:  module, a cohesive collection of constants, types, and functions with both a public specification (expressed in XDCspec) and a private implementation (typically expressed in C); interface, an abstract module (spec only) that other interfaces can inherit and that concrete modules can ultimately implement; and package, a higher-level programmatic namespace that in turn contains modules/interfaces within its scope.

2  Implementation

RTSC modules are typically implemented in C (though you can use C++ or assembly code where desired or appropriate) using a very clean, crisp, and portable style of programming that is both easy-to-read and fun-to-write. In general, though, all RTSC modules maintain implementations in two complementary programming domains:  a resource-constrained target-domain, where C functions are eventually bound into application programs executing on specific hardware platforms; and a host-based, resource-rich meta-domain, where corresponding XDCscript meta-functions play an active role in the design-time configuration as well as the run-time analysis of target programs. To fulfill the role of the companion XDCscript meta-language shown in the earlier figure, RTSC leverages the popularity, simplicity, and C-like "look-and-feel" of industry-standard JavaScript and its openly-available implementation through Mozilla's Rhino.

3  Packaging

Beside serving as a namespace for modules/interfaces declared in XDCspec sources, a RTSC package has a concrete manifestation as a directory that adheres to a prescribed set of conventions; even more important, each RTSC package is built, released, and deployed as an indivisible entity throughout its life-cycle. Specialized package build scripts—written in the XDCscript meta-language—simplify supplier-side management of multiple libraries compiled with multiple tool-chains (RTSC targets) as well as the inclusion of multiple subsets of these libraries in different package releases; manifests generated with each package record information about dependencies on other packages, about tool-chains used to compile constituent libraries, and about compatibility with earlier versions of this package. Client-side command-line and GUI tools streamline the deployment of RTSC packages by managing package repositories (directories containing package directories) as well as the package path (an ordered list of repository directories used to locate packages and their contents); these tools also validate the consistency and completeness of a set of deployed packages by leveraging information recorded in their manifests upon release.

4  Configuration

Driven by an XDCscript meta-program—a script that prescribes the target-domain elements required by a corresponding target-program—RTSC configuration leverages the full resources of the client's host computer to robustly validate this specific combination of elements as well as to optimally generate downstream C compiler/linker inputs tailored for this program's needs. Besides explicitly identifying all modules required by the target-program, the corresponding XDCscript meta-program can selectively assign configuration parameters ("configs" for short)—declared in the module's original XDCspec source file and then used in downstream generation to enable new levels of application-specific optimizations for embedded C programs. Once the client's XDCscript meta-program completes, RTSC configuration systematically invokes a series of special XDCscript meta-functions implemented by each module required by the corresponding target-program: 1) to assign configs to other modules; 2) to issue errors/warning if anything seems "wrong"; 3) to guide generation of linker-commands for this program; and 4) to actively synthesize C code based on knowledge gained through the configuration phase. In general, all XDCscript meta-content executing during RTSC configuration enjoys full access to this program's RTSC target and RTSC platform—more XDCscript meta-content that codifies information about the underlying ISA and memory-map as well as the process of compiling/linking/loading the program using underlying tools.

5  Analysis

Besides guiding its integration into a program, a module's XDCscript meta-functions can also actively participate in the run-time analysis of these programs once execution has commenced (or even terminated). Given the ability to read the program's data memory—via a host-based debugger, an embedded program monitor, or even a snapshot of program state saved to disk—RTSC analysis first decodes target-domain module state-structures into equivalent meta-domain objects and then calls special XDCscript meta-functions on a per-module basis to optionally transform these objects into a more meaningful view of the underlying state. Complementing this state-oriented perspective, another dimension of RTSC analysis involves merging, filtering, and interpreting streams of time-stamped events sourced during the course of program execution; the events themselves could be retained in target-side memory or else transmitted to another processor at run-time using an appropriate output device on the underlying hardware platform.

  Once again, the material found here provides a more systematic explanation of each phase within the RTSC flow, summarized rather quickly in the last five paragraphs. Either way, read on—the "hardest" (most technically challenging) material is now behind you.

Can I leverage RTSC with my existing code, or am I starting from scratch?

The C programming language has been widely used for over a quarter-century; and without question, large bases of legacy C code delivered to clients as headers and libraries already exist. While flanking C with both a specification language (XDCspec) as well as a meta-language (XDCscript) may seem rather exotic at first glance, you can actually employ RTSC in a more incremental manner that allows content leveraging select aspects of the RTSC flow to ultimately co-exist with legacy libraries/headers in embedded application programs. RTSC is revolutionary and evolutionary.

Creating Managed Content.  Without changing a line of legacy source code, you can leverage RTSC packages for their ability to manage software content from production through consumption; with only modest effort, existing libraries and headers could now flow from supplier to client via a standardized container that is built, released, and deployed using widely-available command-line and GUI tools.

  The RTSC packaging mechanism is actually quite general: anything can be delivered within a package, including other RTSC packages.

From the client's perspective, all packages exhibit a common "look-and-feel" that streamlines the installation and integration of their contents—independent of whether this content further leverages other elements of RTSC or else remains in its legacy form. From the supplier's perspective, RTSC packages afford new opportunities to explicitly codify dependencies upon other named packages as well as to automate (say) the inclusion of specific versions of legacy libraries during program configuration via special XDCscript meta-functions; content suppliers may also employ RTSC package build scripts to better insulate themselves from underlying compiler details as well as to better manage the compilation of common (legacy) source code for multiple targets.

  RTSC package build support is also quite general, with build scripts ultimately transformed in a standard GNU makefile that is robust and portable while readily extensible and customizable for the supplier's own production flow.

Creating Spec'd Content.  As general-purpose containers, RTSC packages need not further define any RTSC modules within their logical scope; but when they do, these packages can participate more actively in downstream program configuration as well as present a more uniform face to client C programmers via generated headers. By wrapping legacy libraries with spec'd RTSC modules whose bodies trivially defer to existing implementations, this content becomes indistinguishable from newly developed modules in the eyes of the component consumer; at the end of the day, each RTSC module has an XDCspec source that draws a programmatic boundary between clients and suppliers using a mixture of formal declarations and informal commentary, regardless of the implementation strategy chosen by the module supplier. Wrapper modules become an ideal locale for "adding value" atop existing libraries, through:

  • clarity of XDCspec sources over .h header files
  • configuration parameters for greater flexibility during integration
  • XDCscript meta-functions to support program synthesis and analysis
  • run-time parameter validation and event logging across function boundaries
  • and so forth, as an exercise for the legacy content supplier

Complementing the wrapper module approach—which typically requires no changes to legacy code—content suppliers can also introduce RTSC modules specifying just config parameters, replacing #define symbols, static constants, or readonly extern data in the existing code base. Straightforward to implement on the supplier's side, benefit accrues when downstream clients can robustly assign these configs in XDCscript meta-programs in lieu of more ad hoc and error-prone editing of C source files. And thanks to whole-program optimization technology available in many contemporary C compilers, module configs can become as efficient as #define symbols used in conjunction with #if directives; and wrapper modules can become as efficient as #define macros or inline functions.

Finally, RTSC enables content suppliers to define so-called metaonly modules in XDCspec—a module whose implementation is limited to host-based XDCscript meta-functions—used to otherwise facade legacy content when even wrapper modules are inappropriate. Though lacking an automatically generated C header from its XDCspec source, a metaonly facade module becomes indistinguishable from others during design-time configuration (and even during run-time analysis). Leveraging some of the more sophisticated back-end generation capabilities of RTSC configuration (e.g., synthesis of C code via parameterized templates), metaonly facade modules can achieve some of the benefits of target modules containing configs—but without any modification of legacy code, a possible constraint. Erecting software facades itself emerges as a general design pattern employed to encapsulate a myriad of underlying details—whether configuring legacy libraries or even calling other spec'd modules—through a simpler, more focused programmatic contract; what's important here is that RTSC modules defined in XDCspec can broadly subsume a virtually unlimited continuum of techniques for layering new functionality atop existing content.

What's the relationship between RTSC and the XDCtools product?

Think of RTSC as an ever-expanding "space" populated with target-content—managed packages of spec'd modules—supported with host-based tooling used throughout the software life-cycle. Taking this perspective, the XDCtools product lies at RTSC's point-of-origin, providing the necessary foundational elements for others to fill the RTSC space with target-content as well as additional tooling.

Language Support.  Virtually everything in XDCtools—and by extension, RTSC—starts with the XDCspec language. The XDCtools product itself includes an XDCspec translator used to first parse source files and then (among other things) to generate corresponding C headers as well as documentation for the client programmer.

XDCtools likewise delivers the XDCscript meta-language, integrated more closely with XDCspec through an extended version of Rhino/JavaScript also shipped with the product; besides relying heavily on XDCscript to support the build/release/deploy cycle of RTSC packages as well as the configure/execute/analyze cycle of RTSC programs, XDCtools encourages use of the XDCscript meta-language as a general-purpose scripting engine that leverages the power and familiarity of JavaScript.

As for C itself, the XDCtools product does not necessarily bundle any particular compiler tool-chain(s); indeed, the XDCtools can interoperate with any ANSI C compiler. At the same time, the product does include knowledge of literally dozens of different C compilers from multiple vendors—so-called RTSC targets, which are actually spec'd XDCspec metaonly modules coupled with a XDCscript implementation which (with only modest effort) anyone could develop and deliver in their own package.

Core Packages.  Moving up a level in the figure, the bulk of XDCtools comprises over a hundred packages containing even more modules/interfaces that broadly fall into three major groups:

  • packages with metaonly modules/interfaces that themselves support the general build/release/deploy life-cycle of other RTSC packages; just as many contemporary programming environments bootstrap themselves (e.g., all of Java is Java classes), RTSC packages are ultimately managed through other well-known RTSC packages that lie at the core of XDCtools.

  • packages with metaonly modules/interfaces that support the general configure/execute/analyze life-cycle of RTSC programs; here again, XDCtools builds upon itself through special XDCscript meta-content that in turn drives the synthesis and analysis of target-content elements in executable programs.

  • packages with target modules/interfaces that provide a first layer of run-time support for C programs containing other RTSC modules; portable across all targets, these modules augment the standard C runtime library with better embedded support for pluggable memory allocators, event logging and error handling, entry/exit of critical sections, as well as overall program startup/shutdown.

The latter content—shipped with the XDCtools product in C source-code form, to support migration to new RTSC targets—has its origins in some of the more rudimentary elements of TI's DSP/BIOS kernel. Separating out this functionality from the kernel enables RTSC to serve a much broader class of embedded system environments. By making the XDCtools product openly and freely available (not unlike Sun's Java Runtime Environment or Microsoft's .NET Framework), we anticipate an ever-growing inventory of inter-operable third-party target content populating the world of RTSC.

  TI's new System/BIOS 6.00—itself a collection of RTSC modules/interfaces delivered as RTSC packages—will simply presume the presence of the XDCtools run-time. Information about its popular predecessor—DSP/BIOS, one of the industry's most widely deployed embedded kernels—can be found here.

Essential Utilities.  Finally, the XDCtools product incorporates a number of basic utilities for:

  • building and releasing packages
  • invoking other tools implemented in XDCscript
  • generating documentation from specs
  • managing package repositories
  • and so forth, as an exercise for the toolsmith

These utilities can be used directly from the command-line or else invoked through extension points within your own development environment. In some instances, we'll also include a corresponding GUI tool—delivered as a package (of course) containing metaonly modules implemented in XDCscript in concert with the Java-based (and JavaScript-friendly) Eclipse/SWT graphical environment. In other situations, we'll leave the provision of higher-level GUI tooling to others.

Here again, we see broad availability of the XDCtools as a catalyst for others contributing compatible yet complimentary tooling—not only for general-purpose use, but also to address the needs of more specialized vertical markets through integrated solutions that synergistically couple tooling and content.

Who's using RTSC today, and what's the roadmap going forward?

Internal teams within TI have used XDCtools for several years now, beginning with a 1.00 product first released in 2004; for reference, XDCtools 3.00 became available in mid-2007. The XDCtools product has broadly maintained the same functionality throughout its lifetime, though (referring back to the earlier figure) has generally solidified from left-to-right:

  • first-generation users began migrating legacy content into managed packages with minimal impact to existing development flow, as we described earlier;

  • second-generation users began leveraging the power of RTSC configuration, albeit through metaonly facade modules layered atop legacy content; and now

  • third-generation users are enjoying full entitlement to target-domain modules supported by a robust yet extensible set of core run-time services.

Because of its maturity, XDCtools 3.00 also establishes a firm baseline for backward compatibility. Through some rather novel code-generation techniques coupled with whole-program optimization, we believe that binary compatibility for deployed packages containing spec'd target modules is now possible going forward.

A premier user of the most recent XDCtools product, System/BIOS 6.00 represents a complete top-to-bottom "RTSC-ization" of the legacy DSP/BIOS code base:

  • about a dozen discrete packages containing spec'd target modules with clean, portable C-based implementations of functionality already familiar to DSP/BIOS users;

  • a number of low-level target interfaces used to abstract differences from one HW family to the next, also contributing to greater flexibility as well as portability;

  • robust XDCscript meta-content accompanying each target module which supports design-time configuration as well as run-time analysis; and

  • a set of supplementary target modules that inherit and implement several of the open target interfaces included in the XDCtools run-time, yielding full entitlement to the latter.

But even more noteworthy:  despite all of the higher-level programmatic structure introduced through RTSC, the run-time performance of System/BIOS 6.00 exceeds that of its predecessor, both in terms of time and space.

TI's recent CodecEngine product represents another leading user of XDCtools, with the emphasis here more on packaging and facading legacy content rather than re-working existing code. Because CodecEngine amalgamates a rather diverse set of elements—DSP/BIOS and Linux, I/O and communication drivers, DSP algorithms plus framework middleware, etc.—fielding this sort of product demands a common delivery format (RTSC packages) as well as a standard integration process (RTSC configuration) for these various sub-elements.

As a consequence of CodecEngine and its recent proliferation, a growing number of content suppliers have already embraced RTSC packaging as a de-facto standard; others have also begun facading existing xDAIS algorithms with metaonly modules whose configs guide integration of legacy code into CodecEngine program images. As part of its own roadmap, CodecEngine looks to drive use of RTSC's XDCspec language more aggressively when publishing target-domain C interfaces that codify different classes of algorithms—what might well become a RTSC-ization of the long-standing xDAIS algorithm standard along with its derivatives.

  More information about CodecEngine and xDAIS can be found here.

As for RTSC itself, broad dissemination of the latest generation of XDCtools becomes an ever-higher priority moving forward. And while System/BIOS and CodecEngine will themselves continue to draw other producers and consumers of real-time software components into the RTSC circle, we plan a number of independent initiatives—such as amassing an online RTSCpedia, starting with this article—to make the XDCtools product and its underlying technology available and accessible to the broader community of embedded C programmers.

To that end, we also anticipate launching an open-source initiative—think rtsc.org—that would truly render XDCtools openly and freely available to all. Besides encouraging third-parties to field higher-level forms of tooling (such as integration with the Eclipse/CDT programming environment, popular among embedded developers), a de-facto standard component model that addresses the specific needs of embedded software can breath some necessary life back into the C programming community as a whole. Recognizing an opportunity to provide much-needed leadership within an otherwise highly fragmented population of almost a half-million embedded C programmers, RTSC is quite simply:  a better way to leverage C.

What are my next-steps with RTSC, and where do I go from here?

Enough of our grand vision from 10,000 feet.... While we plan other technical overviews for the RTSCpedia, you should now drop-down to ground-level and start engaging directly with the XDCtools product, available here. If you would, however, prefer to just sample the documentation, we have begun work on a series of guided Primers that introduce different aspects of the XDCtools through graduated programming examples:

  • The RTSC Module Primer focuses squarely on spec'd target modules implemented in C, starting with their consumption by client programs and then moving to their production by content suppliers; a highly-recommended entry-point for any C programmer, this Primer also introduces necessary elements of RTSC packaging and RTSC configuration along the way.

  • The RTSC Interface Primer takes your knowledge of concrete modules to the next level of generality by examining abstract interfaces; this Primer also illustrates some important design-patterns involving abstract interfaces, used to increase flexibility in concrete module implementations but without increasing run-time overhead.

  • The RTSC Packaging Primer delves a little deeper into the mechanics of RTSC packages and how you manage them through their build/release/deploy life-cycle, regardless of the types of elements they may contain; independent of the previous Primers, anyone electing to deliver legacy content in standard RTSC packages should master this material.

  • The RTSC Scripting Primer introduces the unique extensions the XDCscript meta-language brings to JavaScript, without regard for any particular usage of the language in package management, in program configuration and analysis, or even in stand-alone tooling; also independent of other the Primers, this material become important to potential suppliers of supplementary tools and utilities (delivered as spec'd metaonly modules).

Each of these Primers in turn leads to a family of more comprehensive User's Guides that address very specific subject-matter at the next level detail: these documents surgically dissect each of the packages bundled with XDCtools respectively supporting the RTSC package life-cycle, the RTSC program life-cycle, and the RTSC program run-time; these documents also introduce more advanced features of the XDCspec language together with corresponding programming idioms used in XDCscript meta-content as well as C target-content.

Finally, Reference documentation exists for each package bundled with the XDCtools product; this material is automatically synthesized from special documentation comments embedded in XDCspec sources contained within these packages—no different from any third-party package. You'll also find language reference material covering XDCspec syntax and semantics, XDCscript extensions to JavaScript, and conventions for mapping higher-level programming constructs like modules and interfaces onto ANSI C.

  And there you have it.... So welcome to the world of RTSC.

[offline version]

Views
Personal tools