| Date post: | 30-Oct-2014 |
| Category: |
Technology |
| Upload: | boisgera |
| View: | 783 times |
| Download: | 0 times |
2. model is a data-ow language described in the sec- tion 2.2, an execution model widely used in the design safety-critical systems. The details of their integration that forms the HMI meta-model is discussed in section 2.3.2.1Data-driven Scene Graph The graphics layer of the E DONA / HMI model is a 2D data-driven scene graph. A scene graph or hierarchi- cal display list is a structured collection of graphic nodes, commonly used in 2D and 3D vector graphics. It is ultimately composed of primitive nodes such as vector graphic shapes (lines, circles, etc.), images and text. On the other hand, compound nodes are node containers that structure the scene graph.All types of graphic nodes may be congured. Con- sider for example the text node denitiontextptext hello, (1) Figure 1: A hybrid mechanical and screen-based transform rotatep90q, instrument cluster. Displays may be used toll rgbp255, 0, 0q, reproduce the traditional features of mechanical family Helveticaq systems (here RPM and speed dials) as demon- strated in the top gure but may also be easily Such an attribute assignment is used as a single recongured to display other kinds of vehicule in-and uniform mechanism to congure node core data formation.(text), and the geometrical transformations (trans- form) as well as style options (ll and family) that raises several issues. Interoperability the ability are applied to the node. to easily exchange components specications and de-Some attributes used in this assignment are type- signs is a major industrial issue. Also, as theyspecic: the text data and font family for example present information that may directly inuence theare meaningless for all but text nodes. Other attributes driver behavior, these HMIs shall also be treated asare shared between several node types: the ll color safety-critical software components. We propose hereis applicable to all vectorial constructs and the geomet- a model-based approach that relies on a scene graph rical transform to all graphic nodes. The list of all graphics layer and a synchronous functional layer.attributes applicable to a given node type is its context. Synchronous languages [9, 13, 3] are standard rep-The union of all such contexts is called the graphics con- resentations for safety critical software, they enjoy a text. clear and simple semantics that allows various pro-Finally, node denitions may be partial: although gram analysis like automatic test generation [15] orthe opacity attribute belongs to the text node context, model-checking [10]. An efcient design process and it does not appear in the node denition (1). Such un- the ability to do some rapid prototyping are crucial as dened values are to be interpreted as a convenience well. Our model comes together with dedicated tools mecanism for default values 1.0 would be the opac- for the assisted design and the compilation of compo- ity natural default for example. They may also be used nents into executable graphics interfaces.to support context inheritance.We formalize briey the structure of this scene graph grammar. More information about the set of 2 HMI Modelsnode types and corresponding context attributes that we actually consider may be found in section 2.4. Let We formalize in this section the meta-model that un-P be the set of primitive node types, let g the group derlies the whole E DONA / HMI design process. Thebe the single compound node, and let G be the graph- HMI graphics content is described as a data-drivenics context. Our graphics model is given by the system ppG val , val , q, p P scene graph, a powerful yet simple data model de- scribed in section 2.1. The document structure is xednode 12 (2) and no graphics node is ever created at runtime ;| gpG val1 , val2 , qxnodey(3) the model is therefore amenable to extensive analysis. This graphics layer is complemented with a functional| node1 , node2(4) layer that provides a component model for HMI to be The notation pG val1 , val2 , q is only a shortcut for handled at the proper abstraction level. The functional the graphics context assignment pai vali | ai Gq 2 3. as it appears in the example (1). We denote Gptq the we focus on issues that directly impact the HMI com- context of the node type t and u the undened value. ponent model and the integration of its functional and The graphics context assignment G val1 , val2 , in graphics layer. (2) is subject to the consistency constraint Scoping. Every statement denes implicitely inputand output data ows determined by the functions I tai G | vali $ uu Gppq and O: and accordingly the same assignment in (3) toIpow exprq Ipexprq Ipnextpow1 q ow2 q tow2 u tow1 u tai G | vali $ uu GpgqIpstmt1 , stmt2 q pIpstmt1 q Opstmt2 qq For the sake of simplicity, we may assume here thatpIpstmt2 q Opstmt1 qq all context attribute values belong to the same value Ipstmt when owq Ipstmtq towu space. A concrete instance of value space would gatherIpconstantq heterogeneous attribute types such as: Ipfctpow1 , ow2 , . . .qq tow1 , ow2 , . . .u Ipow1 ? ow2 : ow3 q tow1 , ow2 , ow3 u val u (14)| int |oat | matrixp3, 3q(5) and| text|color| fontnameOpow exprq towu A scene graph is in our terminology data-driven if theOpnextpow1 q ow2 q tow1 u value space used in (2) and (3) is composed of unde- Opstmt1 , stmt2 q Opstmt1 q Opstmt2 q ned values and data ows. Opstmt when owq Opstmtqval u | ow (6)(15)In order to gain an explicit control of the signals visi- During the HMI execution, the functional layer a bility we add an extra construct the component to data-ow program whose structure is described in the the list of available statements (7-10). section 2.2 computes the graphics state: the set of con- crete values that are substituted to the data ows. In stmt | cpinputs, outputsqxstmty(16) this data model, the structure and nodes of the graph- Components input and output ows are given in their ics document are given but the graphics state is muta- declaration: ble.Ipcpinputs, outputsqxstmtyqinputsOpcpinputs, outputsqxstmtyq(17)outputs 2.2 Functional Model In the E DONA / HMI model, the graphics state is drivenComponents interface denition (16) is subject to the by a synchronous data-ow program, formally de-following consistency conditions: ned by the following grammar:inputs Ipstmtqand outputs Opstmtq (18) stmt ow expr(7) |nextpow1 q ow2(8) 2.3 HMI Model Integration |stmt1 , stmt2 (9)The HMI model integrates the graphics and functional |stmt when ow(10) models dened in the sections 2.1 and 2.2. The coreobject of HMI models is the HMI element. It is either a The , operator denotes concurrent execution of state-graphics node or a functional statement. ments, next the delay operator and the when con- struct represents conditional execution. Our actual functional model also has default values for constructselt node | stmt (19) (8) and (10) so that data ow values are always well Graphics and functional denition of (2, 4, 6) and (7- dened. This feature is not presented for the sake of13) are modied so that their right-hand side accept simplicity. For the same reason we do not discuss typ- hmi elements instead of nodes and statements. ing issues: expression expr are either constants, func- Graphics nodes being now integrated with the func- tion calls or the if-then-else construct:tional layer, data ows from the functional layer may expr constant (11)drive the graphics state evolution. From the functionalpoint of view, primitive node provide the following |fctpow1 , ow1 , . . .q (12) data ows: |ow1 ? ow2 : ow3 (13)IpppG val1 , val2 , qq tai G | vali $ uu We do not detail the well-known semantics of suchOpppG val1 , val2 , qq data-ow programs but refer the reader to [4]. Instead (20)3 4. how we extended it with appropriate HMI-specicconstructs. Format Selection. The selection of SVG as the mod-elling language for the description of graphic contentwas driven by our research for solutions that wouldimprove interoperability in HMI design. Despite sig-nicant differences, many leading solutions for em-bedded HMI design (such as VAPS XT, ALTIA Designor Scade Display [16]) have adopted a similar model-driven process. Their model for graphics share themost important structural features such as a tree-likedocument structure, support for afne transforms, ba-sic and complex shapes, styling, etc. They are how-ever based however on proprietary formats. On thecontrary, SVG is an open and mature recommandationfrom the World Wide Web (W3C) consortium, an inter-national standards organization known for standardssuch as the XML technology. SVG is a language for describing two-dimensional Figure 2: The scene graph representation of agraphics in XML. While the original SVG version gauge. As a data-driven scene graph, all but the 1.0 in 2001 was designed to support vector graph- needle rotation attribute may be assigned con- ics on the Web, SVG version 1.1 is used nowadays for stant values. If the gauge is used as a speedome-all kind of vector graphics descriptions. It has a large ter, functional modelling may be used to convert feature set that proved to be suitable for the kind of the vehicle speed into an angle and to ensure that graphics content that was presented in section 2.1. It the maximal angle threshold is never exceeded. also has given birth to two standard subsets namedproles in the SVG recommandation SVG Tiny andSVG Mobile. They are specically designed for mobile Finally, the graphics group dened in (3) and functiondevices, platforms that are primarily characterized by component dened in (16) are replaced by a single con-specic constraints in terms of CPU speed, memory struct the hmi group component that acts as a HMIsize, color support, etc. The family of mobile devices element container and merges the graphics and func-obviously contains mobile phones and personal digital tional hierarchies:assistants but the target platforms for embedded HMIgcpgraphics, interfaceqxelty (21) in vehicles share similar constraints. The usage scenar-ios for those proles explicitely include the modelling whereof graphical user interfaces. graphicsG val1 , val2 , (22) More generally, SVG 1.1 is not a monolithic standard as it provides consistent policies to prole (restrict) andinterfaceinputs, outputs (23) extend the standard in order to adapt to specic us-age scenarios. As SVG 1.1 is described as a set of in- Its inputs and outputs are given by: dependent modules, the denition of proles is sim-ple. The standard extensibility policy recommands the Ipgcpgraphics, interfaceqxeltyq use foreign namespaces to complement SVG data withinputs tai G | vali $ uu(24) application-specic content. The extra information issimply ignored by conformant SVG applications. This Opgcpgraphics, interfaceqxeltyq outputs (25) policy is used effectively in generic SVG authoring This hmi group component interface (21) is subject totools [12] and also in SVG models exported by some the consistency conditions (18). embedded interface designers. The existence of several SVG authoring tools makes 2.4Model Format and Serializationthe design of new HMI graphic content a simpletask. SVG being an authoritative description for vec- The format used to represent HMI models whosetor graphics, some HMI design tools used in the auto- structure was described in the previous sections motive industry already have a partial support for it. has a considerable practical impact. In this section Therefore, the selection of SVG as a core format has the we present the motivations behind the design of thepotential to increases interoperability between generic E DONA / HMI format, explain why we have selected aand application-specic design tools. Beyond existing subset of an existing graphics standard Scalable Vec-authoring tools, we also greatly benet from software tor Graphics (SVG) as the basis for our format and libraries that support SVG such as the Apache Batik4 5. SVG Toolkit [1]. Existing SVG software reduce the de- The data-driven scene graph model of section 2.1 is velopment cost of E DONA / HMI model editors as well described through the embedding of input XML el- HMI graphics rendering engines.ements into graphics node elements that specify theE DONA / HMI SVG prole. We nally adopted acontext attribute they act upon. Although this is an custom mobile prole, very similar to SVG Tiny 1.1.aspect of the model that was not described in the pre- This prole brings a welcome simplication with re-vious sections, a symmetric output element exists to spect to the full standard. Notably, the style of graphicmake available the initial graphics state to the func- nodes can only be set through individual XML at- tional layer. tributes called presentation attributes and not the Functional expressions (constants, function calls style or CSS styling mecanism. This simplication al-and the if-then-else construct) as well as the delay lows a uniform implementation of context assignmentoperator are mapped to XML elements and conditional in data-driven scene graphs. The handling of numer-execution is handled at group component level exclu- ical data is also simpler because length data have nosively. Functional expressions have explicit input and units and always refer to the local coordinate system. outputs interfaces. Sharing data ows between state- Text graphics nodes are also more manageable because ments is handled by a list of links in the parent com- a single style can be applied to them. ponent element. This simple block-diagram modelThe study of HMI design use cases has also taught allows easy generation and processing of functional us that the SVG Tiny prole supports almost everymodel at the expense of a verbose XML model. In par- features we needed and conversely that many of the ticular, no micro-language is used to encode functional graphics elements and context attributes that have information in attributes as the SVG format does for been removed (lters, the support for group opacity, path, transform, etc. so that all functional processing etc., see [17] for a detailled list of supported constructs) may be done with standard XML tools. were costly to render and not adapted to the contextA standard library of functions has also been de- of dynamic document rendering. We have only rein-ned. It contains the classical logic, numeric and com- troduced three features that all belong to the full SVGparison operators. It also provides text processing 1.1 specication: gradients, opacity and clipping. Wefunction and type conversions operators between nu- however limit them to their simplest form: gradients meric and text types the only primitive types that and opacity constructs are supported as in the SVG we actually consider. SVG array-like attributes such Tiny 1.2 candidate recommandation and clipping as in as path and transform are not handled with spe- SVG Basic 1.1. We believe that this set of constructscic types. Instead they are either considered as text is currently a good trade-off between model simplicity attributes and as such are fully mutable or considered and the support for a large range of HMI models. as numeric arrays with a xed structure, the numericcontent being then their only mutable part.On the other hand, we exclude from the E DONA / HMI prole the support for declarative animation of SVG Tiny 1.1: the kind of dynamic docu- 3 E DONA / HMI Software Tools ment that these constructs allow are strictly contained in our full HMI model provided that a time input Tool support is crucial to ensure the success of the is available to HMI components. On the other hand, E DONA / HMI model. Our E DONA / HMI Prototyping the data-driven model support direct update of the toolchain includes a set modelling tools, a generator of graphic state that cannot be expressed within theHMI software components and a runtime architecture time-based model.based on J AVA. This platform is specialized for sim-Data Flows Format. The E DONA / HMI format ex-ulation in the design loop, component testing and de- tends the SVG graphics format to make the sceneployment on car prototypes, particularly hosts of intel- graph data-driven and to describe functional con-ligent systems transportation (ITS) applications. While structs. The extension design is compatible with the runtime component of this platform is not com- W3C recommandation and enables conformant SVGpatible with usual embedded systems constraints, it is interpreters and viewers to properly process all HMI open-source and intended to simplify the generation graphics content. First of all, all data ow elementsof E DONA / HMI components for such targets. belong to the E DONA / HMI namespace http://www. edona.fr/hmi namespace, so that non-graphic data3.1 Model Management are ignored during SVG processing. Then, instead of mapping group components to an XML element in theSeveral types of software contributions were made E DONA / HMI namespace, we implement them as a svg to support the E DONA / HMI design process. Among group that contain an XML element with local namethem, a schema-based validator that checks the con- component in the E DONA / HMI namespace. As a con- sistency of XML model les against the E DONA / HMI sequence, graphic content in group components aregrammar, several gateways between E DONA / HMI always visible to SVG interpreters and properly pro- models and other formats, as well as internationaliza- cessed as groups.tion tools and a documentation generator.5 6. Moreover, a Python library was developed that per- part of HMI components is straightforward: it con- forms XML data binding on HMI model les: it au-sists in a model simplication step (see section 3.1) tomatically establish a mapping between HMI model followed by a code generation step. Code generation constructs and Python classes. This library was de- is a simple matter for such data-ow models where signed to be a suitable basis for many E DONA / HMI-static scheduling information is available [4]; its is also related development for example, to speed up themore efcient at runtime than an interpreter when development of a HMI model editors. Internally, itmany low-level operations are performed. More im- was used for automatic generation of HMI models portantly, in the context of HMI model prototyping, it and model transformations. The need we had to start is also simpler to implement: the complexity of hier- with models that may be incorrect for example witharchy and conditional activation have been eliminated SVG constructs that do not belong to the mobile pro-in the model simplication step while these constructs le or are not complete models and will not be un-have to be maintained in the interpreter scheme. til the nal steps of their design has motivated the The optimal execution policy for the graphics part of design of a lazy data binding scheme where instancesHMI component is a more complex issue and largely of E DONA / HMI constructs are only bound when they depends on design process and target requirements. are effectively accessed. As a consequence, it may be For HMI as software components in a context with used as the foundation of a validators that goes be-strong embedding and safety constraints, the simplic- yond grammar-level consistency checks.ity, size and performance that result from an exten-The XML data binding library is also used in the sive code generation process is likely to be preferred. early stage of the code generation process: the func- A typical code generation would then linearize the tional layers of HMI models are simplied into equiv- scene-graph and translate graphic primitives into calls alent models with no hierarchy and no conditional ac- to a low-level hardware-accelerated graphics platform tivations. Graphic constants that feed the functional such as OpenGL ES, a subset of OpenGL that targets layer are also extracted to fully decouple the functional embedded platforms. Considering the SVGL toolkit, model from the graphics one so that the functionalan OpenGL based SVG library, display lists can also be code that is generated is testable and also supports theused to drastically speed up rendering performances master-slave model explained in the next section. for static parts of an SVG document [5]. On the other hand, in the context of simulation in the design loop, testing and prototyping, the availability of debugging 3.2 Runtime Architecture information and model snapshots, complete with the HMI execution policy. Both graphics and functionalcurrent graphics state is crucial. Due to the complexity part of HMI components may be generated with dif- of the actual graphics model, maintaining in memory ferent execution policy, the two extreme choices be-the graphics data in a structure similar to the model is ing either a full interpreter architecture that executesthe simpler strategy. The signicant performance loss the model at runtime or a more extensive code gen-that may result from this strategy may be partially off- eration phase that produces model-independent exe-set by pre-rendering of the static graphics content cutable programs. We discuss in this section the pros something the E DONA / HMI model is totally suitable and cons of both options. for.The E DONA / HMI Prototyping component genera- tor therefore uses a mixed strategy: J AVA code is gen- erated for the functional layer whereas the graphics model is interpreted and rendered using the B ATIK SVG toolkit a component of the A PACHE XML G RAPHICS project in a way that updates of the graphics model are always accessible through a stan- dard XML API. This strategy allows at any time dur- ing execution to serialize the state of a HMI compo- nent as a new model and also allows for several graph- ics backends: no graphics (for testing purposes), J AVA AWT and image buffer.There also exist some alternatives to the B ATIK SVG toolkit, including the SVGL toolkit as mentioned ear- lier using OpenGL as backend. The RSVG library is also a SVG toolkit based on Cairo, a software li- Figure 3: architecture of the E DONA / HMI Proto- brary used to provide a vector graphics-based, device- typing runtimeindependent API for software developers. Cairo is de- signed to provide primitives for 2-dimensional draw- Our strategy for the generation of the functional ing across a number of different backends and it is de- 6 7. signed to use hardware acceleration when available.HMI runtime interface. A pure synchronous inter- face for HMI components would be conceptually sim- ple: a sequence of activation signals, coming with new values for the HMI inputs, would trigger an execution step for the functional layer and at the same time up- date of the HMI graphics. However this scheme would have a severe drawback: by synchronizing the render- ing of the graphics with the input ow, it may impose a fast rendering cycle, beyond what the target is able to execute due to the cost of this step. In the most com- mon use case, a xed frame rate is given that provides a sufcient quality in the user experience and no extra rendering steps should be performed, or the rendering should be synchronized with an external video stream. For other situations, such as execution for testing pur- poses, rendering is a step that should occur on specic Figure 4: EDONA-LOVe interface for the safety of breakpoints or explicit demands, the functional layer pedestrians being most of the time the only active component.Therefore, to avoid unnecessary performance issues 4 Conclusion and provide the needed exibility, to the synchronous activation signals is added a possibly asynchronous We presented the E DONA / HMI design process for the rendering signal. To ensure the correctness of exe- design of Human-Machine Interfaces on screens in car cutions, a duplication of the graphics state is requiredcockpits. This method successfully addresses typical in the functional component so that every new set ofindustrial issues in the domain of HMI design: in- input values may update the graphics state. This stateteroperability, proper modelling, exibility and rapid is however only transferred to the graphics layer uponprototyping. In addition to the design process ef- request. This partial decoupling and resulting layers ciency gain, our solution enables to promote the de- architecture is depicted in gure 3.sign process of HMI in cars to the same level as other safety-critical domains like avionics or power plants. Safety-critical issues are addressed by relying on a data driven scene graph model for the graphics part and a 3.3 Intelligent Transportation System Use synchronous model of computation for the functional Casepart of the E DONA / HMI system.Finally, to adress the needs of intelligent transporta- tion systems (ITS) embedded applications, we ex-References tended the B ATIK SVG toolkit to support the display of graphic and textual information as video overlays [1] Apache XML Graphics Batik SVG Toolkit. as well as the inclusion of embedded controls. A full- http://xmlgraphics.apache.org/batik/. edged HMI was also designed for the LOV E [7, 14] [2] S. Boisg rault, M. O. Abdallah, and J.-M. Tem-e project. LOV E aims to use multi-sensor tracking sys-mos. SVG for automotive user interfaces. In SVG tem to improve the road safety for pedestrians. OurOpen, Nuremberg, Germany, 2008. HMI interface was used to display video streams and through specically designed interfaces data such[3] F. Boussinot and R. de Simone. The Esterel lan- as pedestrians location and risk of collision. Real- guage. IEEE, 79(9):12931304, 1991. time data acquisition and management is performed by RTMaps2 , as well as the communication with the [4] P. Caspi, D. Pilaud, N. Halbwachs, and J. Plaice. HMI stack. LUSTRE: a declarative language for program-ming synchronous systems. In POPL 87: Proceed-This use case validated our approach, showing itsings of the 14th ACM SIGACT-SIGPLAN sympo- expressivity to quickly create custom components, andsium on Principles of programming languages, pages the possibility to quikly integrate these components178188, New York, NY, USA, January 1987. into a full consistent system. External video streamsACM. were also integrated, showing its capacity to interface with an asynchronous environment.[5] S. Conversy and J.-D. Fekete. The svgl toolkit: en-abling fast rendering of rich 2d graphics. Tech- 2 Real-Time Mutisensor Advanced Prototyping Software,nical Report 02/1/INFO, Ecole des Mines de http://www.intempora.com.Nantes, janvier 2002. 7 8. [6] EDONA: Environnements de d veloppement ou-everts aux normes de lautomobile website.http://www.edona.fr. [7] G. Gate, A. Breheret, and F. Nashashibi. Central-ized fusion based algorithm for fast people detec-tion in dense environment. In Proc. of the IEEE In-ternational Conference on Robotics and Automation,Kobe, Japan, May 2009. [8] N. Halbwachs. Synchronous programming of reac-tive systems. Kluwer, 1993.[9] N. Halbwachs, P. Caspi, P. Raymond, and D. Pi-laud. The synchronous dataow programminglanguage LUSTRE. Proceedings of the IEEE,79(9):13051320, September 1991. [10] N. Halbwachs, F. Lagnier, and C. Ratel. Program-ming and verifying real-time systems by meansof the synchronous data-ow language LUSTRE.IEEE Transactions on Software Engineering (TSE),Special issue: Specication and Analysis of Real-TimeSystems, 18(9):785793, September 1992. [11] Human Machine Interface Work Package 4.EDONA HMI Format. Report, EDONA, 2008. [12] Inkspace - open source scalable vector graphicseditor. Technical report. http://www.inkscape.org/.[13] P. Le Guernic, T. Gauthier, M. Le Borgne, andC. Le Maire. Programming real-time applicationswith SIGNAL. IEEE, 79(9):13211336, 1991.[14] LOVe: LogicieldObservation desVuln rables,eproject website. http://love.univ-bpclermont.fr.[15] V. Papailiopoulou. Automatic test generation forlustre/scade programs. In ASE 08: Proceedingsof the 2008 23rd IEEE/ACM International Conferenceon Automated Software Engineering, pages 517520,Washington, DC, USA, 2008. IEEE Computer So-ciety. [16] The standard for the development of crit-ical embedded display software.http://www.esterel-technologies.com/products/scade-display/.[17] Mobile SVG Proles: SVG Tiny and SVG Basic.W3C recommendation, W3C, June 2009. http://www.w3.org/TR/SVGMobile/.[18] Scalable Vector Graphics (SVG) 1.1 specica-tion. W3C recommendation, W3C, January 2003.http://www.w3.org/TR/SVG.8
