1
• Mobile Indoor Robot ARTOS 0 50 100 150 200 250 300 GraphFromFileLoader GraphPathPlanner PathTraversal Odometry LaserScanner + Xtion UltrasonicProcessing Safety Behaviors Joystick Xtion filter DiffDriveKinematics Multiplexer Ultrasonic Filter LocalPlanner Time in seconds With Visualization Without Visualization 0 10 20 30 40 Number of aspects mentioned Related Work Experiments Design Considerations On the Benefits of Component-Defined Real-Time Visualization of Robotics Software Max Reichardt, Gregor Zolynski, Michael Arndt, and Karsten Berns Robotics Research Lab, University of Kaiserslautern, Germany Contact Max Reichardt, Gregor Zolynski, Michael Arndt {reichardt, zolynski, arndt}@cs.uni-kl.de The Idea Conclusion and Outlook Two Visualized Systems Each component provides a visualization that illustrates its current state and what it is doing (use e.g. as animated thumbnails in component graph) Motivated by: • Challenges in robotics software development – such as system maintenance, component integration, debugging • Control panels in process automation • Real-time visualization of behavior networks (tool-defined) [1] • Online visualization of effect networks in procedural graphics generators (e.g. Nodewerks, NeoTextureEdit) Long-range Navigation Mid-range Navigation Short-range Navigation Mecha- tronics and Sensor Data Pre- processing • Component visualization in manually created illustrations Sensor Output Sensor Input Controller Input Grown Difference Height Analysis Height Difference Heightmap Max (flat) Heightmap Min (flat) Point Cloud Collector Sensor Buffer (Left) Sensor Buffer (Right) Steep/Flat Splitter Thresholded Difference Trailer Detection 3D • Autonomous Bucket Excavator THOR Classic component graph of Perception subsystem Basic component model with optional, dedicated visualization outputs Sensor Output Sensor Input Controller Input Controller Output Asus Xtion Rotation + Ceiling Filter Behaviors Backward (Layer 0) Backward (Layer 1) BasicDifferentialDriveBehaviors Forward (Layer 0) Forward (Layer 1) SafetyBehaviors Turn Left (Layer 0) Turn Left (Layer 1) Turn Right (Layer 0) Turn Right (Layer 1) DifferentialDriveKinematicsWithTwist GraphFromFileLoader GraphPathPlanner JoystickToTwist Laser Scanner and Depth Camera to Sectormap Local Pathplanner Multiplexer Odometry PathTraversal Current Current UltrasonicProcessing Classic component graph of control group Sensor Output Sensor Input Controller Input Controller Output Asus Xtion Rotation + Ceiling Filter Behaviors DifferentialDriveKinematicsWithTwist GraphFromFileLoader GraphPathPlanner JoystickToTwist Laser Scanner and Depth Camera to Sectormap Local Pathplanner Multiplexer Odometry PathTraversal UltrasonicProcessing Update() Parameters Server Ports Client Ports Inputs Outputs Visualization visualization-low visualization-mid visualization-high Targets: • Simple design, minimal framework changes • High update rates with possibly many subscribers • Low computational overhead & option to deactivate Design Decisions: • Use annotated data ports (simple and scalable) • Existing or dedicated component outputs • Provide different levels of detail: low, mid, high • Support multiple data types for visualization (including bitmaps and vector graphics) Mobile Robot Control Analysis (ARTOS) • Task 1: “Please look at the robot control group and describe what you see and what you think the purpose of the components is.“ • Six components were visualized (below dashed line) • Attention is shifted towards visualized components • Task 2: “Why did the robot cease moving?“ (hint: hardware failure) • There are bugs that can be found quicker Understanding Unknown Systems (THOR) • Questions: Q1) What are the inputs of the Point Cloud Collector? Q2) What is the input and the function of Height Analysis? Q3) What are the inputs and the function of Height Difference? Q4) What is the function of Thresholded Difference? Q5) What is the function of Grown Difference? Q6) What are the inputs and the outputs of Steep/Flat Splitter? Q7) What is the purpose of the processing chain from 2 to 6 ? Q8) What are the outputs of Heightmap Max/Min (flat)? • The visualization helps to understand systems Computational Overhead • Low, typically negligible 0-100 100-200 200-300 0 1 2 3 4 Time in seconds Number of Results With Visualization Without Visualization Proband Vis. Time (m:s) 1 – 4:35 2 0:03 3 – 4:04 4 0:08 5 – 3:10 6 0:51 7 – – 8 0:27 9 1:25 10 0:26 11 – 3:18 12 – 4:50 [min] Q8 with component visualization Q1 Q8 only classic visualization Q1 1 2 5 10 15 THOR Control ARTOS Control Visualization absent disconnected connected absent disconnected connected CPU 201.0% 201.0% 201.0% 80.0% 80.0% 83.0% Memory 784.9 MB 785.5 MB 794.7 MB 32.7 MB 32.7 MB 33 MB Cycle time 84 ms 84 ms 90 ms 16 ms 16 ms 17 ms • Feasible software visualization approach • Can be implemented with minimal framework changes (~50 LOC) and low computational overhead • Pros • Helps to explain and understand systems (relevance for system maintenance, also e.g. live demos) • Helps identifying critical system behavior and malfunction (notably, visualization output can be recorded) • Runtime construction: immediate visual feedback to changes to the system • Insights on behavior of black-box components • Cons • Development overhead (tradeoff with benefits) • Experience • For some (generic) components, it is hard to come up with suitable visualization • Visualizations vary in complexity, uniform thumnbnail size not always ideal • Future Work • Combination with tool-defined visualization approaches • Tools with interactive component visualization views (e.g. view transformations, hide elements) • Data compression to reduce network bandwidth requirements (e.g. video codecs for bitmaps) • More experience, further experiments Finroc Implementation A O road Indoor Climbing Humanoid THOR Gator Icarus SUGV Icarus LUGV Agricultural Robots RAVON CROMSCI CREA Biped ROMAN MARVIN ARTOS VIONA (Robot Makers) Finroc (Framework for Intelligent Robot Control) • Our implementation of component-defined visualization is availabe on http://finroc.org • Included in Finroc 14.08 release • Requires Linux with gcc 4.8+/clang 3.4+ • Key features • Slim and highly modular framework core • Can run without an operating system • Efficient, lock-free, real-time implementation • Intra-process runtime construction • Scales up to thousands of components • Support for multiple component models (via plugins) • Composite components • Dynamic component interfaces Finroc projects at the Robotics Research Lab and Robot Makers GmbH Illustration of RAVON‘s control architecture [2] • In most robotic frameworks and toolkits, applications are constructed from reusable software entities (often called “components“). To our knowledge, no solution currently supports component-defined real-time visualization in its component model or tooling that visualizes the component graph. • Some solutions visualize component meta-information. [1] M. Proetzsch, T. Luksch, K. Berns. Development of Complex Robotic Systems Using the Behavior-Based Control Architecture iB2C. Robotics and Autonomous Systems 58(1):46–67, January 2010. [2] C. Armbrust, M. Proetzsch, K. Berns. Behaviour-Based Off-Road Robot Navigation. KI - Künstliche Intelligenz 25(2):155–160, May 2011. http://dx.doi.org/10.1007/s13218-011-0090-2. [3] C. Jang, S.-I. Lee, S.-W. Jung, B. Song, R. Kim, S. Kim, C.-H. Lee. OPRoS: A New Component-Based Robot Software Platform. ETRI Journal 32:646–656, 2010. [4] N. Ando, T. Suehiro, T. Kotoku. A Software Platform for Component Based RT-System Development: OpenRTM-Aist. In Carpin et al. (eds.), Simulation, Modeling, and Programming for Autonomous Robots. Lecture Notes in Computer Science 5325, pp. 87–98. Springer Berlin / Heidelberg, 2008. [5] T. H. J. Collett, B. A. MacDonald. An Augmented Reality Debugging System for Mobile Robot Software Engineers. Journal of Software Engineering for Robotics (JOSER) 1(1):18–32, January 2010. ROS rx_graph and rqt_graph: Topic statistics can be visualized on connections [image licenses: Creative Commons Attribution 3.0; from http://wiki.ros.org/rxgraph and http://wiki.ros.org/rqt_graph] Simulink model of a wind turbine [image license: CC BY-SA 3.0; author: "Mcarone1"; http://commons.wikimedia.org/wiki/File:Simulink_model_of_a_wind_turbine.tif] Visual Programming: LabView (Robotics) [image license: Public Domain; author: "Labvi"; http://commons.wikimedia.org/wiki/File:LabVIEW_Block_diagram.JPG] Player Project: Generic, tool-defined visualization approach focused on augmented reality [5] [images from [5]] SmartMDSD toolchain [http://servicerobotik-ulm.de/drupal/sites/default/files/toolchain-v2_0.png image reproduced with kind permission from Christian Schlegel] RT System Editor/RtcLink: OpenRTM-aist [4] online design tool [image license: CC BY-SA 3.0; author: "Ysuga"; http://commons.wikimedia.org/wiki/File:RT_System_Editor.jpg] OPRoS [3] Component Composer NeoTextureEdit effect network (tool available on http://neotextureedit.sourceforge.net/)
