+ All Categories
Home > Documents > 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense...

1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense...

Date post: 29-Dec-2015
Category:
Upload: justina-grant
View: 219 times
Download: 0 times
Share this document with a friend
64
1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu [email protected] http://www.cis.uab.edu/wuh 10/29/2007 Advisor : Dr. Jeff Gray Thesis Committee : Dr. Barrett Bryant Dr. Brian Toone Dr. Chengcui Zhang Dr. Marjan Mernik Dr. Mikhail Auguston Software Composition and Modeling Laboratory D epartm entofCom puterand Inform ation Sciences U niversity ofA labam a atB irm ingham S o ftC o m
Transcript
Page 1: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

1

Grammar-Driven Generation of Domain-Specific Language

Testing Tools Using Aspects

Ph.D. Defense

Hui [email protected]

http://www.cis.uab.edu/wuh

10/29/2007

Advisor:

Dr. Jeff Gray

Thesis Committee:

Dr. Barrett Bryant

Dr. Brian Toone

Dr. Chengcui Zhang

Dr. Marjan Mernik

Dr. Mikhail Auguston

Software Composition and Modeling Laboratory

Department of Computer and Information SciencesUniversity of Alabama at Birmingham

S o f t C o m

Page 2: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

22

Overview of Presentation

Motivation

BackgroundDomain-SpecificLanguages

My Approach

Lack of end-user DSL testing tools

Research Goals

Testing support for DSLs

Evaluation

Raise AOP to a higher level of abstraction

public class Robot{… x=0; y=0; time=0; //move left move_left();

Eclipse PDE

Generative Programming

AOP Experimental Evaluation and Generality Analysis

Page 3: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

33

Categories of DSL End-Users

AdminAssistants

Business Person

Auto Factory Worker

Scientist

Spreadsheet

BusinessQuery Systems

Modeling Language

DSL forPhysics

Page 4: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

4

The Benefits of Using DSLs

• Case studies and empirical evaluations documented in the research literature have identified the following benefits of DSLs– DSLs can hide the lower level programming language

details from end-user programmers– DSLs assist in software maintenance whereby end-

users can directly use the DSLs to make required routine modifications

– DSLs assist end-users in writing more concise, descriptive, and platform independent programs

– The use of DSLs increases flexibility, productivity, reliability, and usability; shorten the application development time and reduce the development cost significantly

Page 5: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

55

Motivation of Proposed Research

Domain Experts program at DSL level

DSL translated into General Purpose Language (GPL)

Domain Experts deal with DSL

Integrated Development Environment (IDE)

subsel ect mebegin l eft ri ght up downend

Transl ater

subsel ect mepubl i c cl ass Robot{ publ i c stati c voidmain(Stri ng[] args) { Robot robot =new Robot(0,0,0); / /move l eft robot.move_l eft(); / /move down robot.move_down();robot.x = 5; robot.y = 6;

EditorEditor

CompilerCompiler

Visualizer

Visualizer

DebuggerDebugger

Domain Experts deal with GPL

Test EngineTest Engine

ProfilerProfiler

Page 6: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

66

Mismatch Between Abstraction Levels

….commands : ( c:command cs:commands | );command : ( RIGHT { fileio.print("//move right"); fileio.print("x=x+1;"); fileio.print("time=time+1;"); fileio.print(" "); } |LEFT { fileio.print("//move left"); fileio.print("x=x-1;"); fileio.print("time=time+1;"); fileio.print(" ");…

….public final void commands() throws RecognitionException, TokenStreamException {

try { // for error handling {

switch ( LA(1)) { case CALL: case INIT: case SET: case PRINT: { command(); commands(); break; } case END: { break; } default: { throw new NoViableAltException(LT(1), getFilename());

…public final void function_name() throws RecognitionException, TokenStreamException { try { // for error handling { switch ( LA(1)) { case RIGHT: { match(RIGHT); fileio.print(" //move right"); fileio.print(" move_right();"); fileio.print(" "); break; } case LEFT: { match(LEFT); sllinenumber=dsllinenumber+1; fileio.print(" //move left"); fileio.print(" move_left();"); fileio.print(" "); break; }…

ANTLR GrammarGenerated Java Parser Code

Page 7: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

Research Goal• A Matrix of DSL Testing Tools

• Generalized approach to automatically produce a software product line of DSL testing tools from DSL grammars

Imperative DSLDebugger

AspectGImperative DSL

Test Engine

Imperative DSLProfiler

Declarative DSLDebugger

Declarative DSLTest Engine

Declarative DSLProfiler

Hybrid DSLDebugger

Hybrid DSLTest Engine

Hybrid DSLProfiler

Future Work after PhD

Page 8: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

8

Outline

• Background

• DSL Debugging Framework (DDF)

• DSL Unit Testing Framework (DUTF)

• AspectG

• Future Work and Conclusion

Page 9: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

9

Different Types of Debuggers

• Categorized by [Zellweger, 1984] based on behavior of a debugger– expected behavior debugger– truthful behavior debugger

• Categorized by [Auguston, 1998] based on the behavioral models of higher level debugging mechanisms– algorithmic debugger – declarative debugger– event-based debugger

• The work in this dissertation represents expected behavior debuggers that perform typical debugging tasks on DSL programs

Page 10: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

1010

Categories of Domain-Specific Languages

• Imperative DSL– Robot DSL

• Declarative DSL– Feature Description Language (FDL)

– Backus–Naur Form (BNF) language syntax specification

• Hybrid DSL– Swing User-interface Language (SWUL)

– Hybrid Robot DSL

Page 11: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

1111

Categories of Domain-Specific Languages

• Imperative DSL: Centered around assignment expressions or control flow statements– Robot Language

…17 Down: 18 position(+0,-1) 19 Down: …21 Init position(0,0)22 Call left23 Call down24 Call knight25 Set position(5,6)26 Call up27 Call right28 Print position…

Page 12: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

1212

• Declarative DSL: declares the relationship between inputs and outputs– Feature Description Language (FDL)

1 Car : all (Carbody, Transmission, Engine, Horsepower, opt(pullsTrailer))2 Transmission : oneof (automatic, manual)3 Engine : moreof (electric, gasoline)4 Horsepower : oneof (lowPower, mediumPower, highPower)5 include pullsTrailer6 pullsTrailer requires highPower

Categories of Domain-Specific Languages

– Backus–Naur Form (BNF) language syntax

specification: a set of derivation rules

START ::= begin COMMANDS end ; COMMANDS :: = COMMAND COMMANDS | epsilon ; COMMAND :: = left | right | up | down ;

Page 13: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

1313

• Hybrid DSL: embedded GPL code within the DSL description (Robot)

… Random: { String answer; int max; JOptionPane myGUI = new JOptionPane(); Random rand = new Random(); answer = myGUI.showInputDialog("Generate a random number for X-axis between 1 and "); max = Integer.parseInt(answer); x = rand.nextInt(max); answer = myGUI.showInputDialog("Generate a random number for Y-axis between 1 and "); max = Integer.parseInt(answer); y = rand.nextInt(max); myGUI.showMessageDialog(null, "Generated Position(" + x + "," + y+ ")"); } Random:… Init position(0,0) Call left Call down Call knight Set position(5,6) Call up Call random Call right Print position…

Categories of Domain-Specific Languages

Page 14: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

1414

• Hybrid DSL: embedded DSL description (SWUL) within the GPL code

import javax.swing.*;import java.awt.*; public class WelcomeSwing { public static void main(String[] ps) { JFrame frame = frame { title = "Welcome!" content = panel of border layout { center = label { text = "Hello World" } south = panel of grid layout { row = { button { text = "cancel" } button { text = "ok" } } } } }; frame.pack(); frame.setVisible(true); }}

Categories of Domain-Specific Languages

Page 15: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

1515

Eclipse Plug-In Based Software Development

• Eclipse Debug Perspective • JUnit Eclipse Plug-in

a) Variable Inspector

b) Navigation Toolbar

Page 16: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

1616

Program Transformation with the Design Maintenance System

• Design Maintenance System (DMS) is a transformation and re-engineering toolkit developed by Semantic Designs

• Provides lower level transformation functions such as parsing, AST generation and manipulation, pretty printing, powerful pattern matching, and source translation capabilities

• Pre-constructed domains for several dozen GPLs (e.g., Java, C++, and Object Pascal)

• This research required the creation of a new domain for ANTLR

Page 17: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

1717

Plug-In Based Software

Development

Syntax-Directed Translation

Aspect-Oriented Programming on DSL Grammars

Overview of my Approach

Design Patterns

Generative Programming

Eclipse

Model-View-ControllerAdapter Pattern

AspectG

ANTLR

pointcut productions(): within(command.*);

before(): productions() { dsllinenumber=dsllinenumber+1;}…

public class Robot{… x=0; y=0; time=0; //move left move_left();

Page 18: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

18

Outline

• Background

• DSL Debugging Framework (DDF)

• DSL Unit Testing Framework (DUTF)

• AspectG

• Future Work and Conclusion

Page 19: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

1919

DSL Debugging Framework (DDF)

Re-interpreter GPL Debugging Sever

Source CodeMapping

Debugging Results Mapping

Debugging Methods Mapping

ANTLR Translator

GPL

DSL Debugging Actions

DSL Debugging Variable View

DSL Level

GPL Level

End-User

GPL Debugging Commands

DSL

Page 20: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

2020

Process of DSL Debugger Generation

12

6

1 begin2 left3 down4 up5 right6 end

4

Lexer,Parser

public class Robot{ public static void main(String[] args) { …… //move left x=x-1; time=time+1;

//move down y=y-1; time=time+1; …… } }

3

import java.util.ArrayList;

public class Mapping { ArrayList mapping; public Mapping(){ mapping=new ArrayList(); mapping.add(new Map(1, "Robot.java",2,8)); mapping.add(new Map(2, "Robot.java",10,14)); …… } }

Mapping Component

5

Robot DSL Grammar In ANTLR Notation

Robot DSL

Generated Lexer, and Parser by ANTLR

Robot.java and Mapping.java

Java Command Line Debugger

Robot DSL Debugging Perspective in Eclipse

DebuggerView

DSLEditor

Variables View

DSL grammar is defined using

ANTLR Notation

DSL is the input of the Lexer and

Parser generated by ANTLR

DSL translation

process

Robot DSL corresponding

GPL code generated in

JavaAdditional

Mapping code generated in

Java

Corresponding GPL and Mapping Code

are inputs of Mapping component

jdb communicates with mapping component

Eclipse Debugging perspective

communicates with mapping component

Page 21: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

2121

Source Code Mapping…

3 knight:

4 position(+0,+1);

5 position(+0,+1);

6 position(+1,+0);

7 knight:

8 …

9 Init position(0,0);

10 left;

11 down;

12 knight;

13 Set position(5,6);

14 up;

15 right;

16 Print position;

6 public static void move_knight(){

7 x=x+0;

8 y=y+1;

9 x=x+0;

10 y=y+1;

11 x=x+1;

12 y=y+0;}

13 public static void main(String[] args) {

14 x=0;

15 y=0;

18 move_knight();

20 x = 5;

21 y = 6;

26 System.out.println("x coordinate="+x+""+

27 "y coordinate= " + y);}

{13, "Robot.java", 20, 21, "main", "none"}

Page 22: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

2222

Debugging Methods Mapping

Mapping DSL GPL

Source Coden_i maps to m_i to m_j

Line Number:n_1n_2n_...n_in_i+1n_...n_jn_j+1n_...

Line Number: m_1m_2m_...m_im_i+1m_...m_jm_j+1m_...

Breakpoint Set breakpoint at n_i Set breakpoint at m_i

Step Over Step over line at n_i Step Over algorithm

Step Into Step into line at n_i Step Into algorithm

Terminate Terminate at line n_i Terminate at line m_i

Resume Resume at line n_i Resume at line m_i

Page 23: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

2323

DSL Debugging Step Over Algorithm1 if (function name equals “main”) {

2 if (dsl_line_number < last line number of DSL code) {

3 set breakpoint at gpl_line_number corresponding to dsl_line_number+1

4 call cont()

5 }

6 else {

7 call cont()

8 step over last line of DSL code, debugging session terminated

9 }

10 current dsl_line_number increased by one;

11 }

12 else {

13 get function_type from mapping information base

14 if (function_type equals "functionbody") {

15 current dsl_line_number increased by one

16 for all the statements corresponding to this one line of DSL code {

17 call step()

18 }

19 } else if (function_type equals "functionend") {

20 call step()

21 assign current dsl_line_number as previous_dsl_line_number + 1

22 }

23 }

Page 24: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

2424

Debugging Results Mapping

1 | INIT var:VARIABLES LPAREN init_num1:NUMBER COMMA init_num2:NUMBER RPAREN

2 {

3 dsllinenumber=dsllinenumber+1;

4 fileio.print("x="+init_num1.getText()+";");

5 fileresult.print("x_coordinate=print x");

6 gplbeginline=fileio.getLinenumber();

7 fileio.print("y="+init_num2.getText()+";");

8 fileresult.print("y_coordinate=print y");

9 fileio.print("time=0"+";");

10 gplendline=fileio.getLinenumber();

11 filemap.print("mapping.add(new Map("+dsllinenumber+",\"Robot.java\","+

12 gplbeginline +","+gplendline+","+"\""+funcname+"\""+","+"\""+funcall+"\""+"));");

13 fileresult.print(variable=var.getText()+"(x_coordinate,y_coordinate)");

14 }

Black: Basic functionalityBlue: Source code mappingRed: Debugging results mapping

Page 25: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

2525

Generated DSL Debuggers

Page 26: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

2626

Imperative DSL Debugger

Page 27: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

2727

Declarative DSL (FDL) Debugger

Page 28: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

2828

Declarative DSL (BNF) Debugger Version 1

Page 29: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

2929

Declarative DSL (BNF) Debugger Version 2

Page 30: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

3030

Hybrid Robot Debugger (Java embedded in DSL)

Page 31: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

3131

Hybrid SWUL Debugger (DSL embedded in Java)

Page 32: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

Hybrid SWUL Debugger Video Demo

• There are 7 sets of demos in 3 different video formats

• Available at:

www.cis.uab.edu/wuh/ddf/demos.html

Page 33: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

3333

Generality Analysis of DDF

• 19 software components: 3,429 lines of code can be reused

DSL Category DSL NameNumber of Specific

Functions or ClassesCustomized

Lines of Code

Imperative DSL Robot Language 2 69

Declarative DSL FDL 3 89

BNF 5 261

Hybrid DSL Hybrid Robot Language 4 117

SWUL 5 152

Page 34: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

3434

JDB, GDB, and Cordbg Basic Debugging Commands Comparison

Debugging Actions

JDB (Java) GDB (C++) Cordbg (C#)

Set a Breakpoint stop at class: line number

break filename:linenumber

b[reak] [[file:] line number] | [[ class::] function [:offset]]

Step Over step step [count] n[ext] [count]

Display Value print class.staticfield

print expr p[rint] [variable name]

Terminate Exit quit Exit

Resume cont continue [ignore-count]

cont [count]

Page 35: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

3535

Related Work in the Area of Domain-Specific Language Debuggers

• Khepera– Support optimization code debugger; store transformation information in

a database– Execution time and cost are high

• JSR-045– Establishes standardized debugging tools for correlating Java bytecode

to the source code of languages other than Java – Does not have a debugging result mapping process

• ANTLR Studio– An example of a hybrid debugger – Based on the JSR-045 mapping mechanism– Display only Java variable values

• TIDE– Although uses gdb, TIDE requires the end-user to understand a precise

language specification in ASF+SDF – Knowledge of the rewrite rules in the built-in library is required to

construct a debugger for a new language

Page 36: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

36

Outline

• Background

• DSL Debugging Framework (DDF)

• DSL Unit Testing Framework (DUTF)

• AspectG

• Future Work and Conclusion

Page 37: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

37

DSL Application Software Errors!

• Before locating software errors how do we know there are bugs inside a DSL application?

Page 38: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

3838

DSL Unit Testing Framework (DUTF)

• Complement to the DDF – the DUTF assists in identifying the presence of errors and the DDF assists in isolating the specific location of the error

• Architecture and process of construction is similar to the DSL Debugging Framework (DDF) architecture

Page 39: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

3939

DSL Unit Testing Framework (DUTF)

Re-interpreter GPL Unit Test Engine

Source CodeMapping

Test Results Mapping

DSL

Test Cases Mapping

ANTLR Translator

DSL Test Cases

DSL Eclipse Test Result View

DSL Level

GPL Level

End-User

GPL Unit Test Cases

GPL

Page 40: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

4040

DSL Unit Testing Framework (DUTF)

• Test Cases Mapping

Robot DSL Unit Test Case1 TestCase testknight {2 Init position(0,0);3 Expectedposition(1,2);4 knight;5 AssertEqual (Expectedposition, position);6 }…

GPL Unit Test Case (JUnit)11 public void testkinght() {12 robot.x = 0;13 robot.y =0;14 int x=1;15 int y=2;16 robot.move_knight();17 assertEquals(x, robot.x);18 assertEquals(y, robot.y);19 }…

{1, “TestRobot.java”,11,“testknight”}

Page 41: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

4141

DSL Unit Testing Framework (DUTF)• Test Cases Mapping

Car FDL Unit Test Case1 TestCase testFeatures {2 Expectedfeature:(carbody, manual, highPower);3 use Car.FDL(All);4 Constraint C1: include pullsTrailer;5 AssertTrue(contain(Expectedfeature, feature));6 AssertEqual(6, numberof feature);7 }

GPL Unit Test Case (JUnit)11 public void testFeatures () {12 testFeatures.add("carbody");13 testFeatures.add("manual");14 testFeatures.add("highPower");…27 assertTrue(compareFeatures(testFeatures,parse(fc,root,cons)));28 assertEquals(6,getFeatureListNumber(parse(fc,root,cons)));…

Page 42: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

4242

DSL Unit Testing Framework (DUTF)Robot Language Unit Test Engine

Page 43: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

4343

DSL Unit Testing Framework (DUTF)FDL Unit Test Engine

Page 44: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

4444

Generality Analysis on DUTF

• Among 22 software components in DUTF, there are 3,001 lines of code that are generalized and reused to generate the different DSL unit test engines.

DSL Category DSL Name Number of Specific Functions or Classes

Customized Lines of Code

Imperative DSL Robot Language 2 239

Declarative DSL FDL 4 482

Page 45: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

4545

JUnit and NUnit Basic Unit Test Actions Comparison

Unit Test Actions JUnit (Java) NUnit (.Net languages)

Equality Assertion

assertEquals(expected, actual)

Assert.AreEqual(expected, actual);

Condition Test

assertTrue(actual) Assert.IsTrue(actual)

assertFalse(actual) Assert.IsFalse(actual)

assertNull(actual) Assert.IsNull(actual)

Utility Method fail() Assert.Fail()

Identity Assert assertSame(expected, actual)

Assert.AreSame(expected, actual)

Page 46: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

4646

Related Works in the Area ofDomain-Specific Language Tools

• ASF+SDF– Generate program analysis and transformation tools, and produce software

renovation tools – The unit testing tool support has not been reported

• JST– Complicated mechanism (e.g., Language extension)– Does not focus on language tools generation

• LISA– Can generate editor, parser, and visualizer for new languages– Debugger, test engine, and profiler are not the target language tools, LISA can

be used as the front-end of our framework• SmartTools

– Base on Java and XML technologies – Debugger, test engine, and profiler are not target language tools

• Other Related Testing Tools – Parameterized unit testing – Testing grammar-driven functionality – Generating unit tests using symbolic execution – Generating test inputs of AspectJ programs

Page 47: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

47

Outline

• Background

• DSL Debugging Framework (DDF)

• DSL Unit Testing Framework (DUTF)

• AspectG

• Future Work and Conclusion

Page 48: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

48

Crosscutting Concerns in Tool Generation Process

48

Crosscutting concerns observed in DSL grammars during the generation of DSL debuggers and unit test engines

How do we address such concerns in DSL grammars?

Page 49: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

4949

AspectG

• Crosscutting concerns emerge in different software artifacts (e.g., model, grammar, and source code)

• AspectG: A domain-specific aspect language for language grammars written in ANTLR

Page 50: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

5050

dsllinenumber=dsllinenumber+1;

gplbeginline=fileio.getLinenumber();

gplendline=fileio.getLinenumber();

filemap.print(" mapping.add(new Map("+dsllinenumber+",\"Robot.java\","+gplbeginline+","+gplendline+"));");

Addition of Debugging Support: A Crosscutting Grammar Concern

dsllinenumber=dsllinenumber+1;

gplbeginline=fileio.getLinenumber();

gplendline=fileio.getLinenumber();

filemap.print(" mapping.add(new Map("+dsllinenumber+",\"Robot.java\","+gplbeginline+","+gplendline+"));");

…command :( RIGHT {

fileio.print(" //move right");fileio.print(" x=x+1;");

fileio.print(" time=time+1;");

fileio.print(" ");

} |LEFT {

fileio.print(" //move left");fileio.print(" x=x-1;");

fileio.print(" time=time+1;");

fileio.print(" ");

}|…

Base Grammar

Duplicate Debugging Aspect Code

What if this line changes?Change here

Change here

Change ……

Page 51: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

5151

AspectG Design Challenge

• Design of a join point model for AspectG– What are the join points that will be captured in

AspectG?– Are AspectG join points static or dynamic?– What is an appropriate pointcut language syntax to

describe these join points?– What are advice in this domain?– Does the ordering of the advice matter? If so, how is

the weaving order prioritized? – How to deal with weaving conflicts and avoid infinite

weaving?

Page 52: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

5252

pointcut productions(): within(command.*);

pointcut count_gpllinenumber(): within(command.*) && match (fileio.print("time=time+1;"));

before(): productions() {dsllinenumber=dsllinenumber+1;}

before(): count_gpllinenumber() {gplbeginline=fileio.getLinenumber();}

after(): count_gpllinenumber() {gplendline=fileio.getLinenumber();}

after(): productions() {filemap.print(" mapping.add(new Map("+ dsllinenumber+", \"Robot.java\","+gplbeginline+ ","+gplendline+"));"); }

…command :( RIGHT { dsllinenumber=dsllinenumber+1; fileio.print("x=x+1;"); gplbeginline=fileio.getLinenumber(); fileio.print("time=time+1;"); gplendline=fileio.getLinenumber(); fileio.print(" "); filemap.print(" mapping.add(new Map("+ dsllinenumber+", \"Robot.java\","+gplbeginline+ ","+gplendline+"));"); }…

Join Point Model for AspectG

ANTLR Grammar AspectG

Page 53: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

5353

Weaving into DSL Grammars Using AspectG aspect debugging (

pointcut productions(): within{command.*};

pointcut count_gpllinenumber(): within {command.*} && match {fileio.print("time=time+1;")};

begin(): productions() { dsllinenumber=dsllinenumber+1;}

after(): count_gpllinenumber() {gplbeginline=fileio.getLinenumber(); gplendline=fileio.getLinenumber();})

default base domain Antlr.

external pattern mid_advice(bef_adv:semantic_strings, semico: QUOTED_STRING,

orig_stmt:semantic): semantic = 'mid_advice' in domain Antlr.

rule print_mid_stree(java_seq: semantic): semantic -> semantic = " \java_seq " -> "\mid_advice\(\mi_advice\(\) \, \semi\(\)\, \java_seq\)" if java_seq ~= "\:semantic \mid_advice\(\mi_advice\(\) \, \semi\(\)\, \java_seq\)".

pattern mi_advice(): semantic_strings = "command;gplbeginline=fileio.getLinenumber();".

pattern semi(): QUOTED_STRING = "\"time=time+1;\"".

public ruleset a = { print_mid_stree }.

DMS

DSL GrammarPARLANSE

Functions

DSL Grammar’ with Debugging

Aspect Weaved in

Transformation Rules

AspectG Compiler

Debugging Aspect Specification (AspectG)

…command :( RIGHT {

fileio.print("//move right"); fileio.print("x=x+1;");

fileio.print("time=time+1;");

fileio.print(" ");…

…command :( RIGHT { dsllinenumber=dsllinenumber+1; fileio.print("//move right"); gplbeginline=fileio.getLinenumber(); fileio.print("x=x+1;"); gplendline=fileio.getLinenumber(); fileio.print("time=time+1;"); filemap.print(" mapping.add(new Map("+dsllinenumber+", \"Robot.java\","+gplbeginline+","+gplendline+"));"); fileio.print(" ");…

Page 54: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

5454

Steps to Weave Debugging Aspects into an ANTLR Grammar

1. Specify ANTLR grammar specification

2. Specify Java semantic actions using DMS regular expression

3. Generate ANTLR Parser

4. Generate abstract syntax tree with ANTLR_ACTION nodes

5. Search ANTLR_ACTION nodes from the generated AST

6. Retrieve ANTLR_ACTION nodes and store them in a hash map

7. Retrieve associated string expression from each ANTLR_ACTION node

8. Modify the regular Java parser by changing the starting production

9. Parse the associated string expressions as regular Java statement lists

10. Transform the statement lists using the ASTInterface API

11. Regenerate the ANTLR_ACTION nodes with debugging aspects weaved in

12. Output the complete ANTLR AST (with modified action nodes)

Page 55: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

5555

aspect dsllinenumber (

pointcut productions(): within{command.*};

after(): productions()

{dsllinenumber=dsllinenumber+1;}

}

1 default base domain Antlr.

2

3 external pattern after_advice(af_adv:statement_string,

4 lefthandside: IDENTIFIER,

5 orig_stmt:semantic):

6 semantic = 'after_advice' in domain Antlr.

7

8 rule print_after_tree(java_seq: semantic): semantic -> semantic

9 =" \java_seq " -> "\after_advice\(\aft_advice\(\) \, \lhs\(\)\, \java_seq\)"

10 if java_seq~="\:semantic \after_advice\(\aft_advice\(\)\,\lhs\(\)\,\java_seq\)".

11 pattern aft_advice(): statement_string = "dsllinenumber=dsllinenumber+1".

12

13 pattern lhs(): IDENTIFIER ="command".

14

15 public ruleset a = { print_after_tree }.

DSL Line Number Counter Aspect in AspectG

Corresponding Generated Low-level Rule Transformation

Page 56: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

AspectG Video Demo

• There are 7 sets of demos in 3 different video formats

• Available at

www.cis.uab.edu/wuh/ddf/demos.html

Page 57: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

5757

Related Work in the Area of Aspect-Oriented Grammars

• AspectLISA– Uses an explicit join point model – Inheritance of advice and pointcuts– Used for language extension, not tool generation

• AspectASF – Weave crosscutting properties into an ASF+SDF

specification – Unit test engine is not mentioned in the generation list

Page 58: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

58

Outline

• Background

• DSL Debugging Framework (DDF)

• DSL Unit Testing Framework (DUTF)

• AspectG

• Future Work and Conclusion

Page 59: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

5959

Future Work• An extension of the current framework that enables

DSL profiling • Investigation into the scalable, reliable, and

extensible DSL testing framework – Application of different IDE platforms and GPLs – Adaptation of DDF and DUTF to address more complex

DSLs

• Generalize the framework in the areas of tool-independence and language-independence – A DSL IDE for end-user programmers– Adaptation to event grammar approach

• Exploiting further areas of aspect-oriented features within the language grammar

Page 60: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

6060

Contributions

• Provides a software framework empowering end-user developers with traditional software engineering debugging and unit testing capabilities at the DSL level

• Developed techniques for debugging and unit testing a diverse set of DSLs

• Developed an aspect language for grammars and impacted the area of Grammarware

Page 61: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

Publications Journal papers: 4 (2 under review), Conference and workshop papers: 7

– Hui Wu, Jeff Gray, and Marjan Mernik, “Unit Testing Support for Domain-Specific Languages,” under review, Journal of Research and Practice in Information Technology.

– Damijan Rebernak, Marjan Mernik, Hui Wu, and Jeff Gray, “Domain-Specific Aspect Languages for Modularizing Crosscutting Concerns in Grammars,” under review, IET Special Issue on Domain-Specific Aspect Languages.

– Hui Wu, Jeff Gray, and Marjan Mernik, “Grammar-Driven Generation of Domain-Specific Language Debuggers,” accepted for publication, Software Practice and Experience, 2007.

– Pedro Rangel Henriques, Maria Joao Varanda Pereira, Marjan Mernik, Mitja Lenic, Jeff Gray, and Hui Wu, “Automatic Generation of Language-based Tools using LISA,” IEE Proceedings - Software, vol. 152, no. 02, April 2005, pp. 54-69.

– Hui Wu, Jeff Gray, Suman Roychoudhury, and Marjan Mernik, “Weaving a Debugging Aspect into Domain-Specific Language Grammars,” ACM Symposium for Applied Computing-Programming for Separation of Concerns Track, Santa Fe, NM, March 2005, pp. 1370-1374.

– Jeff Gray, Jing Zhang, Yuehua Lin, Hui Wu, Suman Roychoudhury, Rajesh Sudarsan, Aniruddha Gokhale, Sandeep Neema, Feng Shi, and Ted Bapty, “Model-Driven Program Transformation of a Large Avionics Framework,” Generative Programming and Component Engineering, Springer-Verlag LNCS, Vancouver, BC, October 2004, pp. 361-378.

– Yuehua Lin, Song Zhou, Chuanxi Xu, Hui Wu, Jing Zhang, Juanqin Wang, Jeff Gray, and Barrett Bryant, “E-R Modeler: A Database Modeling Toolkit for Eclipse,” Annual ACM SE Conference, Huntsville, Alabama, April 2004, pp. 160-165.

– Suman Roychoudhury, Jeff Gray, Hui Wu, Jing Zhang, and Yuehua Lin, “A Comparative Analysis of Meta-programming and Aspect-Orientation,” Annual ACM SE Conference, Savannah, GA, March 2003, pp. 196-201.

– Damijan Rebernak, Marjan Mernik, Hui Wu, and Jeff Gray, “Domain-Specific Aspect Languages for Modularizing Crosscutting Concerns in Grammars,” GPCE Workshop on Domain-Specific Aspect Languages, Portland, OR, October 2006.

– Hui Wu, “Grammar-Driven Generation of Domain-Specific Language Testing Tools,” Object-Oriented Programming, Systems, Languages & Applications, Doctoral Symposium 2006, Portland, OR, October 2006.

– Hui Wu, “Automated Generation of Testing Tools for Domain-Specific Languages,” IEEE/ACM International Conference on Automated Software Engineering, Doctoral Symposium, Long Beach, CA, November, 2005.

Page 62: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

6262

Questions?

Video demonstrations and papers available at:http://www.cis.uab.edu/wuh/ddf

Acknowledgement:The work presented in this dissertation was supported in part by the IBM Innovation Grant.

Page 63: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

63

Backup Slides

Page 64: 1 Grammar-Driven Generation of Domain-Specific Language Testing Tools Using Aspects Ph.D. Defense Hui Wu wuh@cis.uab.edu  10/29/2007.

6464

aspect gplbeginlinenumber (

pointcut count_gpllinenumber():

within {command.*} && match {fileio.print("time=time+1;")};

after(): count_gpllinenumber()

{gplbeginline=fileio.getLinenumber();})

1 default base domain Antlr.

2

3 pattern semi(): QUOTED_STRING ="\"time=time+1;\"".

4

5 external pattern mid_advice(bef_adv:semantic_strings,

6 semico: QUOTED_STRING,

7 orig_stmt:semantic):

8 semantic = 'mid_advice' in domain Antlr.

9

10 rule print_mid_stree(java_seq: semantic): semantic -> semantic

11 = " \java_seq " -> "\mid_advice\(\mi_advice\(\) \, \semi\(\)\, \java_seq\)"

12 if java_seq~="\:semantic\mid_advice\(\mi_advice\(\) \, \semi\(\)\, \java_seq\)".

13

14 pattern m_advice():semantic_strings="command;gplbeginline=fileio.getLinenumber();".

15

16 public ruleset a = { print_mid_stree }.

GPL Line Number Counter Aspect in AspectG

Corresponding Generated Low-level Rule Transformation


Recommended