Model-Based Approaches to Reengineering Web Pages Laurent Bouillon, Jean Vanderdonckt Université catholique de Louvain Belgian Lab. of Computer-Human Interaction IAG-ISYS, Place des Doyens, 1 B-1348 Louvain-la-Neuve, Belgium +32-10/47.{8349, 8525} {bouillon, vanderdonckt}@isys.ucl.ac.be Jacob Eisenstein University of Southern California Department of Computer Science 941 W. 37th Place Los Angeles, CA 90089-0781 USA +1 213 740 4496 [email protected]ABSTRACT Today's web sites are increasingly being accessed through a wide variety of computing platforms ranging from the workstation to a laptop and through multiple access devices such as Internet Screen Phone, TV Set Top box, PDA, and cellular phones. Web sites are rarely de-signed and devel- oped to fit such a large variety of contexts of use as each context (e.g., each computing platform, each device) has its own set of constraints. This pa-per describes a model-based approach for reengineering web pages into a presentation and a dialog model stored with XIML, a model-based user- interface specification language. These models are then further exploited to reengineer other user interfaces either for the same context of use (by changing presentation de- sign options) or for different contexts of use (by changing properties of computing platform model). For this purpose, three key elements of the presentation model (i.e. presenta- tion units, logical windows, and abstract interaction objects) and two key elements of the dialog model (i.e., navigational structure and transition) were defined. Keywords Model-based user-interfaces, reengineering, knowledge bases, web sites, presentation model. INTRODUCTION The growing interest in web site design, development, and evaluation, as well as the underlying cost, has increased the demand for engineering methods that are more structured than traditional ad hoc development or “rush to code” ap- proaches. It is obviously desirable that any web site should be made accessible from the widest range of computing platforms, including the variation of browsers on each platform, for the widest range of users. The rapid evolution of the HTML language itself exacerbated the need for backward compati- bility, for instance, from version 2.0 to 4.0. Moreover, the technological push of newly marketed access devices—such as PDAs and WAP-enabled telephones—has generated a new need to access the same web site, from different access devices. With the demand for Internet access increasing for e-mail, shopping, electronic commerce, current events, and quick information, the need for Internet-capable access de- vices has extended beyond the professional desktop to the home office, the other rooms of the house, and beyond. This demand can only increase, as we will surely become more dependent on the web for information. To address these demands, ad hoc development is no longer acceptable in terms of the cost and time required for soft- ware engineering and maintenance. Forward engineering [3] has been consequently considered as a good candidate for producing quality web sites. For instance, model-based approaches [22,23,26] can produce a user interface (UI) for a web site by exploiting knowledge captured in various models, such as the presentation and dialog models. Forward engineering is said to be simple when one UI is generated from the initial models; when many UIs are gen- erated, it is said to be multiple. However, most existing web sites have been developed without any model-based ap- proaches, thus creating a need for reverse engineering to generate the models, which can later be exploited by future forward engineering, simple or multiple. In this paper, we first report on what the assumptions are for adopting a model-based approach for forward engi- neering. Then, we describe a model-based approach for re- verse engineering web pages implemented as an automatic or mixed-initiative process in the VAQUITA software, with an eye to applying forward engineering subsequently. Reengineering methods are then considered to produce new UIs for other contexts of use, thus creating a capability to rapidly produce UIs for different computing platforms, various access devices, etc. As supporting all variations of contexts of use is very am- bitious, we will focus on accommodating those features that are most likely to vary across platforms. We will then com- pare the above model-based approach with respect to re- lated work and present some conclusions in a general framework for UI reverse engineering of web pages. In this framework, other forms of re-engineering will be situated with respect to the approach we adopted. We will also in- troduce the category of redesigning with respect to previ- ously defined approaches.
Transcript
Model-Based Approaches to Reengineering Web Pages
Laurent Bouillon Jean VanderdoncktUniversiteacute catholique de Louvain
Belgian Lab of Computer-Human InteractionIAG-ISYS Place des Doyens 1
B-1348 Louvain-la-Neuve Belgium+32-10478349 8525
bouillon vanderdoncktisysuclacbe
Jacob EisensteinUniversity of Southern CaliforniaDepartment of Computer Science
941 W 37th PlaceLos Angeles CA 90089-0781 USA
+1 213 740 4496jacobisiedu
ABSTRACTTodays web sites are increasingly being accessed through awide variety of computing platforms ranging from theworkstation to a laptop and through multiple access devicessuch as Internet Screen Phone TV Set Top box PDA andcellular phones Web sites are rarely de-signed and devel-oped to fit such a large variety of contexts of use as eachcontext (eg each computing platform each device) has itsown set of constraints This pa-per describes a model-basedapproach for reengineering web pages into a presentationand a dialog model stored with XIML a model-based user-interface specification language These models are thenfurther exploited to reengineer other user interfaces eitherfor the same context of use (by changing presentation de-sign options) or for different contexts of use (by changingproperties of computing platform model) For this purposethree key elements of the presentation model (ie presenta-tion units logical windows and abstract interaction objects)and two key elements of the dialog model (ie navigationalstructure and transition) were defined
KeywordsModel-based user-interfaces reengineering knowledgebases web sites presentation model
INTRODUCTIONThe growing interest in web site design development andevaluation as well as the underlying cost has increased thedemand for engineering methods that are more structuredthan traditional ad hoc development or ldquorush to coderdquo ap-proachesIt is obviously desirable that any web site should be madeaccessible from the widest range of computing platformsincluding the variation of browsers on each platform forthe widest range of users The rapid evolution of the HTMLlanguage itself exacerbated the need for backward compati-bility for instance from version 20 to 40 Moreover thetechnological push of newly marketed access devicesmdashsuchas PDAs and WAP-enabled telephonesmdashhas generated anew need to access the same web site from different accessdevices With the demand for Internet access increasing fore-mail shopping electronic commerce current events andquick information the need for Internet-capable access de-
vices has extended beyond the professional desktop to thehome office the other rooms of the house and beyondThis demand can only increase as we will surely becomemore dependent on the web for informationTo address these demands ad hoc development is no longeracceptable in terms of the cost and time required for soft-ware engineering and maintenance Forward engineering[3] has been consequently considered as a good candidatefor producing quality web sites For instance model-basedapproaches [222326] can produce a user interface (UI) fora web site by exploiting knowledge captured in variousmodels such as the presentation and dialog models Forward engineering is said to be simple when one UI isgenerated from the initial models when many UIs are gen-erated it is said to be multiple However most existing websites have been developed without any model-based ap-proaches thus creating a need for reverse engineering togenerate the models which can later be exploited by futureforward engineering simple or multiple In this paper we first report on what the assumptions arefor adopting a model-based approach for forward engi-neering Then we describe a model-based approach for re-verse engineering web pages implemented as an automaticor mixed-initiative process in the VAQUITA software withan eye to applying forward engineering subsequentlyReengineering methods are then considered to produce newUIs for other contexts of use thus creating a capability torapidly produce UIs for different computing platformsvarious access devices etcAs supporting all variations of contexts of use is very am-bitious we will focus on accommodating those features thatare most likely to vary across platforms We will then com-pare the above model-based approach with respect to re-lated work and present some conclusions in a generalframework for UI reverse engineering of web pages In thisframework other forms of re-engineering will be situatedwith respect to the approach we adopted We will also in-troduce the category of redesigning with respect to previ-ously defined approaches
FORWARD ENGINEERING OF WEB PAGESFor many years the research and development communityof model-based approaches has shown interest for forwardengineering [2326] with many models (fig 1) A domain model [4] defines the objects that a user can
view access and manipulate through a UI In nature it isvery similar to a data model but it is also intended to ex-plicitly represent the attributes of objects and the rela-tionships among the various domain objects (eg an en-tity-relationship model or an object-oriented model)
A presentation model [292425] is a representation ofthe visual haptic and auditory elements provided by a UIto its users For example a presentation element might bea window that contains additional elements such as wid-gets that appear in that window This model also includesstatic presentation attributes such as font styles and ori-entation of button groups
A dialog model [819] defines the way in which the pres-entation model interacts with the user It represents theactions that a user may initiate via the presentation ele-ments and the reactions that the application communi-cates via those same elements
An application model [22] captures some abstraction ofse-man-tic functions of the semantic core componentcalled by the UI and maintains connections with it Itspecifies services applications provided to the UI
For almost a decade models have been exploited to auto-matically generate a UI in two ways (fig 1) [26]1 At design time by exploiting passive models models are
parsed to generate the UI code for a given computingplatform The generation process ranges from completelyautomatic (the designer does not see anything during thisprocess) to full mixed-initiative (where the systemprompts the designer for design options to decide for
control parameters) The generated code is integratedwith the code of the semantic core component and com-piled to obtain the intended application
2 At run time by exploiting active models the specifica-tions contained in the models are scanned and interpretedat run-time to obtain a running UI connected to the codeof the semantic core component For instance FUSE [8]uses an attributed grammar to abstract a UI to execute thespecification contained in the models Apart from beingexecutable at run-time this approach is more suitable forgenerating UIs depending on parameters that only be-come known at run-time The alternative is to attempt topredict all alternatives at design-time and then generate aUI that supports all these circumstances But the resultingcode would be inefficient more complex to produce notso compact In some cases it may be impossible to haveall UI alternatives contained into one single piece ofcode
Whether at design-time or at run-time one or many modelsare typically exploited to generate one UI for one applica-tion To generate many UIs for the same application and inorder to support user-centered design the traditional model-based approach has been extended to a task-based approach(fig 2) where two new models are introduced A task model is a description of the task to be accom-
plished by the user of an application through the UI In-dividual elements in this model represent specific actionsthat the user may undertake Information regarding sub-task ordering (eg sequence unordered) as well as con-ditions on task execution is also included
A user model represents the different types of users of anapplication It defines attributes roles of users and mayinclude information describing their preferences
Scenario Interactiveapplication
Taskmodel
Domainmodel
Usermodel
Semanticcore code
User inter-face code
UI I
nteg
ratio
n
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
User-task elicitation User interface modeling and generation
Application modeling and development
Interactiveapplication
Domainmodel
Semanticcore code
User inter-face code
UI I
nteg
ratio
n
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
Domain modeling User interface modeling and generation
Application modeling and development
Fig 1 Traditional model-based approach
Fig 2 Task-based approach
REVERSE ENGINEERING OF WEB PAGESSeveral environments today support forward engineering ofWeb pages as presented in Section 2 For example AllegroServe (httpallegroservesourceforgenet) is a web serverthat can dynamically generate Web pages and web-enableexisting applications with a browser front-end However asmost existing Web sites were not built according to such anapproach adopting a reverse engineering of these webpages is a required preliminary step to further benefit fromany future forward engineering The goal of reverse engi-neering here is to recover both presentation and dialogmodels of a set of web pages (fig 3) The UI code shouldbe extracted from the global code and separated from thecode of the semantic core component From this UI codepresentation and dialog models need to be reverse engi-neered For this purpose relevant abstractions need to bedefined for each of these two models
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering
Fig 3 Reverse engineering
Abstractions for the Presentation ModelUI presentation of web pages can be abstracted using fourconcepts the hierarchy of which is depicted in fig 41 Concrete Interaction Object (CIO) this is a real object
belonging to the UI world that any user can see (eg textimage animation) or manipulate such as a push button alist box a check box A CIO is said to be simple if it can-not be decomposed into smaller CIOs A CIO is said tobe composite if it can be decomposed into smaller unitsTwo categories are distinguished presentation CIOwhich is any static CIO allowing no user interaction andcontrol CIO which support some interaction or UI con-trol by the user
2 Abstract Interaction Object (AIO) this consists of an ab-straction of all CIOs from both presentation and behav-ioral viewpoints that is independent of any given com-puting platform AIOs have been used success-fully forboth forward [892024] and reverse engineering[101314] Each AIO is here identified by a unique ge-neric name (eg check box) general and particular ab-stract attributes (eg height width color states) abstractevents (eg value selection mouse click) and abstractprimitive (eg Pr-EditBoxContent) By definition anAIO does not have any graphical appearance but eachAIO is connected to 0 1 or many CIOs having differentnames and presentations in various computing platforms32 AIOs were described in a ldquois-ardquo hierarchy of classesinto a knowledge base [25]
3 Logical Window (LW) this root window can be consid-ered either as a logical container for AIOs or as a physi-cal window a dialog box or a panel Every window is it-self a composite AIO as it is composed of other simple orcomposite AIOs All LWs are supposed to be physicallyconstrained by the users screen The three abstractionsthat have been considered so far are quite related to ex-isting presentation objects However none of the presen-tation abstractions described thus far are closely relatedto task aspects The following abstraction is introducedfor this purpose
4 Presentation Unit (PU) a PU is assumed to be the com-plete presentation environment required for carrying out aparticular interactive task Each PU can be decomposedinto one or many LWs which may or may not be all dis-played on the screen simultaneously Each PU is com-posed of at least one window called the basic windowfrom which it is possible to navigate to the other win-dows For instance a tabbed dialog box is here mappedonto a PU which is itself de-composed into LWs corre-sponding to the dialog box appearances depending on theactive tab conversely a web form can be mapped onto acomposite AIO in a particular LW of a given PU
Presentation Unit
Logical Window
Composite AIO
Simple AIO
Presentation AIO Control AIO
1-n 1-n
0-n
0-n
0-n
0-n
0-n0-n
is-a
is composed of
1-n 1-n
1-n 1-n
Fig 4 Hierarchy of presentation concepts
Abstractions for the Dialog ModelWeb sites are typically developed within a graphical ortextual editor according to a page amp link approach thepresentation of a family of web pages is first designed andthe links are added as this design proceeds The abstractionlevel of these design concepts can be raised to more en-compassing concepts of a transition and navigational struc-ture A transition is defined as any control capability ena-bling users to move from a source LW to a target LW Itconsequently consists in a control AIO (eg an anchor anicon a push button) allowing a directional control (egunidirectional or bi-directional) between LWs with opera-tions on these LWs maximization titling minimizationdisplay with tiling technique display with normal overlap-ping technique display with user-defined overlapping tech-nique display with system-defined overlapping techniqueor closing A navigational structure applies any graph pat-tern between LWs of a given PU of a given type
Sequential navigation This navigation structure enables auser to move between LWs according to a pre-defined se-quence In this structure unidirectional transition allows auser to move from one LW to another (fig 5) while bi-directional transition allows a user to switch between a setof LWs these LWs being closed restored or redisplayed
Fig 5 Unidirectional and bi-directional sequences
Indexed navigation This structure is aimed at providing aregular index mechanism in which multiple target LWs aresuggested from one single source LW Having unidirec-tional transition prevents the user to come back to thesource LW while bi-directional transition will (fig 6)
PAGE
PAGE
Fig 6 Unidirectional and bi-directional indexesGuided navigation Guided navigation is similar to aguided tour a source LW suggests the user a first LW toswitch to this first LW is connected to a second one anddo forth to the last LW suggesting the user to come back tothe source LW In some sense this navigation is similar to asequential navigation where the dialog is closed Unidirec-tional transition only allows the user to move to the con-nected LW one way whereas bi-directional transition al-lows a round trip (fig 7)
PAGE
PAGE
Fig 7 Unidirectional and bi-directional guided toursMixed navigation This navigation type groups two previ-ously defined abstractions into a new one a guided naviga-tion is augmented by an index to combine navigation flexi-bility provided by an index and guidance provided by aguided tour This type therefore allows a user to be guidedin a guided tour and leave the tour or reinitiate it at everystep (fig 8)
PAGE
PAGE
Fig 8 Unidirectional and bi-directional mixed tours
Global navigation This navigation type is considered tobe the most complete as it allows the user to move from anyLW to any other one Bi-directional transitions are thereforeconsidered not useful as they reproduce the same transition(fig 9)
Fig 9 Global navigationUsing transition and navigation structures as basic conceptsnot only raise the abstraction level with respect to the tradi-tional page amp link model but also enables designers to ma-nipulate entire navigation structures and to combine them ina more logical way Rather than modifying transition con-trols on each LW a collection of behaviorally-equivalenttransition controls is established and maintained across aseries of related LWs Navigation structures can be definedin isolation or directly combined by referring to a particularLW inside their definitionIn fig 10 for instance the designer can first decide to havea sequential navigation A across a family of LWs named A1through A4 Any LW of this navigational structure can thenbe a good candidate to form the source LW for any othertype of navigational structure For example the logicalwindow A3 can be designed as the source LW for an unidi-rectional indexed navigation Any combination of the basicnavigational structures can be imagined However someobserved structures do not exactly match these basic orcombined structures In this case the closest combination isselected instead thus introducing some modification in theoriginal navigation
B1 B2 B3 B4
B
A1 A2 A3 A4A
Fig 10 Combining navigation structures
REVERSE ENGINEERING ALGORITHMHaving defined abstractions relevant to presentation anddialog models we can exploit them to reverse engineerthese abstractions for a series of web pages This processbasically consists in (i) downloading the HTML code ofWeb pages of interest (ii) extracting from their code the UIpart and separating it from the part belonging to the seman
tic core component and (iii) submitting it to the staticanalysis algorithm for reverse engineering depicted in fig11 Thanks to the HTML code accessibility remote reverseengineering of Web pages is permitted in addition the de-sign of automated tools for reverse engineering should bemuch easier for web pages than for compiled program codein C or Pascal The phases of this algorithm are now de-tailed in the subsequent subsections as they are consideredin VAQUITA (httpwwwisysuclacbebchiresearchvaquitahtm) a tool that enables designers to reverse engineer aweb page into one or several presentation models Cur-rently only the presentation model is reverse engineered
Inclusionexclusion of presentation elementsInitialization of the Web pages pool
Web pages pool empty
Selection of an individual Web page
Identification of Concrete Interaction Objects
Transformation of Concrete Interaction Objectsinto Abstract Interaction Objects
Selection of Logical Window
Hierarchy building for LWs and PUs
Update vectors of links and Web pages pool
Identification of links
Abstraction of links into Windows Transitions
Identification of Navigation Structures
Buildingofpresentationmodel
Buildingofdialogmodel
yes
no
Fig 11 Reverse engineering algorithmInitialization To initiate the process the designer specifieswhether she wants to reverse engineer a whole web site (byproviding the home pagersquos URL) any portion of a site (byproviding a starting URL and the depth level up to whichpages will be considered) or a series of web pages (by en-tering their URLs locally andor from the Web) A first poolof candidate pagesrsquo URLs is formedTo allow more flexibility in the reverse engineering proc-ess the designer can include or exclude not only anyHTML tag or element but also any attribute of each tag Toavoid reparametrization this setting canbe saved in a con-figuration file for future usage
Presentation modeling An individual web page is firstselected in the pagesrsquo pool and then submitted to identifica-tion of CIOs The basic idea of the algorithm is to identifyindividual HTML elements to map them onto relevantAIO while building a hierarchy of AIOs The HTML codeof the web page is parsed to identify these HTML elementsThe mapping is based on a set of heuristics that depend onthe HTML element or tag analyzed Examples of such heu-ristics include Each web page is mapped onto a LW in the selection of
LW phase The ltTITLEgt if any is mapped onto the labelslot of this AIO Any HTML element can be mapped toone or many AIOs with one or many properties included
Heading levels (eg H1 H2 H3) are mapped onto re-spective composite presentation AIOs and to somestructure in the progressively forming hierarchy The textprovided within the ltHgtltHgt scope is mapped onto theLabel abstract attribute of the AIO
Each ltFORMgt tag is mapped onto a composite controlAIO Multiple or embedded forms generate multiplecomposite AIOs on corresponding hierarchy levelsltTRgt ltTDgt tags denoting tables are treated similarly
Web pagersquos frames are mapped onto separate LWs asthey are themselves partial views on other pages
Static banners icons graphics illustrations and GIF filesare listed in the ldquoimagesrdquo category of a ldquoGraphicsrdquo sim-ple presentation AIO animated GIFs or video are listedin its ldquoanimationrdquo category while JPEG files are believedto relate to the ldquophotographrdquo category
Portions of text are equally treated like labels with theirstatic properties being fed into the abstract attributes ofthe AIO eg the currently active BGCOLOR is mappedonto BackGroundColor and TEXT is mapped onto Fore-GroundColor When text is subject to an anchor an addi-tional control AIO is created
HTML proprietary CIOs are mapped onto their corre-sponding simple control AIOs Examples include theltSELECTgt and ltTEXTAREAgt tags and the TYPE at-tribute of the ltINPUTgt tag The NAME tag is mappedonto an identification label for the related AIO The con-tent of the VALUE tag is mapped onto the abstract attrib-ute DefaultValue and so forth
Dialog modeling To reverse engineer a dialog model andfinding out instances of transitions and navigational struc-tures a vector of links is created for each page according tothe following format
V(URL)=list of intra-page links list of extra-page linkswhere each link has the form
L = (AIO_name target_URL link_type occurrence)where AIO_name = identifier in the presentation model
of the AIO holding the link source consideredtarget_URL= URL of the page linkedlink_type = (intra-page extra-page request)occurrence = amount of same links in the page
Once any individual page has been processed regarding toits own presentation model and regarding to its own vectorof links this vector is parsed to update the pool of candi-date web pages to analyze In the identification of links thisanalysis exploits positive and negative filters defined in theinitialization For example all external links or all extra-page links up to a certain depth level are omitted The ana-lyzed web page is then withdrawn from the pagesrsquo poolThe pool is examined to find a candidate recursively untilno candidate page still exists After exhausting the wholepagesrsquo pool the remainder of both dialog and presentationmodels is reverse engineered First vectors of links thathave been built for the collection of web pages in consid-eration are examined to identify LW transitions from eachpair of LWs Transformation rules include Any link with multiple occurrences within the same page
is reduced to a single link to avoid repetition All links with multiple sources within the same page
(eg a textual link a push button an icon) to the sametarget page are gathered Depending on parameters allinstances of such links can be kept or reduced to any par-ticular type For instance links can be restricted to onlyone occurrence of anchor and one occurrence of icon Inthis case the presentation model can combine both AIOsinto a single one of the ldquoiconrdquo type whose label is theanchor text Again positive and negative filters canautomatically expand or reduce the scope of AIOs inconcern for example considering only push buttons andforgetting all other types of AIOs for links
Any one-way link is transformed into a unidirectionaltransition Depending on the link type the operations onthe source and target LWs are specified1 When a link replaces one page by another the transi-
tion will contain a ldquocloserdquo operation on the source LWand a ldquomaximizerdquo operation on the target LW
2 When a link just pops up another page the transitionwill contain only a ldquodisplay with normal overlappingrdquooperation on the target LW
3 When a link pops up another page in a constrainedwindow the transition will contain a ldquodisplay withsystem-defined overlappingrdquo on the target LW
Any pair of reciprocal links between two pages is trans-formed into a single bi-directional transition between thetwo LWs
Any intra-page link is transformed into a control AIObranching to the related composite AIO belonging to thesame LW Normally intra-page links are intended tofasten visualization and avoid scrolling rather than pro-viding LW transitions This information is still re-cordedas in the future these AIOs may be replaced by some-thing else for example another LW In this design optionparts of a web page that are accessed by intra-page linksmay be replaced by separate LWs
Transitions between logical windows abstracting these linksare subsequently stored in a matrix Each matrix columndenotes a source LW while each matrix line denotes a tar-get LW Thus any cell denotes a window transition be-tween a pair of windowsThe resulting matrix is then passed to a pattern-matchingalgorithm that is in charge of identifying navigationalstructures among the LWs This algorithm begins byguessing the simplest navigational structures (such as se-quential or indexed navigation) and continues with morerich structures (such as guided or mixed navigations) Asthe window transition matrix can be rather sparse or nonregular finding the exact materialization of these naviga-tional structures can be infrequentThe next step is to generalize the existing structure to theclosest abstract structure type Of course each existingstructure can be generalized into a global navigation struc-ture But this network of LWs probably generates too manywindow transitions In order to moderate this generalizationprocess a threshold can be specified This parameter canfor example regulate how many transitions can be added inorder to match to the closest strict navigational structure Asthis algorithm may be incomplete the designer is allowed tocheck the resulting navigational structures The designermay accept reject or modify the structure produced the al-gorithm and she also may declare undiscovered structuresThis concludes the building of the dialog model
RE-ENGINEERING OF WEB PAGESNow that presentation and dialog models are available fromreverse engineering of web pages these two models cantheoretically benefit from forward engineering any model-based approach should be able to generate a new UI fromthese two models to be further re-integrated with the se-mantic core component Before reengineering web pagesinto a new UI according to the process depicted in fig 12 itis important to specify the context in which this new UI will
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and reengineering
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Fig 12 UI Reengineering
take place As indicated in the introduction technologicalpush and user pull are both increasing the demand for ac-cessing web sites in various contexts of use These varia-tions may depend on1 User several users will access the site each user be-
longing to a specific stereotype of the user populationEach stereotype ha its own set of parameters such aspreferences customized features interaction historycognitive profile typing vs selection skills motor- sight- or auditive-impairment and specific needs
2 Computing platform many different computing plat-forms can be used to access a site and each platformbrings its own set of constraints such as screen resolu-tion number of colors color palettes planes UI lan-guage (programming language script language ortab)based language are very different) network and in-teraction devices These two last constraints possessthemselves further constraints such as network band-width interaction devices capabilities and availability
3 Environment users access a site in different environ-ments Ambient conditions add a new set of constraintsincluding physical parameters (eg noise light workingspace room heat) and psychological social parameters(eg stressful time slot productivity task criticality un-stable environment collaborative or cooperative condi-tions home vs work)
Reengineering web pages to support all these variations ofthe context of use would be an ambitious task As we as-sumed in the introduction to mainly address cross-plat-formsupport this paper is focused on adding a computing plat-form model only We feel that techniques that are found tobe useful for addressing platform constraints will also beable to handle constraints that come from the user and theenvironment The rapid proliferation of various platformsfor web accessibility makes this a particularly timely con-cern Users increasingly want to compute while movingbetween locations thus moving from one platform (eg aWindows PC) to another (eg a Windows CE-compatibledevice) Other users do not want to move from one place toan-other but want to have multiple access from a singleplace (eg connecting a PDA to a PC)Many parameters vary across platforms VAQUITA onlysupport screen resolution variation for one UI language(ie HTML) as summarized in table 1 The concepts re-main the same without loss of generality
Context with no parameter varyingIn this context a new presentation model can be inferredfrom the existing models by considering at least 5 designs1 Redistribution of LWs within a single PU2 Redistribution of Composite AIOs within a single LW3 Redistribution of Simple AIOs within a single LW4 Redistribution of all AIOs within a single LW5 Redistribution of all AIOs within a single PU
Individual AIOs should remain unchanged but any redistri-bution of these elements can be imagined provided that thescreen constraints are satisfied To verify these constraintsthe presentation model needs to be enriched by two AIOabstract attributes typical height and length These attrib-utes are assigned to an average value in pixels for dimen-sion-fixed widgets (eg a check box a radio button)
These attributes are assigned to a value that equals a multi-plying coefficient in pixels times the character length fordimension-varying widgets (eg an edit box a list box)Other dimensions are inferred or estimated from the origi-nal CIOs themselves (eg an image thanks to theHEIGHT=x WIDTH=y tags) Analyzing all possible abovereconfigurations is beyond the scope of this paper We referto [225] for analyzing redistribution of LWs within a samePU (ie minimal maximal inputoutput functional userdefined grouped ungrouped)
Context with varying UI languageIn this context only the underlying UI language changesThis often occurs when multiple computing platforms run-ning different operating systems and presentation managersare considered Some languages are Internet-compatible(eg HTML Java WML) some are not (eg Visual Ba-sic C++) Beyond redistributions described in Section 41this context poses a new challenge all these languages donot share similar capabilities in terms of CIOs of the sourcecomputing platformFor instance an AIO may be mapped onto zero CIOs (thecorresponding CIO does not exist in the target language)one (the most frequent and easy case where a one-to-onerelationship exists between an AIO and its correspondingCIO) or many CIOs (several CIOs are required to repro-duce the same behavior of the intended AIO) To allowthese mappings a platform-specific transformation tableallows any AIO instance to be assigned to correspondingCIOsWhen no corresponding CIO exists an alternative CIO isproposed (ultimately the edit box is suggested) but can betailored by the designer Nevertheless most access devicesor specific computing platforms coming with their ownproprietary language also impose some screen resolutionchange For instance Wireless Markup Language (WML)can only be used on cellular phones Voice XML only onwith a screen reader A third context is then considered
Context with varying screen resolutionIt is a property of HTML that many HTML-compliantbrowsers on many types of screen can display it Althoughthis display is technically feasible (ie the browser adaptsthe presentation with respect to the platform) it may be un-usable to the end-user for one or another reason eg lossof information structure overcrowded screen too manyscroll bars image reduction) Therefore it makes sense toconsider how to re-engineer the UI for different screenresolutions This need is exacerbated by multiple HTMLbrowsers On different platforms equipped with different monitors
(ranging from workstation screen to small laptop) On different access devices with built-in browser (rang-
ing from the Internet ScreenPhone with 640x480 reso-lution to HTML-compatible PDA)
If the screen resolution is increasing the centered table withthree columns can be used to keep a presentation centeredon the screen with borders (fig 13) If the resolution is de-creasing the above redistributions are no longer applicableThree strategies have been explored so far
Fig 13 Presentation centered on the screen1 AIO replacement each AIO definition is augmented
with a list of potential replacements considered as eitherbehaviorally equivalent but consuming less space (ega list box to a drop-down list box) or degraded (eg anaccumulator to a multiple-value list box)
2 AIO size reduction typical height and length of AIOsare submitted to a possible reduction depending on us-ability constraints For example images and icons canbe reduced to a certain limit specified by a thresholdeg the minimally usable size of icons
3 AIO ungrouping when individual AIOs cannot be re-duced according to the two first strategies the distribu-tion of AIOs may be reconfigured AIOs can be groupedor ungrouped and new LWs can be created to contain alimited amount of AIOs
Again the development of these strategies bears further ex-planation which we hope to conduct in a later paper
Context with varying language and resolutionThis context is most likely to occur when a completely dif-ferent computing platform of access device is considered
For example a cellular phone only supports WML WebTVonly accepts a subset of HTML V3 and screen readersonly accept VoiceXML This context can be handled usinga combination of the approaches described in previouscontexts
RELATED WORKCross-platform development is not new as several environ-ments provide support for this purpose Galaxy [6] andOpen Interface [17] render the same UI on different plat-forms with their native look amp feel while SUIT [18] em-ploys a unique UI definition that can be processed on mul-tiple platforms However not one of these systems trulyadopts a model-based approach although SUITs commondefinition holds some presentation abstractions CT-UIMS[9] pioneered the platform model by supporting some AIO[15] redistribution for OSFMotif large screen and smallMacintosh screens AUIDL [101112] is probably the firstset of abstractions for reverse and re-engineering UIs frominternal hierarchical structures type and variable declara-tions a UI can be recovered in IDL with a presentationmodel (based on OO paradigm) and a dialog model (basedon Milnerrsquos process algebra) MORPH exploits productionrules [13] to infer AIOs [14] from CIOs and thus producesa graphical UI from a textual UI [15] This transformationalapproach is similar in principle to ours Forward engineer-ing can be executed by model composition [20] binding[21] or derivation thus proving that the approach is feasi-ble MORALE [1] is an extensive set of techniques andtools for reverse engineering legacy systems rather thanweb pages CELLEST [7] reverse engineers similar sys-tems but into DHTML thus adopting active models ratherthan passive models
CONCLUSIONThis paper has introduced a model-based approach sup-porting both reverse and re-engineering of web pages Thearchitecture outlined in fig 14 assumes that all models arestored in a model repository each model being specifiedwith the eXtensible Interface Markup Language (XIML)promoted by the XIML Consortium [28] This model tex-tual specification is declarative analyzable and editable[22]
HTMLpage
VAQUITA Web pagereverse engineering
User interfaceRe-engineering
XIMLPresentation
model
HTMLpage
WMLdeck
hellip
Model editor validator
Fig 14 Model-based approach architecture
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and redocumentation
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Platformmodel(1) (2)
Fig 15 UI Redocumentation
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
User interface reverse engineering
Taskmodel
Domainmodel
Usermodel
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
Application reverse engineering
Use
r ta
sk a
nd d
omai
nre
vers
e en
gine
erin
g
Log filesPredictive modelsDesign heuristics
Usability guidelinesFig 16 UI Design recovery
This ongoing work is just at the beginning as it will requirea significant amount of work to create a comprehensive en-vironment with tool-support for model-based reengineeringof web sites The described approach inevitably suffersfrom many restrictions and lack of support such as Non-standard HTML elements (eg embedded objects
browser-specific tags) or languages (eg JavaScriptfunctions ActiveX controls) are ignored as their supportwould require expensive development efforts
Dynamically created web pages are not supported sincetheir HTML code is generated on the fly The callbackroutines attached to AIOs producing dynamic pages canbe replaced by direct calls to the semantic core Activemodels are in this case too expensive to manage re-engineering on the fly However we can imagine to par-tially support applications that use explicit template-based page generation (eg XSLT JSP) in this case thestatic analysis should be able to differentiate variableparts from run-time provided parts
Style sheets are unsupported in the outlined algorithm asare other relevant presentation aspects For instancegraphical images designed as tabs (eg on www ama-zoncom) or links placed on top of such graphics are notrecognized and therefore cannot be mapped onto severalLWs of a same PU
On the other hand the environment we have described mayenvision other forms of UI reverse engineering [3] as repre-sented in table 1
Redocumentation (fig 15) this flow is similar to re-engineering except that the platform model is no longerneeded as it remains constant over time
Restructuring (fig 15 with arrow (1) used only) the UIremains basically the same for semantics and dialog butthe presentation model changes The scope of the plat-form model is restricted to re-engineering
Redesigning (fig 15 with arrows (1) and (2) used) theUI remains identical regarding its semantics but both thedialog (eg other interaction styles techniques or gen-res) and the presentation (eg any redistribution) modelscan change due to platform constraints This may consistsin a new category of UI re-engineering
Design recovery (fig 16) in this process it is expectedthat task and domain models could be recovered Guess-ing these models from HTML code seems impracticalbut other information sources might be investigated suchas log files produced by user traffic comparison withpredictive models design heuristics to identify usagepatterns and automated usability analysis based onguidelines
REFERENCES[1] Barclay PJ Griffiths T Mc Kirdy J Paton NW Coo-
per R Kennedy J ldquoThe Teallach Tool Using Models forFlexible User Interface Designrdquo Proc of 3rd Int Conf onComputer-Aided Design of User Interfaces CADUIrsquo99 (Lou-vain-la-Neuve 21-23 October 1999) Kluwer AcademicsDordrecht (1999) 139ndash158 Accessible at httpimgcsmanacukteallachpublicationsCadui99CADUI99ps
[2] G Abowd A Goel DF Jerding M McCracken MM
Moore JW Murdock C Potts S Rugaber and L WillsldquoMORALEndashMission Oriented Architectural Legacy Evolu-tionrdquo Proc of Int Conf on Software Maintenance (1997)
[3] F Bodart A-M Hennebert J-M Leheureux and JVanderdonckt ldquoComputer-Aided Window Identification inTRIDENTrdquo Proc of the 5th IFIP TC13 Conf on Human-Computer Interaction Interactrsquo95 (Lillehammer 25-29 June1995) Chapman amp Hall London 1995 pp 331-336 Acces-sible at httpwwwinfofundpacbecgi-publipub-spec-paperRP-95-021
[4] EJ Chikofsky and JH Cross ldquoReverse Engineering andDesign Recovery A Taxonomyrdquo IEEE Software Vol 1 No7 January 1990 pp 13-17
[5] J-M De Baud and S Rugaber ldquoA Software Re-engineeringMethod Using Domain Modelsrdquo Proc of Int Conf on Soft-ware Maintenance (October 1995) pp 204-213
[6] J Eisenstein and A Puerta ldquoAdaptation in Automated User-Interface Designrdquo Proc of ACM Int Conf on IntelligentUser Interfaces IUIrsquo2000 (New Orleans 9-12 January 2000)ACM Press New York 2000 pp 74-81
[8] L Kong E Strouila B Matichuk ldquoLegacy Interface Migra-tion A Task-Centered Approachrdquo Proc of 8th Int Conf onHuman-Computer Interaction HCI Internationalrsquo99 (Mu-nich 22-27 August 1999) H-J Bullinger and J Ziegler(eds) Lawrence Erlbaum Associates MahwahLondon1999 pp 1167-1171 Accessible at httpwwwcsualbertaca~strouliaPapershci99ps
[9] F Lonczewski and S Schreiber ldquoThe FUSE-System an In-tegrated User Interface Design Environmentrdquo Proc of 2nd
Int Workshop on Coputer-Aided Design of User InterfacesCADUIrsquo96 (Namur 5-7 June 1996) Presses Universitairesde Namur Namur 1996 pp 37-56 Accessible at ftphpeick7informatiktu-muenchendepubpaperssisfuse_cadui96psgz
[10] Ch Maumlrtin ldquoA UIMS for Knowledge Based Interface Tem-plate Generation and Interactionrdquo Proc of Interactrsquo90 El-sevier Science Pub Amsterdam 1990 pp 651-657
[11] E Merlo JF Girard K Kontogiannis P Panangaden andR De Mori ldquoReverse Engineering of User Interfacesrdquo Procof 1st Working Conference on Reverse EngineeringWCRErsquo93 (Baltimore 21-23 May 1993) RC Waters EJChikofsky (eds) IEEE Computer Society Press LosAlamitos 1993 pp 171-179
[12] E Merlo P-Y Gagneacute and A Thiboutocirct ldquoInference ofgraphical AUIDL specifications for the reverse engineeringof user interfacesrdquo Proc of Int Conf on Software Mainte-nance (19-23 September 1994) IEEE Computer SocietyPress Los Alamitos 1994 pp 80-88
[13] E Merlo P-Y Gagneacute J-F Girard K Kontogiannis LHendren P Panagaden and R De Mori ldquoReengineeringUser Interfacesrdquo IEEE Software Vol 12 No 1 January1995 pp 64-73
[14] MM Moore ldquoRule-Based Detection for Reverse Engineer-ing User Interfacesrdquo Proc of 3rd Working Conf on ReverseEngineering WCRErsquo96 (Monterey 8-10 November 1996) L
Wills I Baxter E Chikofsky (eds) IEEE Computer SocietyPress Los Alamitos 1996 pp 42-48 Accessible athttpwwwccgatechedufacMelodyMoorepapersWCRE96ps
[15] MM Moore ldquoRepresentation Issues for Reengineering In-teractive Systemsrdquo ACM Computing Surveys Vol 28 No 4December 1996 Article 199 Accessible at httpwwwacmorgpubsarticlesjournalssurveys1996-28-4esa199-moorea199-moorehtml
[16] MM Moore and S Rugaber ldquoUsing Knowledge Repre-sentation to Understand Interactive Systemsrdquo Proc of theFifth International Workshop on Program ComprehensionIWPC97 (Dearborn 28-30 May 1997) IEEE Computer So-ciety Press Los Alamitos 1997 Accessible at httpwwwccgatechedufacMelodyMoorepapersWPC97ps
[17] MM Moore and S Rugaber ldquoDomain Analysis for Trans-formational Reuserdquo Proc of 4th Working Conf on ReverseEngineering WCRErsquo97 (6-8 October 1997) IEEE ComputerSociety Press Los Alamitos 1997
[18] ldquoOpen Interfacetraderdquo Neuron Data 156 University AvenuePalo Alto CA 94301 1992
[19] R Pausch M Conway and R DeLine ldquoLessons Learnedfrom SUIT the Simple User Interface Toolkitrdquo ACM Transon Office Information Systems Vol 10 No 4 October1992 pp 320-344 Accessible at httpwwwcsvirginiaedu~uigroupdocspublicationsSuitlessonspaper ps
[20] AR Puerta ldquoThe MECANO Project Comprehensive andIntegrated Support for Model-Based Interface DevelopmentrdquoProc of the 2nd Int W on Computer-Aided Design of UserInterfaces CADUIrsquo96 (Namur 5-7 June 1996) Presses Uni-versitaires de Namur Namur 1996 pp 19-36
[21] REK Stirewalt and S Rugaber ldquoAutomating UI Genera-tion by Model Compositionrdquo Journal of Automated Soft-ware Engineering Vol 7 No 2 1998 pp 101-124 Acces-sible at httpwwwccgatechedureverserepositorygenerps
[22] REK Stirewalt ldquoMDL A Language for Binding User-Interface Modelsrdquo in [26] pp 159-184
[23] P Szekely P Luo and R Neches ldquoBeyond Interface Build-ers Model-Based Interface Toolsrdquo Proc of ACM Conf onHuman Aspects in Computing Systems InterCHIrsquo93 ACMPress New York 1993 pp 383-390
[24] P Szekely ldquoRetrospective and Challenges for Model-BasedInterface Developmentrdquo Proc of 3rd Int Workshop on Com-puter-Aided Design of User Interfaces CADUIrsquo96 (Namur5-7 June 1996) Presses Universitaires de Namur Namur1996 pp xxi-xliv
[25] J Vanderdonckt and F Bodart ldquoEncapsulating Knowledgefor Intelligent Interaction Objects Selectionrdquo Proc of In-terCHIrsquo93 ACM Press New York 1993 pp 424-429
[26] J Vanderdonckt and P Berquin ldquoTowards a Very LargeModel-based Approach for User Interface DevelopmentrdquoProc of 1st Int Workshop on User Interfaces to Data Inten-sive Systems UIDISrsquo99 IEEE Computer Society Press LosAlamitos 1999 pp 76-85
[27] J Vanderdonckt and A Puerta (eds) Proc of the 3rd IntConf on Computer-Aided Design of User Interfaces CA-DUIrsquo99 Kluwer Academics Publishers Dordrecht 1999
[28] The XIML Consortium httpwwwximlorg 2002
FORWARD ENGINEERING OF WEB PAGESFor many years the research and development communityof model-based approaches has shown interest for forwardengineering [2326] with many models (fig 1) A domain model [4] defines the objects that a user can
view access and manipulate through a UI In nature it isvery similar to a data model but it is also intended to ex-plicitly represent the attributes of objects and the rela-tionships among the various domain objects (eg an en-tity-relationship model or an object-oriented model)
A presentation model [292425] is a representation ofthe visual haptic and auditory elements provided by a UIto its users For example a presentation element might bea window that contains additional elements such as wid-gets that appear in that window This model also includesstatic presentation attributes such as font styles and ori-entation of button groups
A dialog model [819] defines the way in which the pres-entation model interacts with the user It represents theactions that a user may initiate via the presentation ele-ments and the reactions that the application communi-cates via those same elements
An application model [22] captures some abstraction ofse-man-tic functions of the semantic core componentcalled by the UI and maintains connections with it Itspecifies services applications provided to the UI
For almost a decade models have been exploited to auto-matically generate a UI in two ways (fig 1) [26]1 At design time by exploiting passive models models are
parsed to generate the UI code for a given computingplatform The generation process ranges from completelyautomatic (the designer does not see anything during thisprocess) to full mixed-initiative (where the systemprompts the designer for design options to decide for
control parameters) The generated code is integratedwith the code of the semantic core component and com-piled to obtain the intended application
2 At run time by exploiting active models the specifica-tions contained in the models are scanned and interpretedat run-time to obtain a running UI connected to the codeof the semantic core component For instance FUSE [8]uses an attributed grammar to abstract a UI to execute thespecification contained in the models Apart from beingexecutable at run-time this approach is more suitable forgenerating UIs depending on parameters that only be-come known at run-time The alternative is to attempt topredict all alternatives at design-time and then generate aUI that supports all these circumstances But the resultingcode would be inefficient more complex to produce notso compact In some cases it may be impossible to haveall UI alternatives contained into one single piece ofcode
Whether at design-time or at run-time one or many modelsare typically exploited to generate one UI for one applica-tion To generate many UIs for the same application and inorder to support user-centered design the traditional model-based approach has been extended to a task-based approach(fig 2) where two new models are introduced A task model is a description of the task to be accom-
plished by the user of an application through the UI In-dividual elements in this model represent specific actionsthat the user may undertake Information regarding sub-task ordering (eg sequence unordered) as well as con-ditions on task execution is also included
A user model represents the different types of users of anapplication It defines attributes roles of users and mayinclude information describing their preferences
Scenario Interactiveapplication
Taskmodel
Domainmodel
Usermodel
Semanticcore code
User inter-face code
UI I
nteg
ratio
n
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
User-task elicitation User interface modeling and generation
Application modeling and development
Interactiveapplication
Domainmodel
Semanticcore code
User inter-face code
UI I
nteg
ratio
n
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
Domain modeling User interface modeling and generation
Application modeling and development
Fig 1 Traditional model-based approach
Fig 2 Task-based approach
REVERSE ENGINEERING OF WEB PAGESSeveral environments today support forward engineering ofWeb pages as presented in Section 2 For example AllegroServe (httpallegroservesourceforgenet) is a web serverthat can dynamically generate Web pages and web-enableexisting applications with a browser front-end However asmost existing Web sites were not built according to such anapproach adopting a reverse engineering of these webpages is a required preliminary step to further benefit fromany future forward engineering The goal of reverse engi-neering here is to recover both presentation and dialogmodels of a set of web pages (fig 3) The UI code shouldbe extracted from the global code and separated from thecode of the semantic core component From this UI codepresentation and dialog models need to be reverse engi-neered For this purpose relevant abstractions need to bedefined for each of these two models
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering
Fig 3 Reverse engineering
Abstractions for the Presentation ModelUI presentation of web pages can be abstracted using fourconcepts the hierarchy of which is depicted in fig 41 Concrete Interaction Object (CIO) this is a real object
belonging to the UI world that any user can see (eg textimage animation) or manipulate such as a push button alist box a check box A CIO is said to be simple if it can-not be decomposed into smaller CIOs A CIO is said tobe composite if it can be decomposed into smaller unitsTwo categories are distinguished presentation CIOwhich is any static CIO allowing no user interaction andcontrol CIO which support some interaction or UI con-trol by the user
2 Abstract Interaction Object (AIO) this consists of an ab-straction of all CIOs from both presentation and behav-ioral viewpoints that is independent of any given com-puting platform AIOs have been used success-fully forboth forward [892024] and reverse engineering[101314] Each AIO is here identified by a unique ge-neric name (eg check box) general and particular ab-stract attributes (eg height width color states) abstractevents (eg value selection mouse click) and abstractprimitive (eg Pr-EditBoxContent) By definition anAIO does not have any graphical appearance but eachAIO is connected to 0 1 or many CIOs having differentnames and presentations in various computing platforms32 AIOs were described in a ldquois-ardquo hierarchy of classesinto a knowledge base [25]
3 Logical Window (LW) this root window can be consid-ered either as a logical container for AIOs or as a physi-cal window a dialog box or a panel Every window is it-self a composite AIO as it is composed of other simple orcomposite AIOs All LWs are supposed to be physicallyconstrained by the users screen The three abstractionsthat have been considered so far are quite related to ex-isting presentation objects However none of the presen-tation abstractions described thus far are closely relatedto task aspects The following abstraction is introducedfor this purpose
4 Presentation Unit (PU) a PU is assumed to be the com-plete presentation environment required for carrying out aparticular interactive task Each PU can be decomposedinto one or many LWs which may or may not be all dis-played on the screen simultaneously Each PU is com-posed of at least one window called the basic windowfrom which it is possible to navigate to the other win-dows For instance a tabbed dialog box is here mappedonto a PU which is itself de-composed into LWs corre-sponding to the dialog box appearances depending on theactive tab conversely a web form can be mapped onto acomposite AIO in a particular LW of a given PU
Presentation Unit
Logical Window
Composite AIO
Simple AIO
Presentation AIO Control AIO
1-n 1-n
0-n
0-n
0-n
0-n
0-n0-n
is-a
is composed of
1-n 1-n
1-n 1-n
Fig 4 Hierarchy of presentation concepts
Abstractions for the Dialog ModelWeb sites are typically developed within a graphical ortextual editor according to a page amp link approach thepresentation of a family of web pages is first designed andthe links are added as this design proceeds The abstractionlevel of these design concepts can be raised to more en-compassing concepts of a transition and navigational struc-ture A transition is defined as any control capability ena-bling users to move from a source LW to a target LW Itconsequently consists in a control AIO (eg an anchor anicon a push button) allowing a directional control (egunidirectional or bi-directional) between LWs with opera-tions on these LWs maximization titling minimizationdisplay with tiling technique display with normal overlap-ping technique display with user-defined overlapping tech-nique display with system-defined overlapping techniqueor closing A navigational structure applies any graph pat-tern between LWs of a given PU of a given type
Sequential navigation This navigation structure enables auser to move between LWs according to a pre-defined se-quence In this structure unidirectional transition allows auser to move from one LW to another (fig 5) while bi-directional transition allows a user to switch between a setof LWs these LWs being closed restored or redisplayed
Fig 5 Unidirectional and bi-directional sequences
Indexed navigation This structure is aimed at providing aregular index mechanism in which multiple target LWs aresuggested from one single source LW Having unidirec-tional transition prevents the user to come back to thesource LW while bi-directional transition will (fig 6)
PAGE
PAGE
Fig 6 Unidirectional and bi-directional indexesGuided navigation Guided navigation is similar to aguided tour a source LW suggests the user a first LW toswitch to this first LW is connected to a second one anddo forth to the last LW suggesting the user to come back tothe source LW In some sense this navigation is similar to asequential navigation where the dialog is closed Unidirec-tional transition only allows the user to move to the con-nected LW one way whereas bi-directional transition al-lows a round trip (fig 7)
PAGE
PAGE
Fig 7 Unidirectional and bi-directional guided toursMixed navigation This navigation type groups two previ-ously defined abstractions into a new one a guided naviga-tion is augmented by an index to combine navigation flexi-bility provided by an index and guidance provided by aguided tour This type therefore allows a user to be guidedin a guided tour and leave the tour or reinitiate it at everystep (fig 8)
PAGE
PAGE
Fig 8 Unidirectional and bi-directional mixed tours
Global navigation This navigation type is considered tobe the most complete as it allows the user to move from anyLW to any other one Bi-directional transitions are thereforeconsidered not useful as they reproduce the same transition(fig 9)
Fig 9 Global navigationUsing transition and navigation structures as basic conceptsnot only raise the abstraction level with respect to the tradi-tional page amp link model but also enables designers to ma-nipulate entire navigation structures and to combine them ina more logical way Rather than modifying transition con-trols on each LW a collection of behaviorally-equivalenttransition controls is established and maintained across aseries of related LWs Navigation structures can be definedin isolation or directly combined by referring to a particularLW inside their definitionIn fig 10 for instance the designer can first decide to havea sequential navigation A across a family of LWs named A1through A4 Any LW of this navigational structure can thenbe a good candidate to form the source LW for any othertype of navigational structure For example the logicalwindow A3 can be designed as the source LW for an unidi-rectional indexed navigation Any combination of the basicnavigational structures can be imagined However someobserved structures do not exactly match these basic orcombined structures In this case the closest combination isselected instead thus introducing some modification in theoriginal navigation
B1 B2 B3 B4
B
A1 A2 A3 A4A
Fig 10 Combining navigation structures
REVERSE ENGINEERING ALGORITHMHaving defined abstractions relevant to presentation anddialog models we can exploit them to reverse engineerthese abstractions for a series of web pages This processbasically consists in (i) downloading the HTML code ofWeb pages of interest (ii) extracting from their code the UIpart and separating it from the part belonging to the seman
tic core component and (iii) submitting it to the staticanalysis algorithm for reverse engineering depicted in fig11 Thanks to the HTML code accessibility remote reverseengineering of Web pages is permitted in addition the de-sign of automated tools for reverse engineering should bemuch easier for web pages than for compiled program codein C or Pascal The phases of this algorithm are now de-tailed in the subsequent subsections as they are consideredin VAQUITA (httpwwwisysuclacbebchiresearchvaquitahtm) a tool that enables designers to reverse engineer aweb page into one or several presentation models Cur-rently only the presentation model is reverse engineered
Inclusionexclusion of presentation elementsInitialization of the Web pages pool
Web pages pool empty
Selection of an individual Web page
Identification of Concrete Interaction Objects
Transformation of Concrete Interaction Objectsinto Abstract Interaction Objects
Selection of Logical Window
Hierarchy building for LWs and PUs
Update vectors of links and Web pages pool
Identification of links
Abstraction of links into Windows Transitions
Identification of Navigation Structures
Buildingofpresentationmodel
Buildingofdialogmodel
yes
no
Fig 11 Reverse engineering algorithmInitialization To initiate the process the designer specifieswhether she wants to reverse engineer a whole web site (byproviding the home pagersquos URL) any portion of a site (byproviding a starting URL and the depth level up to whichpages will be considered) or a series of web pages (by en-tering their URLs locally andor from the Web) A first poolof candidate pagesrsquo URLs is formedTo allow more flexibility in the reverse engineering proc-ess the designer can include or exclude not only anyHTML tag or element but also any attribute of each tag Toavoid reparametrization this setting canbe saved in a con-figuration file for future usage
Presentation modeling An individual web page is firstselected in the pagesrsquo pool and then submitted to identifica-tion of CIOs The basic idea of the algorithm is to identifyindividual HTML elements to map them onto relevantAIO while building a hierarchy of AIOs The HTML codeof the web page is parsed to identify these HTML elementsThe mapping is based on a set of heuristics that depend onthe HTML element or tag analyzed Examples of such heu-ristics include Each web page is mapped onto a LW in the selection of
LW phase The ltTITLEgt if any is mapped onto the labelslot of this AIO Any HTML element can be mapped toone or many AIOs with one or many properties included
Heading levels (eg H1 H2 H3) are mapped onto re-spective composite presentation AIOs and to somestructure in the progressively forming hierarchy The textprovided within the ltHgtltHgt scope is mapped onto theLabel abstract attribute of the AIO
Each ltFORMgt tag is mapped onto a composite controlAIO Multiple or embedded forms generate multiplecomposite AIOs on corresponding hierarchy levelsltTRgt ltTDgt tags denoting tables are treated similarly
Web pagersquos frames are mapped onto separate LWs asthey are themselves partial views on other pages
Static banners icons graphics illustrations and GIF filesare listed in the ldquoimagesrdquo category of a ldquoGraphicsrdquo sim-ple presentation AIO animated GIFs or video are listedin its ldquoanimationrdquo category while JPEG files are believedto relate to the ldquophotographrdquo category
Portions of text are equally treated like labels with theirstatic properties being fed into the abstract attributes ofthe AIO eg the currently active BGCOLOR is mappedonto BackGroundColor and TEXT is mapped onto Fore-GroundColor When text is subject to an anchor an addi-tional control AIO is created
HTML proprietary CIOs are mapped onto their corre-sponding simple control AIOs Examples include theltSELECTgt and ltTEXTAREAgt tags and the TYPE at-tribute of the ltINPUTgt tag The NAME tag is mappedonto an identification label for the related AIO The con-tent of the VALUE tag is mapped onto the abstract attrib-ute DefaultValue and so forth
Dialog modeling To reverse engineer a dialog model andfinding out instances of transitions and navigational struc-tures a vector of links is created for each page according tothe following format
V(URL)=list of intra-page links list of extra-page linkswhere each link has the form
L = (AIO_name target_URL link_type occurrence)where AIO_name = identifier in the presentation model
of the AIO holding the link source consideredtarget_URL= URL of the page linkedlink_type = (intra-page extra-page request)occurrence = amount of same links in the page
Once any individual page has been processed regarding toits own presentation model and regarding to its own vectorof links this vector is parsed to update the pool of candi-date web pages to analyze In the identification of links thisanalysis exploits positive and negative filters defined in theinitialization For example all external links or all extra-page links up to a certain depth level are omitted The ana-lyzed web page is then withdrawn from the pagesrsquo poolThe pool is examined to find a candidate recursively untilno candidate page still exists After exhausting the wholepagesrsquo pool the remainder of both dialog and presentationmodels is reverse engineered First vectors of links thathave been built for the collection of web pages in consid-eration are examined to identify LW transitions from eachpair of LWs Transformation rules include Any link with multiple occurrences within the same page
is reduced to a single link to avoid repetition All links with multiple sources within the same page
(eg a textual link a push button an icon) to the sametarget page are gathered Depending on parameters allinstances of such links can be kept or reduced to any par-ticular type For instance links can be restricted to onlyone occurrence of anchor and one occurrence of icon Inthis case the presentation model can combine both AIOsinto a single one of the ldquoiconrdquo type whose label is theanchor text Again positive and negative filters canautomatically expand or reduce the scope of AIOs inconcern for example considering only push buttons andforgetting all other types of AIOs for links
Any one-way link is transformed into a unidirectionaltransition Depending on the link type the operations onthe source and target LWs are specified1 When a link replaces one page by another the transi-
tion will contain a ldquocloserdquo operation on the source LWand a ldquomaximizerdquo operation on the target LW
2 When a link just pops up another page the transitionwill contain only a ldquodisplay with normal overlappingrdquooperation on the target LW
3 When a link pops up another page in a constrainedwindow the transition will contain a ldquodisplay withsystem-defined overlappingrdquo on the target LW
Any pair of reciprocal links between two pages is trans-formed into a single bi-directional transition between thetwo LWs
Any intra-page link is transformed into a control AIObranching to the related composite AIO belonging to thesame LW Normally intra-page links are intended tofasten visualization and avoid scrolling rather than pro-viding LW transitions This information is still re-cordedas in the future these AIOs may be replaced by some-thing else for example another LW In this design optionparts of a web page that are accessed by intra-page linksmay be replaced by separate LWs
Transitions between logical windows abstracting these linksare subsequently stored in a matrix Each matrix columndenotes a source LW while each matrix line denotes a tar-get LW Thus any cell denotes a window transition be-tween a pair of windowsThe resulting matrix is then passed to a pattern-matchingalgorithm that is in charge of identifying navigationalstructures among the LWs This algorithm begins byguessing the simplest navigational structures (such as se-quential or indexed navigation) and continues with morerich structures (such as guided or mixed navigations) Asthe window transition matrix can be rather sparse or nonregular finding the exact materialization of these naviga-tional structures can be infrequentThe next step is to generalize the existing structure to theclosest abstract structure type Of course each existingstructure can be generalized into a global navigation struc-ture But this network of LWs probably generates too manywindow transitions In order to moderate this generalizationprocess a threshold can be specified This parameter canfor example regulate how many transitions can be added inorder to match to the closest strict navigational structure Asthis algorithm may be incomplete the designer is allowed tocheck the resulting navigational structures The designermay accept reject or modify the structure produced the al-gorithm and she also may declare undiscovered structuresThis concludes the building of the dialog model
RE-ENGINEERING OF WEB PAGESNow that presentation and dialog models are available fromreverse engineering of web pages these two models cantheoretically benefit from forward engineering any model-based approach should be able to generate a new UI fromthese two models to be further re-integrated with the se-mantic core component Before reengineering web pagesinto a new UI according to the process depicted in fig 12 itis important to specify the context in which this new UI will
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and reengineering
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Fig 12 UI Reengineering
take place As indicated in the introduction technologicalpush and user pull are both increasing the demand for ac-cessing web sites in various contexts of use These varia-tions may depend on1 User several users will access the site each user be-
longing to a specific stereotype of the user populationEach stereotype ha its own set of parameters such aspreferences customized features interaction historycognitive profile typing vs selection skills motor- sight- or auditive-impairment and specific needs
2 Computing platform many different computing plat-forms can be used to access a site and each platformbrings its own set of constraints such as screen resolu-tion number of colors color palettes planes UI lan-guage (programming language script language ortab)based language are very different) network and in-teraction devices These two last constraints possessthemselves further constraints such as network band-width interaction devices capabilities and availability
3 Environment users access a site in different environ-ments Ambient conditions add a new set of constraintsincluding physical parameters (eg noise light workingspace room heat) and psychological social parameters(eg stressful time slot productivity task criticality un-stable environment collaborative or cooperative condi-tions home vs work)
Reengineering web pages to support all these variations ofthe context of use would be an ambitious task As we as-sumed in the introduction to mainly address cross-plat-formsupport this paper is focused on adding a computing plat-form model only We feel that techniques that are found tobe useful for addressing platform constraints will also beable to handle constraints that come from the user and theenvironment The rapid proliferation of various platformsfor web accessibility makes this a particularly timely con-cern Users increasingly want to compute while movingbetween locations thus moving from one platform (eg aWindows PC) to another (eg a Windows CE-compatibledevice) Other users do not want to move from one place toan-other but want to have multiple access from a singleplace (eg connecting a PDA to a PC)Many parameters vary across platforms VAQUITA onlysupport screen resolution variation for one UI language(ie HTML) as summarized in table 1 The concepts re-main the same without loss of generality
Context with no parameter varyingIn this context a new presentation model can be inferredfrom the existing models by considering at least 5 designs1 Redistribution of LWs within a single PU2 Redistribution of Composite AIOs within a single LW3 Redistribution of Simple AIOs within a single LW4 Redistribution of all AIOs within a single LW5 Redistribution of all AIOs within a single PU
Individual AIOs should remain unchanged but any redistri-bution of these elements can be imagined provided that thescreen constraints are satisfied To verify these constraintsthe presentation model needs to be enriched by two AIOabstract attributes typical height and length These attrib-utes are assigned to an average value in pixels for dimen-sion-fixed widgets (eg a check box a radio button)
These attributes are assigned to a value that equals a multi-plying coefficient in pixels times the character length fordimension-varying widgets (eg an edit box a list box)Other dimensions are inferred or estimated from the origi-nal CIOs themselves (eg an image thanks to theHEIGHT=x WIDTH=y tags) Analyzing all possible abovereconfigurations is beyond the scope of this paper We referto [225] for analyzing redistribution of LWs within a samePU (ie minimal maximal inputoutput functional userdefined grouped ungrouped)
Context with varying UI languageIn this context only the underlying UI language changesThis often occurs when multiple computing platforms run-ning different operating systems and presentation managersare considered Some languages are Internet-compatible(eg HTML Java WML) some are not (eg Visual Ba-sic C++) Beyond redistributions described in Section 41this context poses a new challenge all these languages donot share similar capabilities in terms of CIOs of the sourcecomputing platformFor instance an AIO may be mapped onto zero CIOs (thecorresponding CIO does not exist in the target language)one (the most frequent and easy case where a one-to-onerelationship exists between an AIO and its correspondingCIO) or many CIOs (several CIOs are required to repro-duce the same behavior of the intended AIO) To allowthese mappings a platform-specific transformation tableallows any AIO instance to be assigned to correspondingCIOsWhen no corresponding CIO exists an alternative CIO isproposed (ultimately the edit box is suggested) but can betailored by the designer Nevertheless most access devicesor specific computing platforms coming with their ownproprietary language also impose some screen resolutionchange For instance Wireless Markup Language (WML)can only be used on cellular phones Voice XML only onwith a screen reader A third context is then considered
Context with varying screen resolutionIt is a property of HTML that many HTML-compliantbrowsers on many types of screen can display it Althoughthis display is technically feasible (ie the browser adaptsthe presentation with respect to the platform) it may be un-usable to the end-user for one or another reason eg lossof information structure overcrowded screen too manyscroll bars image reduction) Therefore it makes sense toconsider how to re-engineer the UI for different screenresolutions This need is exacerbated by multiple HTMLbrowsers On different platforms equipped with different monitors
(ranging from workstation screen to small laptop) On different access devices with built-in browser (rang-
ing from the Internet ScreenPhone with 640x480 reso-lution to HTML-compatible PDA)
If the screen resolution is increasing the centered table withthree columns can be used to keep a presentation centeredon the screen with borders (fig 13) If the resolution is de-creasing the above redistributions are no longer applicableThree strategies have been explored so far
Fig 13 Presentation centered on the screen1 AIO replacement each AIO definition is augmented
with a list of potential replacements considered as eitherbehaviorally equivalent but consuming less space (ega list box to a drop-down list box) or degraded (eg anaccumulator to a multiple-value list box)
2 AIO size reduction typical height and length of AIOsare submitted to a possible reduction depending on us-ability constraints For example images and icons canbe reduced to a certain limit specified by a thresholdeg the minimally usable size of icons
3 AIO ungrouping when individual AIOs cannot be re-duced according to the two first strategies the distribu-tion of AIOs may be reconfigured AIOs can be groupedor ungrouped and new LWs can be created to contain alimited amount of AIOs
Again the development of these strategies bears further ex-planation which we hope to conduct in a later paper
Context with varying language and resolutionThis context is most likely to occur when a completely dif-ferent computing platform of access device is considered
For example a cellular phone only supports WML WebTVonly accepts a subset of HTML V3 and screen readersonly accept VoiceXML This context can be handled usinga combination of the approaches described in previouscontexts
RELATED WORKCross-platform development is not new as several environ-ments provide support for this purpose Galaxy [6] andOpen Interface [17] render the same UI on different plat-forms with their native look amp feel while SUIT [18] em-ploys a unique UI definition that can be processed on mul-tiple platforms However not one of these systems trulyadopts a model-based approach although SUITs commondefinition holds some presentation abstractions CT-UIMS[9] pioneered the platform model by supporting some AIO[15] redistribution for OSFMotif large screen and smallMacintosh screens AUIDL [101112] is probably the firstset of abstractions for reverse and re-engineering UIs frominternal hierarchical structures type and variable declara-tions a UI can be recovered in IDL with a presentationmodel (based on OO paradigm) and a dialog model (basedon Milnerrsquos process algebra) MORPH exploits productionrules [13] to infer AIOs [14] from CIOs and thus producesa graphical UI from a textual UI [15] This transformationalapproach is similar in principle to ours Forward engineer-ing can be executed by model composition [20] binding[21] or derivation thus proving that the approach is feasi-ble MORALE [1] is an extensive set of techniques andtools for reverse engineering legacy systems rather thanweb pages CELLEST [7] reverse engineers similar sys-tems but into DHTML thus adopting active models ratherthan passive models
CONCLUSIONThis paper has introduced a model-based approach sup-porting both reverse and re-engineering of web pages Thearchitecture outlined in fig 14 assumes that all models arestored in a model repository each model being specifiedwith the eXtensible Interface Markup Language (XIML)promoted by the XIML Consortium [28] This model tex-tual specification is declarative analyzable and editable[22]
HTMLpage
VAQUITA Web pagereverse engineering
User interfaceRe-engineering
XIMLPresentation
model
HTMLpage
WMLdeck
hellip
Model editor validator
Fig 14 Model-based approach architecture
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and redocumentation
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Platformmodel(1) (2)
Fig 15 UI Redocumentation
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
User interface reverse engineering
Taskmodel
Domainmodel
Usermodel
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
Application reverse engineering
Use
r ta
sk a
nd d
omai
nre
vers
e en
gine
erin
g
Log filesPredictive modelsDesign heuristics
Usability guidelinesFig 16 UI Design recovery
This ongoing work is just at the beginning as it will requirea significant amount of work to create a comprehensive en-vironment with tool-support for model-based reengineeringof web sites The described approach inevitably suffersfrom many restrictions and lack of support such as Non-standard HTML elements (eg embedded objects
browser-specific tags) or languages (eg JavaScriptfunctions ActiveX controls) are ignored as their supportwould require expensive development efforts
Dynamically created web pages are not supported sincetheir HTML code is generated on the fly The callbackroutines attached to AIOs producing dynamic pages canbe replaced by direct calls to the semantic core Activemodels are in this case too expensive to manage re-engineering on the fly However we can imagine to par-tially support applications that use explicit template-based page generation (eg XSLT JSP) in this case thestatic analysis should be able to differentiate variableparts from run-time provided parts
Style sheets are unsupported in the outlined algorithm asare other relevant presentation aspects For instancegraphical images designed as tabs (eg on www ama-zoncom) or links placed on top of such graphics are notrecognized and therefore cannot be mapped onto severalLWs of a same PU
On the other hand the environment we have described mayenvision other forms of UI reverse engineering [3] as repre-sented in table 1
Redocumentation (fig 15) this flow is similar to re-engineering except that the platform model is no longerneeded as it remains constant over time
Restructuring (fig 15 with arrow (1) used only) the UIremains basically the same for semantics and dialog butthe presentation model changes The scope of the plat-form model is restricted to re-engineering
Redesigning (fig 15 with arrows (1) and (2) used) theUI remains identical regarding its semantics but both thedialog (eg other interaction styles techniques or gen-res) and the presentation (eg any redistribution) modelscan change due to platform constraints This may consistsin a new category of UI re-engineering
Design recovery (fig 16) in this process it is expectedthat task and domain models could be recovered Guess-ing these models from HTML code seems impracticalbut other information sources might be investigated suchas log files produced by user traffic comparison withpredictive models design heuristics to identify usagepatterns and automated usability analysis based onguidelines
REFERENCES[1] Barclay PJ Griffiths T Mc Kirdy J Paton NW Coo-
per R Kennedy J ldquoThe Teallach Tool Using Models forFlexible User Interface Designrdquo Proc of 3rd Int Conf onComputer-Aided Design of User Interfaces CADUIrsquo99 (Lou-vain-la-Neuve 21-23 October 1999) Kluwer AcademicsDordrecht (1999) 139ndash158 Accessible at httpimgcsmanacukteallachpublicationsCadui99CADUI99ps
[2] G Abowd A Goel DF Jerding M McCracken MM
Moore JW Murdock C Potts S Rugaber and L WillsldquoMORALEndashMission Oriented Architectural Legacy Evolu-tionrdquo Proc of Int Conf on Software Maintenance (1997)
[3] F Bodart A-M Hennebert J-M Leheureux and JVanderdonckt ldquoComputer-Aided Window Identification inTRIDENTrdquo Proc of the 5th IFIP TC13 Conf on Human-Computer Interaction Interactrsquo95 (Lillehammer 25-29 June1995) Chapman amp Hall London 1995 pp 331-336 Acces-sible at httpwwwinfofundpacbecgi-publipub-spec-paperRP-95-021
[4] EJ Chikofsky and JH Cross ldquoReverse Engineering andDesign Recovery A Taxonomyrdquo IEEE Software Vol 1 No7 January 1990 pp 13-17
[5] J-M De Baud and S Rugaber ldquoA Software Re-engineeringMethod Using Domain Modelsrdquo Proc of Int Conf on Soft-ware Maintenance (October 1995) pp 204-213
[6] J Eisenstein and A Puerta ldquoAdaptation in Automated User-Interface Designrdquo Proc of ACM Int Conf on IntelligentUser Interfaces IUIrsquo2000 (New Orleans 9-12 January 2000)ACM Press New York 2000 pp 74-81
[8] L Kong E Strouila B Matichuk ldquoLegacy Interface Migra-tion A Task-Centered Approachrdquo Proc of 8th Int Conf onHuman-Computer Interaction HCI Internationalrsquo99 (Mu-nich 22-27 August 1999) H-J Bullinger and J Ziegler(eds) Lawrence Erlbaum Associates MahwahLondon1999 pp 1167-1171 Accessible at httpwwwcsualbertaca~strouliaPapershci99ps
[9] F Lonczewski and S Schreiber ldquoThe FUSE-System an In-tegrated User Interface Design Environmentrdquo Proc of 2nd
Int Workshop on Coputer-Aided Design of User InterfacesCADUIrsquo96 (Namur 5-7 June 1996) Presses Universitairesde Namur Namur 1996 pp 37-56 Accessible at ftphpeick7informatiktu-muenchendepubpaperssisfuse_cadui96psgz
[10] Ch Maumlrtin ldquoA UIMS for Knowledge Based Interface Tem-plate Generation and Interactionrdquo Proc of Interactrsquo90 El-sevier Science Pub Amsterdam 1990 pp 651-657
[11] E Merlo JF Girard K Kontogiannis P Panangaden andR De Mori ldquoReverse Engineering of User Interfacesrdquo Procof 1st Working Conference on Reverse EngineeringWCRErsquo93 (Baltimore 21-23 May 1993) RC Waters EJChikofsky (eds) IEEE Computer Society Press LosAlamitos 1993 pp 171-179
[12] E Merlo P-Y Gagneacute and A Thiboutocirct ldquoInference ofgraphical AUIDL specifications for the reverse engineeringof user interfacesrdquo Proc of Int Conf on Software Mainte-nance (19-23 September 1994) IEEE Computer SocietyPress Los Alamitos 1994 pp 80-88
[13] E Merlo P-Y Gagneacute J-F Girard K Kontogiannis LHendren P Panagaden and R De Mori ldquoReengineeringUser Interfacesrdquo IEEE Software Vol 12 No 1 January1995 pp 64-73
[14] MM Moore ldquoRule-Based Detection for Reverse Engineer-ing User Interfacesrdquo Proc of 3rd Working Conf on ReverseEngineering WCRErsquo96 (Monterey 8-10 November 1996) L
Wills I Baxter E Chikofsky (eds) IEEE Computer SocietyPress Los Alamitos 1996 pp 42-48 Accessible athttpwwwccgatechedufacMelodyMoorepapersWCRE96ps
[15] MM Moore ldquoRepresentation Issues for Reengineering In-teractive Systemsrdquo ACM Computing Surveys Vol 28 No 4December 1996 Article 199 Accessible at httpwwwacmorgpubsarticlesjournalssurveys1996-28-4esa199-moorea199-moorehtml
[16] MM Moore and S Rugaber ldquoUsing Knowledge Repre-sentation to Understand Interactive Systemsrdquo Proc of theFifth International Workshop on Program ComprehensionIWPC97 (Dearborn 28-30 May 1997) IEEE Computer So-ciety Press Los Alamitos 1997 Accessible at httpwwwccgatechedufacMelodyMoorepapersWPC97ps
[17] MM Moore and S Rugaber ldquoDomain Analysis for Trans-formational Reuserdquo Proc of 4th Working Conf on ReverseEngineering WCRErsquo97 (6-8 October 1997) IEEE ComputerSociety Press Los Alamitos 1997
[18] ldquoOpen Interfacetraderdquo Neuron Data 156 University AvenuePalo Alto CA 94301 1992
[19] R Pausch M Conway and R DeLine ldquoLessons Learnedfrom SUIT the Simple User Interface Toolkitrdquo ACM Transon Office Information Systems Vol 10 No 4 October1992 pp 320-344 Accessible at httpwwwcsvirginiaedu~uigroupdocspublicationsSuitlessonspaper ps
[20] AR Puerta ldquoThe MECANO Project Comprehensive andIntegrated Support for Model-Based Interface DevelopmentrdquoProc of the 2nd Int W on Computer-Aided Design of UserInterfaces CADUIrsquo96 (Namur 5-7 June 1996) Presses Uni-versitaires de Namur Namur 1996 pp 19-36
[21] REK Stirewalt and S Rugaber ldquoAutomating UI Genera-tion by Model Compositionrdquo Journal of Automated Soft-ware Engineering Vol 7 No 2 1998 pp 101-124 Acces-sible at httpwwwccgatechedureverserepositorygenerps
[22] REK Stirewalt ldquoMDL A Language for Binding User-Interface Modelsrdquo in [26] pp 159-184
[23] P Szekely P Luo and R Neches ldquoBeyond Interface Build-ers Model-Based Interface Toolsrdquo Proc of ACM Conf onHuman Aspects in Computing Systems InterCHIrsquo93 ACMPress New York 1993 pp 383-390
[24] P Szekely ldquoRetrospective and Challenges for Model-BasedInterface Developmentrdquo Proc of 3rd Int Workshop on Com-puter-Aided Design of User Interfaces CADUIrsquo96 (Namur5-7 June 1996) Presses Universitaires de Namur Namur1996 pp xxi-xliv
[25] J Vanderdonckt and F Bodart ldquoEncapsulating Knowledgefor Intelligent Interaction Objects Selectionrdquo Proc of In-terCHIrsquo93 ACM Press New York 1993 pp 424-429
[26] J Vanderdonckt and P Berquin ldquoTowards a Very LargeModel-based Approach for User Interface DevelopmentrdquoProc of 1st Int Workshop on User Interfaces to Data Inten-sive Systems UIDISrsquo99 IEEE Computer Society Press LosAlamitos 1999 pp 76-85
[27] J Vanderdonckt and A Puerta (eds) Proc of the 3rd IntConf on Computer-Aided Design of User Interfaces CA-DUIrsquo99 Kluwer Academics Publishers Dordrecht 1999
[28] The XIML Consortium httpwwwximlorg 2002
REVERSE ENGINEERING OF WEB PAGESSeveral environments today support forward engineering ofWeb pages as presented in Section 2 For example AllegroServe (httpallegroservesourceforgenet) is a web serverthat can dynamically generate Web pages and web-enableexisting applications with a browser front-end However asmost existing Web sites were not built according to such anapproach adopting a reverse engineering of these webpages is a required preliminary step to further benefit fromany future forward engineering The goal of reverse engi-neering here is to recover both presentation and dialogmodels of a set of web pages (fig 3) The UI code shouldbe extracted from the global code and separated from thecode of the semantic core component From this UI codepresentation and dialog models need to be reverse engi-neered For this purpose relevant abstractions need to bedefined for each of these two models
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering
Fig 3 Reverse engineering
Abstractions for the Presentation ModelUI presentation of web pages can be abstracted using fourconcepts the hierarchy of which is depicted in fig 41 Concrete Interaction Object (CIO) this is a real object
belonging to the UI world that any user can see (eg textimage animation) or manipulate such as a push button alist box a check box A CIO is said to be simple if it can-not be decomposed into smaller CIOs A CIO is said tobe composite if it can be decomposed into smaller unitsTwo categories are distinguished presentation CIOwhich is any static CIO allowing no user interaction andcontrol CIO which support some interaction or UI con-trol by the user
2 Abstract Interaction Object (AIO) this consists of an ab-straction of all CIOs from both presentation and behav-ioral viewpoints that is independent of any given com-puting platform AIOs have been used success-fully forboth forward [892024] and reverse engineering[101314] Each AIO is here identified by a unique ge-neric name (eg check box) general and particular ab-stract attributes (eg height width color states) abstractevents (eg value selection mouse click) and abstractprimitive (eg Pr-EditBoxContent) By definition anAIO does not have any graphical appearance but eachAIO is connected to 0 1 or many CIOs having differentnames and presentations in various computing platforms32 AIOs were described in a ldquois-ardquo hierarchy of classesinto a knowledge base [25]
3 Logical Window (LW) this root window can be consid-ered either as a logical container for AIOs or as a physi-cal window a dialog box or a panel Every window is it-self a composite AIO as it is composed of other simple orcomposite AIOs All LWs are supposed to be physicallyconstrained by the users screen The three abstractionsthat have been considered so far are quite related to ex-isting presentation objects However none of the presen-tation abstractions described thus far are closely relatedto task aspects The following abstraction is introducedfor this purpose
4 Presentation Unit (PU) a PU is assumed to be the com-plete presentation environment required for carrying out aparticular interactive task Each PU can be decomposedinto one or many LWs which may or may not be all dis-played on the screen simultaneously Each PU is com-posed of at least one window called the basic windowfrom which it is possible to navigate to the other win-dows For instance a tabbed dialog box is here mappedonto a PU which is itself de-composed into LWs corre-sponding to the dialog box appearances depending on theactive tab conversely a web form can be mapped onto acomposite AIO in a particular LW of a given PU
Presentation Unit
Logical Window
Composite AIO
Simple AIO
Presentation AIO Control AIO
1-n 1-n
0-n
0-n
0-n
0-n
0-n0-n
is-a
is composed of
1-n 1-n
1-n 1-n
Fig 4 Hierarchy of presentation concepts
Abstractions for the Dialog ModelWeb sites are typically developed within a graphical ortextual editor according to a page amp link approach thepresentation of a family of web pages is first designed andthe links are added as this design proceeds The abstractionlevel of these design concepts can be raised to more en-compassing concepts of a transition and navigational struc-ture A transition is defined as any control capability ena-bling users to move from a source LW to a target LW Itconsequently consists in a control AIO (eg an anchor anicon a push button) allowing a directional control (egunidirectional or bi-directional) between LWs with opera-tions on these LWs maximization titling minimizationdisplay with tiling technique display with normal overlap-ping technique display with user-defined overlapping tech-nique display with system-defined overlapping techniqueor closing A navigational structure applies any graph pat-tern between LWs of a given PU of a given type
Sequential navigation This navigation structure enables auser to move between LWs according to a pre-defined se-quence In this structure unidirectional transition allows auser to move from one LW to another (fig 5) while bi-directional transition allows a user to switch between a setof LWs these LWs being closed restored or redisplayed
Fig 5 Unidirectional and bi-directional sequences
Indexed navigation This structure is aimed at providing aregular index mechanism in which multiple target LWs aresuggested from one single source LW Having unidirec-tional transition prevents the user to come back to thesource LW while bi-directional transition will (fig 6)
PAGE
PAGE
Fig 6 Unidirectional and bi-directional indexesGuided navigation Guided navigation is similar to aguided tour a source LW suggests the user a first LW toswitch to this first LW is connected to a second one anddo forth to the last LW suggesting the user to come back tothe source LW In some sense this navigation is similar to asequential navigation where the dialog is closed Unidirec-tional transition only allows the user to move to the con-nected LW one way whereas bi-directional transition al-lows a round trip (fig 7)
PAGE
PAGE
Fig 7 Unidirectional and bi-directional guided toursMixed navigation This navigation type groups two previ-ously defined abstractions into a new one a guided naviga-tion is augmented by an index to combine navigation flexi-bility provided by an index and guidance provided by aguided tour This type therefore allows a user to be guidedin a guided tour and leave the tour or reinitiate it at everystep (fig 8)
PAGE
PAGE
Fig 8 Unidirectional and bi-directional mixed tours
Global navigation This navigation type is considered tobe the most complete as it allows the user to move from anyLW to any other one Bi-directional transitions are thereforeconsidered not useful as they reproduce the same transition(fig 9)
Fig 9 Global navigationUsing transition and navigation structures as basic conceptsnot only raise the abstraction level with respect to the tradi-tional page amp link model but also enables designers to ma-nipulate entire navigation structures and to combine them ina more logical way Rather than modifying transition con-trols on each LW a collection of behaviorally-equivalenttransition controls is established and maintained across aseries of related LWs Navigation structures can be definedin isolation or directly combined by referring to a particularLW inside their definitionIn fig 10 for instance the designer can first decide to havea sequential navigation A across a family of LWs named A1through A4 Any LW of this navigational structure can thenbe a good candidate to form the source LW for any othertype of navigational structure For example the logicalwindow A3 can be designed as the source LW for an unidi-rectional indexed navigation Any combination of the basicnavigational structures can be imagined However someobserved structures do not exactly match these basic orcombined structures In this case the closest combination isselected instead thus introducing some modification in theoriginal navigation
B1 B2 B3 B4
B
A1 A2 A3 A4A
Fig 10 Combining navigation structures
REVERSE ENGINEERING ALGORITHMHaving defined abstractions relevant to presentation anddialog models we can exploit them to reverse engineerthese abstractions for a series of web pages This processbasically consists in (i) downloading the HTML code ofWeb pages of interest (ii) extracting from their code the UIpart and separating it from the part belonging to the seman
tic core component and (iii) submitting it to the staticanalysis algorithm for reverse engineering depicted in fig11 Thanks to the HTML code accessibility remote reverseengineering of Web pages is permitted in addition the de-sign of automated tools for reverse engineering should bemuch easier for web pages than for compiled program codein C or Pascal The phases of this algorithm are now de-tailed in the subsequent subsections as they are consideredin VAQUITA (httpwwwisysuclacbebchiresearchvaquitahtm) a tool that enables designers to reverse engineer aweb page into one or several presentation models Cur-rently only the presentation model is reverse engineered
Inclusionexclusion of presentation elementsInitialization of the Web pages pool
Web pages pool empty
Selection of an individual Web page
Identification of Concrete Interaction Objects
Transformation of Concrete Interaction Objectsinto Abstract Interaction Objects
Selection of Logical Window
Hierarchy building for LWs and PUs
Update vectors of links and Web pages pool
Identification of links
Abstraction of links into Windows Transitions
Identification of Navigation Structures
Buildingofpresentationmodel
Buildingofdialogmodel
yes
no
Fig 11 Reverse engineering algorithmInitialization To initiate the process the designer specifieswhether she wants to reverse engineer a whole web site (byproviding the home pagersquos URL) any portion of a site (byproviding a starting URL and the depth level up to whichpages will be considered) or a series of web pages (by en-tering their URLs locally andor from the Web) A first poolof candidate pagesrsquo URLs is formedTo allow more flexibility in the reverse engineering proc-ess the designer can include or exclude not only anyHTML tag or element but also any attribute of each tag Toavoid reparametrization this setting canbe saved in a con-figuration file for future usage
Presentation modeling An individual web page is firstselected in the pagesrsquo pool and then submitted to identifica-tion of CIOs The basic idea of the algorithm is to identifyindividual HTML elements to map them onto relevantAIO while building a hierarchy of AIOs The HTML codeof the web page is parsed to identify these HTML elementsThe mapping is based on a set of heuristics that depend onthe HTML element or tag analyzed Examples of such heu-ristics include Each web page is mapped onto a LW in the selection of
LW phase The ltTITLEgt if any is mapped onto the labelslot of this AIO Any HTML element can be mapped toone or many AIOs with one or many properties included
Heading levels (eg H1 H2 H3) are mapped onto re-spective composite presentation AIOs and to somestructure in the progressively forming hierarchy The textprovided within the ltHgtltHgt scope is mapped onto theLabel abstract attribute of the AIO
Each ltFORMgt tag is mapped onto a composite controlAIO Multiple or embedded forms generate multiplecomposite AIOs on corresponding hierarchy levelsltTRgt ltTDgt tags denoting tables are treated similarly
Web pagersquos frames are mapped onto separate LWs asthey are themselves partial views on other pages
Static banners icons graphics illustrations and GIF filesare listed in the ldquoimagesrdquo category of a ldquoGraphicsrdquo sim-ple presentation AIO animated GIFs or video are listedin its ldquoanimationrdquo category while JPEG files are believedto relate to the ldquophotographrdquo category
Portions of text are equally treated like labels with theirstatic properties being fed into the abstract attributes ofthe AIO eg the currently active BGCOLOR is mappedonto BackGroundColor and TEXT is mapped onto Fore-GroundColor When text is subject to an anchor an addi-tional control AIO is created
HTML proprietary CIOs are mapped onto their corre-sponding simple control AIOs Examples include theltSELECTgt and ltTEXTAREAgt tags and the TYPE at-tribute of the ltINPUTgt tag The NAME tag is mappedonto an identification label for the related AIO The con-tent of the VALUE tag is mapped onto the abstract attrib-ute DefaultValue and so forth
Dialog modeling To reverse engineer a dialog model andfinding out instances of transitions and navigational struc-tures a vector of links is created for each page according tothe following format
V(URL)=list of intra-page links list of extra-page linkswhere each link has the form
L = (AIO_name target_URL link_type occurrence)where AIO_name = identifier in the presentation model
of the AIO holding the link source consideredtarget_URL= URL of the page linkedlink_type = (intra-page extra-page request)occurrence = amount of same links in the page
Once any individual page has been processed regarding toits own presentation model and regarding to its own vectorof links this vector is parsed to update the pool of candi-date web pages to analyze In the identification of links thisanalysis exploits positive and negative filters defined in theinitialization For example all external links or all extra-page links up to a certain depth level are omitted The ana-lyzed web page is then withdrawn from the pagesrsquo poolThe pool is examined to find a candidate recursively untilno candidate page still exists After exhausting the wholepagesrsquo pool the remainder of both dialog and presentationmodels is reverse engineered First vectors of links thathave been built for the collection of web pages in consid-eration are examined to identify LW transitions from eachpair of LWs Transformation rules include Any link with multiple occurrences within the same page
is reduced to a single link to avoid repetition All links with multiple sources within the same page
(eg a textual link a push button an icon) to the sametarget page are gathered Depending on parameters allinstances of such links can be kept or reduced to any par-ticular type For instance links can be restricted to onlyone occurrence of anchor and one occurrence of icon Inthis case the presentation model can combine both AIOsinto a single one of the ldquoiconrdquo type whose label is theanchor text Again positive and negative filters canautomatically expand or reduce the scope of AIOs inconcern for example considering only push buttons andforgetting all other types of AIOs for links
Any one-way link is transformed into a unidirectionaltransition Depending on the link type the operations onthe source and target LWs are specified1 When a link replaces one page by another the transi-
tion will contain a ldquocloserdquo operation on the source LWand a ldquomaximizerdquo operation on the target LW
2 When a link just pops up another page the transitionwill contain only a ldquodisplay with normal overlappingrdquooperation on the target LW
3 When a link pops up another page in a constrainedwindow the transition will contain a ldquodisplay withsystem-defined overlappingrdquo on the target LW
Any pair of reciprocal links between two pages is trans-formed into a single bi-directional transition between thetwo LWs
Any intra-page link is transformed into a control AIObranching to the related composite AIO belonging to thesame LW Normally intra-page links are intended tofasten visualization and avoid scrolling rather than pro-viding LW transitions This information is still re-cordedas in the future these AIOs may be replaced by some-thing else for example another LW In this design optionparts of a web page that are accessed by intra-page linksmay be replaced by separate LWs
Transitions between logical windows abstracting these linksare subsequently stored in a matrix Each matrix columndenotes a source LW while each matrix line denotes a tar-get LW Thus any cell denotes a window transition be-tween a pair of windowsThe resulting matrix is then passed to a pattern-matchingalgorithm that is in charge of identifying navigationalstructures among the LWs This algorithm begins byguessing the simplest navigational structures (such as se-quential or indexed navigation) and continues with morerich structures (such as guided or mixed navigations) Asthe window transition matrix can be rather sparse or nonregular finding the exact materialization of these naviga-tional structures can be infrequentThe next step is to generalize the existing structure to theclosest abstract structure type Of course each existingstructure can be generalized into a global navigation struc-ture But this network of LWs probably generates too manywindow transitions In order to moderate this generalizationprocess a threshold can be specified This parameter canfor example regulate how many transitions can be added inorder to match to the closest strict navigational structure Asthis algorithm may be incomplete the designer is allowed tocheck the resulting navigational structures The designermay accept reject or modify the structure produced the al-gorithm and she also may declare undiscovered structuresThis concludes the building of the dialog model
RE-ENGINEERING OF WEB PAGESNow that presentation and dialog models are available fromreverse engineering of web pages these two models cantheoretically benefit from forward engineering any model-based approach should be able to generate a new UI fromthese two models to be further re-integrated with the se-mantic core component Before reengineering web pagesinto a new UI according to the process depicted in fig 12 itis important to specify the context in which this new UI will
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and reengineering
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Fig 12 UI Reengineering
take place As indicated in the introduction technologicalpush and user pull are both increasing the demand for ac-cessing web sites in various contexts of use These varia-tions may depend on1 User several users will access the site each user be-
longing to a specific stereotype of the user populationEach stereotype ha its own set of parameters such aspreferences customized features interaction historycognitive profile typing vs selection skills motor- sight- or auditive-impairment and specific needs
2 Computing platform many different computing plat-forms can be used to access a site and each platformbrings its own set of constraints such as screen resolu-tion number of colors color palettes planes UI lan-guage (programming language script language ortab)based language are very different) network and in-teraction devices These two last constraints possessthemselves further constraints such as network band-width interaction devices capabilities and availability
3 Environment users access a site in different environ-ments Ambient conditions add a new set of constraintsincluding physical parameters (eg noise light workingspace room heat) and psychological social parameters(eg stressful time slot productivity task criticality un-stable environment collaborative or cooperative condi-tions home vs work)
Reengineering web pages to support all these variations ofthe context of use would be an ambitious task As we as-sumed in the introduction to mainly address cross-plat-formsupport this paper is focused on adding a computing plat-form model only We feel that techniques that are found tobe useful for addressing platform constraints will also beable to handle constraints that come from the user and theenvironment The rapid proliferation of various platformsfor web accessibility makes this a particularly timely con-cern Users increasingly want to compute while movingbetween locations thus moving from one platform (eg aWindows PC) to another (eg a Windows CE-compatibledevice) Other users do not want to move from one place toan-other but want to have multiple access from a singleplace (eg connecting a PDA to a PC)Many parameters vary across platforms VAQUITA onlysupport screen resolution variation for one UI language(ie HTML) as summarized in table 1 The concepts re-main the same without loss of generality
Context with no parameter varyingIn this context a new presentation model can be inferredfrom the existing models by considering at least 5 designs1 Redistribution of LWs within a single PU2 Redistribution of Composite AIOs within a single LW3 Redistribution of Simple AIOs within a single LW4 Redistribution of all AIOs within a single LW5 Redistribution of all AIOs within a single PU
Individual AIOs should remain unchanged but any redistri-bution of these elements can be imagined provided that thescreen constraints are satisfied To verify these constraintsthe presentation model needs to be enriched by two AIOabstract attributes typical height and length These attrib-utes are assigned to an average value in pixels for dimen-sion-fixed widgets (eg a check box a radio button)
These attributes are assigned to a value that equals a multi-plying coefficient in pixels times the character length fordimension-varying widgets (eg an edit box a list box)Other dimensions are inferred or estimated from the origi-nal CIOs themselves (eg an image thanks to theHEIGHT=x WIDTH=y tags) Analyzing all possible abovereconfigurations is beyond the scope of this paper We referto [225] for analyzing redistribution of LWs within a samePU (ie minimal maximal inputoutput functional userdefined grouped ungrouped)
Context with varying UI languageIn this context only the underlying UI language changesThis often occurs when multiple computing platforms run-ning different operating systems and presentation managersare considered Some languages are Internet-compatible(eg HTML Java WML) some are not (eg Visual Ba-sic C++) Beyond redistributions described in Section 41this context poses a new challenge all these languages donot share similar capabilities in terms of CIOs of the sourcecomputing platformFor instance an AIO may be mapped onto zero CIOs (thecorresponding CIO does not exist in the target language)one (the most frequent and easy case where a one-to-onerelationship exists between an AIO and its correspondingCIO) or many CIOs (several CIOs are required to repro-duce the same behavior of the intended AIO) To allowthese mappings a platform-specific transformation tableallows any AIO instance to be assigned to correspondingCIOsWhen no corresponding CIO exists an alternative CIO isproposed (ultimately the edit box is suggested) but can betailored by the designer Nevertheless most access devicesor specific computing platforms coming with their ownproprietary language also impose some screen resolutionchange For instance Wireless Markup Language (WML)can only be used on cellular phones Voice XML only onwith a screen reader A third context is then considered
Context with varying screen resolutionIt is a property of HTML that many HTML-compliantbrowsers on many types of screen can display it Althoughthis display is technically feasible (ie the browser adaptsthe presentation with respect to the platform) it may be un-usable to the end-user for one or another reason eg lossof information structure overcrowded screen too manyscroll bars image reduction) Therefore it makes sense toconsider how to re-engineer the UI for different screenresolutions This need is exacerbated by multiple HTMLbrowsers On different platforms equipped with different monitors
(ranging from workstation screen to small laptop) On different access devices with built-in browser (rang-
ing from the Internet ScreenPhone with 640x480 reso-lution to HTML-compatible PDA)
If the screen resolution is increasing the centered table withthree columns can be used to keep a presentation centeredon the screen with borders (fig 13) If the resolution is de-creasing the above redistributions are no longer applicableThree strategies have been explored so far
Fig 13 Presentation centered on the screen1 AIO replacement each AIO definition is augmented
with a list of potential replacements considered as eitherbehaviorally equivalent but consuming less space (ega list box to a drop-down list box) or degraded (eg anaccumulator to a multiple-value list box)
2 AIO size reduction typical height and length of AIOsare submitted to a possible reduction depending on us-ability constraints For example images and icons canbe reduced to a certain limit specified by a thresholdeg the minimally usable size of icons
3 AIO ungrouping when individual AIOs cannot be re-duced according to the two first strategies the distribu-tion of AIOs may be reconfigured AIOs can be groupedor ungrouped and new LWs can be created to contain alimited amount of AIOs
Again the development of these strategies bears further ex-planation which we hope to conduct in a later paper
Context with varying language and resolutionThis context is most likely to occur when a completely dif-ferent computing platform of access device is considered
For example a cellular phone only supports WML WebTVonly accepts a subset of HTML V3 and screen readersonly accept VoiceXML This context can be handled usinga combination of the approaches described in previouscontexts
RELATED WORKCross-platform development is not new as several environ-ments provide support for this purpose Galaxy [6] andOpen Interface [17] render the same UI on different plat-forms with their native look amp feel while SUIT [18] em-ploys a unique UI definition that can be processed on mul-tiple platforms However not one of these systems trulyadopts a model-based approach although SUITs commondefinition holds some presentation abstractions CT-UIMS[9] pioneered the platform model by supporting some AIO[15] redistribution for OSFMotif large screen and smallMacintosh screens AUIDL [101112] is probably the firstset of abstractions for reverse and re-engineering UIs frominternal hierarchical structures type and variable declara-tions a UI can be recovered in IDL with a presentationmodel (based on OO paradigm) and a dialog model (basedon Milnerrsquos process algebra) MORPH exploits productionrules [13] to infer AIOs [14] from CIOs and thus producesa graphical UI from a textual UI [15] This transformationalapproach is similar in principle to ours Forward engineer-ing can be executed by model composition [20] binding[21] or derivation thus proving that the approach is feasi-ble MORALE [1] is an extensive set of techniques andtools for reverse engineering legacy systems rather thanweb pages CELLEST [7] reverse engineers similar sys-tems but into DHTML thus adopting active models ratherthan passive models
CONCLUSIONThis paper has introduced a model-based approach sup-porting both reverse and re-engineering of web pages Thearchitecture outlined in fig 14 assumes that all models arestored in a model repository each model being specifiedwith the eXtensible Interface Markup Language (XIML)promoted by the XIML Consortium [28] This model tex-tual specification is declarative analyzable and editable[22]
HTMLpage
VAQUITA Web pagereverse engineering
User interfaceRe-engineering
XIMLPresentation
model
HTMLpage
WMLdeck
hellip
Model editor validator
Fig 14 Model-based approach architecture
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and redocumentation
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Platformmodel(1) (2)
Fig 15 UI Redocumentation
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
User interface reverse engineering
Taskmodel
Domainmodel
Usermodel
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
Application reverse engineering
Use
r ta
sk a
nd d
omai
nre
vers
e en
gine
erin
g
Log filesPredictive modelsDesign heuristics
Usability guidelinesFig 16 UI Design recovery
This ongoing work is just at the beginning as it will requirea significant amount of work to create a comprehensive en-vironment with tool-support for model-based reengineeringof web sites The described approach inevitably suffersfrom many restrictions and lack of support such as Non-standard HTML elements (eg embedded objects
browser-specific tags) or languages (eg JavaScriptfunctions ActiveX controls) are ignored as their supportwould require expensive development efforts
Dynamically created web pages are not supported sincetheir HTML code is generated on the fly The callbackroutines attached to AIOs producing dynamic pages canbe replaced by direct calls to the semantic core Activemodels are in this case too expensive to manage re-engineering on the fly However we can imagine to par-tially support applications that use explicit template-based page generation (eg XSLT JSP) in this case thestatic analysis should be able to differentiate variableparts from run-time provided parts
Style sheets are unsupported in the outlined algorithm asare other relevant presentation aspects For instancegraphical images designed as tabs (eg on www ama-zoncom) or links placed on top of such graphics are notrecognized and therefore cannot be mapped onto severalLWs of a same PU
On the other hand the environment we have described mayenvision other forms of UI reverse engineering [3] as repre-sented in table 1
Redocumentation (fig 15) this flow is similar to re-engineering except that the platform model is no longerneeded as it remains constant over time
Restructuring (fig 15 with arrow (1) used only) the UIremains basically the same for semantics and dialog butthe presentation model changes The scope of the plat-form model is restricted to re-engineering
Redesigning (fig 15 with arrows (1) and (2) used) theUI remains identical regarding its semantics but both thedialog (eg other interaction styles techniques or gen-res) and the presentation (eg any redistribution) modelscan change due to platform constraints This may consistsin a new category of UI re-engineering
Design recovery (fig 16) in this process it is expectedthat task and domain models could be recovered Guess-ing these models from HTML code seems impracticalbut other information sources might be investigated suchas log files produced by user traffic comparison withpredictive models design heuristics to identify usagepatterns and automated usability analysis based onguidelines
REFERENCES[1] Barclay PJ Griffiths T Mc Kirdy J Paton NW Coo-
per R Kennedy J ldquoThe Teallach Tool Using Models forFlexible User Interface Designrdquo Proc of 3rd Int Conf onComputer-Aided Design of User Interfaces CADUIrsquo99 (Lou-vain-la-Neuve 21-23 October 1999) Kluwer AcademicsDordrecht (1999) 139ndash158 Accessible at httpimgcsmanacukteallachpublicationsCadui99CADUI99ps
[2] G Abowd A Goel DF Jerding M McCracken MM
Moore JW Murdock C Potts S Rugaber and L WillsldquoMORALEndashMission Oriented Architectural Legacy Evolu-tionrdquo Proc of Int Conf on Software Maintenance (1997)
[3] F Bodart A-M Hennebert J-M Leheureux and JVanderdonckt ldquoComputer-Aided Window Identification inTRIDENTrdquo Proc of the 5th IFIP TC13 Conf on Human-Computer Interaction Interactrsquo95 (Lillehammer 25-29 June1995) Chapman amp Hall London 1995 pp 331-336 Acces-sible at httpwwwinfofundpacbecgi-publipub-spec-paperRP-95-021
[4] EJ Chikofsky and JH Cross ldquoReverse Engineering andDesign Recovery A Taxonomyrdquo IEEE Software Vol 1 No7 January 1990 pp 13-17
[5] J-M De Baud and S Rugaber ldquoA Software Re-engineeringMethod Using Domain Modelsrdquo Proc of Int Conf on Soft-ware Maintenance (October 1995) pp 204-213
[6] J Eisenstein and A Puerta ldquoAdaptation in Automated User-Interface Designrdquo Proc of ACM Int Conf on IntelligentUser Interfaces IUIrsquo2000 (New Orleans 9-12 January 2000)ACM Press New York 2000 pp 74-81
[8] L Kong E Strouila B Matichuk ldquoLegacy Interface Migra-tion A Task-Centered Approachrdquo Proc of 8th Int Conf onHuman-Computer Interaction HCI Internationalrsquo99 (Mu-nich 22-27 August 1999) H-J Bullinger and J Ziegler(eds) Lawrence Erlbaum Associates MahwahLondon1999 pp 1167-1171 Accessible at httpwwwcsualbertaca~strouliaPapershci99ps
[9] F Lonczewski and S Schreiber ldquoThe FUSE-System an In-tegrated User Interface Design Environmentrdquo Proc of 2nd
Int Workshop on Coputer-Aided Design of User InterfacesCADUIrsquo96 (Namur 5-7 June 1996) Presses Universitairesde Namur Namur 1996 pp 37-56 Accessible at ftphpeick7informatiktu-muenchendepubpaperssisfuse_cadui96psgz
[10] Ch Maumlrtin ldquoA UIMS for Knowledge Based Interface Tem-plate Generation and Interactionrdquo Proc of Interactrsquo90 El-sevier Science Pub Amsterdam 1990 pp 651-657
[11] E Merlo JF Girard K Kontogiannis P Panangaden andR De Mori ldquoReverse Engineering of User Interfacesrdquo Procof 1st Working Conference on Reverse EngineeringWCRErsquo93 (Baltimore 21-23 May 1993) RC Waters EJChikofsky (eds) IEEE Computer Society Press LosAlamitos 1993 pp 171-179
[12] E Merlo P-Y Gagneacute and A Thiboutocirct ldquoInference ofgraphical AUIDL specifications for the reverse engineeringof user interfacesrdquo Proc of Int Conf on Software Mainte-nance (19-23 September 1994) IEEE Computer SocietyPress Los Alamitos 1994 pp 80-88
[13] E Merlo P-Y Gagneacute J-F Girard K Kontogiannis LHendren P Panagaden and R De Mori ldquoReengineeringUser Interfacesrdquo IEEE Software Vol 12 No 1 January1995 pp 64-73
[14] MM Moore ldquoRule-Based Detection for Reverse Engineer-ing User Interfacesrdquo Proc of 3rd Working Conf on ReverseEngineering WCRErsquo96 (Monterey 8-10 November 1996) L
Wills I Baxter E Chikofsky (eds) IEEE Computer SocietyPress Los Alamitos 1996 pp 42-48 Accessible athttpwwwccgatechedufacMelodyMoorepapersWCRE96ps
[15] MM Moore ldquoRepresentation Issues for Reengineering In-teractive Systemsrdquo ACM Computing Surveys Vol 28 No 4December 1996 Article 199 Accessible at httpwwwacmorgpubsarticlesjournalssurveys1996-28-4esa199-moorea199-moorehtml
[16] MM Moore and S Rugaber ldquoUsing Knowledge Repre-sentation to Understand Interactive Systemsrdquo Proc of theFifth International Workshop on Program ComprehensionIWPC97 (Dearborn 28-30 May 1997) IEEE Computer So-ciety Press Los Alamitos 1997 Accessible at httpwwwccgatechedufacMelodyMoorepapersWPC97ps
[17] MM Moore and S Rugaber ldquoDomain Analysis for Trans-formational Reuserdquo Proc of 4th Working Conf on ReverseEngineering WCRErsquo97 (6-8 October 1997) IEEE ComputerSociety Press Los Alamitos 1997
[18] ldquoOpen Interfacetraderdquo Neuron Data 156 University AvenuePalo Alto CA 94301 1992
[19] R Pausch M Conway and R DeLine ldquoLessons Learnedfrom SUIT the Simple User Interface Toolkitrdquo ACM Transon Office Information Systems Vol 10 No 4 October1992 pp 320-344 Accessible at httpwwwcsvirginiaedu~uigroupdocspublicationsSuitlessonspaper ps
[20] AR Puerta ldquoThe MECANO Project Comprehensive andIntegrated Support for Model-Based Interface DevelopmentrdquoProc of the 2nd Int W on Computer-Aided Design of UserInterfaces CADUIrsquo96 (Namur 5-7 June 1996) Presses Uni-versitaires de Namur Namur 1996 pp 19-36
[21] REK Stirewalt and S Rugaber ldquoAutomating UI Genera-tion by Model Compositionrdquo Journal of Automated Soft-ware Engineering Vol 7 No 2 1998 pp 101-124 Acces-sible at httpwwwccgatechedureverserepositorygenerps
[22] REK Stirewalt ldquoMDL A Language for Binding User-Interface Modelsrdquo in [26] pp 159-184
[23] P Szekely P Luo and R Neches ldquoBeyond Interface Build-ers Model-Based Interface Toolsrdquo Proc of ACM Conf onHuman Aspects in Computing Systems InterCHIrsquo93 ACMPress New York 1993 pp 383-390
[24] P Szekely ldquoRetrospective and Challenges for Model-BasedInterface Developmentrdquo Proc of 3rd Int Workshop on Com-puter-Aided Design of User Interfaces CADUIrsquo96 (Namur5-7 June 1996) Presses Universitaires de Namur Namur1996 pp xxi-xliv
[25] J Vanderdonckt and F Bodart ldquoEncapsulating Knowledgefor Intelligent Interaction Objects Selectionrdquo Proc of In-terCHIrsquo93 ACM Press New York 1993 pp 424-429
[26] J Vanderdonckt and P Berquin ldquoTowards a Very LargeModel-based Approach for User Interface DevelopmentrdquoProc of 1st Int Workshop on User Interfaces to Data Inten-sive Systems UIDISrsquo99 IEEE Computer Society Press LosAlamitos 1999 pp 76-85
[27] J Vanderdonckt and A Puerta (eds) Proc of the 3rd IntConf on Computer-Aided Design of User Interfaces CA-DUIrsquo99 Kluwer Academics Publishers Dordrecht 1999
[28] The XIML Consortium httpwwwximlorg 2002
Sequential navigation This navigation structure enables auser to move between LWs according to a pre-defined se-quence In this structure unidirectional transition allows auser to move from one LW to another (fig 5) while bi-directional transition allows a user to switch between a setof LWs these LWs being closed restored or redisplayed
Fig 5 Unidirectional and bi-directional sequences
Indexed navigation This structure is aimed at providing aregular index mechanism in which multiple target LWs aresuggested from one single source LW Having unidirec-tional transition prevents the user to come back to thesource LW while bi-directional transition will (fig 6)
PAGE
PAGE
Fig 6 Unidirectional and bi-directional indexesGuided navigation Guided navigation is similar to aguided tour a source LW suggests the user a first LW toswitch to this first LW is connected to a second one anddo forth to the last LW suggesting the user to come back tothe source LW In some sense this navigation is similar to asequential navigation where the dialog is closed Unidirec-tional transition only allows the user to move to the con-nected LW one way whereas bi-directional transition al-lows a round trip (fig 7)
PAGE
PAGE
Fig 7 Unidirectional and bi-directional guided toursMixed navigation This navigation type groups two previ-ously defined abstractions into a new one a guided naviga-tion is augmented by an index to combine navigation flexi-bility provided by an index and guidance provided by aguided tour This type therefore allows a user to be guidedin a guided tour and leave the tour or reinitiate it at everystep (fig 8)
PAGE
PAGE
Fig 8 Unidirectional and bi-directional mixed tours
Global navigation This navigation type is considered tobe the most complete as it allows the user to move from anyLW to any other one Bi-directional transitions are thereforeconsidered not useful as they reproduce the same transition(fig 9)
Fig 9 Global navigationUsing transition and navigation structures as basic conceptsnot only raise the abstraction level with respect to the tradi-tional page amp link model but also enables designers to ma-nipulate entire navigation structures and to combine them ina more logical way Rather than modifying transition con-trols on each LW a collection of behaviorally-equivalenttransition controls is established and maintained across aseries of related LWs Navigation structures can be definedin isolation or directly combined by referring to a particularLW inside their definitionIn fig 10 for instance the designer can first decide to havea sequential navigation A across a family of LWs named A1through A4 Any LW of this navigational structure can thenbe a good candidate to form the source LW for any othertype of navigational structure For example the logicalwindow A3 can be designed as the source LW for an unidi-rectional indexed navigation Any combination of the basicnavigational structures can be imagined However someobserved structures do not exactly match these basic orcombined structures In this case the closest combination isselected instead thus introducing some modification in theoriginal navigation
B1 B2 B3 B4
B
A1 A2 A3 A4A
Fig 10 Combining navigation structures
REVERSE ENGINEERING ALGORITHMHaving defined abstractions relevant to presentation anddialog models we can exploit them to reverse engineerthese abstractions for a series of web pages This processbasically consists in (i) downloading the HTML code ofWeb pages of interest (ii) extracting from their code the UIpart and separating it from the part belonging to the seman
tic core component and (iii) submitting it to the staticanalysis algorithm for reverse engineering depicted in fig11 Thanks to the HTML code accessibility remote reverseengineering of Web pages is permitted in addition the de-sign of automated tools for reverse engineering should bemuch easier for web pages than for compiled program codein C or Pascal The phases of this algorithm are now de-tailed in the subsequent subsections as they are consideredin VAQUITA (httpwwwisysuclacbebchiresearchvaquitahtm) a tool that enables designers to reverse engineer aweb page into one or several presentation models Cur-rently only the presentation model is reverse engineered
Inclusionexclusion of presentation elementsInitialization of the Web pages pool
Web pages pool empty
Selection of an individual Web page
Identification of Concrete Interaction Objects
Transformation of Concrete Interaction Objectsinto Abstract Interaction Objects
Selection of Logical Window
Hierarchy building for LWs and PUs
Update vectors of links and Web pages pool
Identification of links
Abstraction of links into Windows Transitions
Identification of Navigation Structures
Buildingofpresentationmodel
Buildingofdialogmodel
yes
no
Fig 11 Reverse engineering algorithmInitialization To initiate the process the designer specifieswhether she wants to reverse engineer a whole web site (byproviding the home pagersquos URL) any portion of a site (byproviding a starting URL and the depth level up to whichpages will be considered) or a series of web pages (by en-tering their URLs locally andor from the Web) A first poolof candidate pagesrsquo URLs is formedTo allow more flexibility in the reverse engineering proc-ess the designer can include or exclude not only anyHTML tag or element but also any attribute of each tag Toavoid reparametrization this setting canbe saved in a con-figuration file for future usage
Presentation modeling An individual web page is firstselected in the pagesrsquo pool and then submitted to identifica-tion of CIOs The basic idea of the algorithm is to identifyindividual HTML elements to map them onto relevantAIO while building a hierarchy of AIOs The HTML codeof the web page is parsed to identify these HTML elementsThe mapping is based on a set of heuristics that depend onthe HTML element or tag analyzed Examples of such heu-ristics include Each web page is mapped onto a LW in the selection of
LW phase The ltTITLEgt if any is mapped onto the labelslot of this AIO Any HTML element can be mapped toone or many AIOs with one or many properties included
Heading levels (eg H1 H2 H3) are mapped onto re-spective composite presentation AIOs and to somestructure in the progressively forming hierarchy The textprovided within the ltHgtltHgt scope is mapped onto theLabel abstract attribute of the AIO
Each ltFORMgt tag is mapped onto a composite controlAIO Multiple or embedded forms generate multiplecomposite AIOs on corresponding hierarchy levelsltTRgt ltTDgt tags denoting tables are treated similarly
Web pagersquos frames are mapped onto separate LWs asthey are themselves partial views on other pages
Static banners icons graphics illustrations and GIF filesare listed in the ldquoimagesrdquo category of a ldquoGraphicsrdquo sim-ple presentation AIO animated GIFs or video are listedin its ldquoanimationrdquo category while JPEG files are believedto relate to the ldquophotographrdquo category
Portions of text are equally treated like labels with theirstatic properties being fed into the abstract attributes ofthe AIO eg the currently active BGCOLOR is mappedonto BackGroundColor and TEXT is mapped onto Fore-GroundColor When text is subject to an anchor an addi-tional control AIO is created
HTML proprietary CIOs are mapped onto their corre-sponding simple control AIOs Examples include theltSELECTgt and ltTEXTAREAgt tags and the TYPE at-tribute of the ltINPUTgt tag The NAME tag is mappedonto an identification label for the related AIO The con-tent of the VALUE tag is mapped onto the abstract attrib-ute DefaultValue and so forth
Dialog modeling To reverse engineer a dialog model andfinding out instances of transitions and navigational struc-tures a vector of links is created for each page according tothe following format
V(URL)=list of intra-page links list of extra-page linkswhere each link has the form
L = (AIO_name target_URL link_type occurrence)where AIO_name = identifier in the presentation model
of the AIO holding the link source consideredtarget_URL= URL of the page linkedlink_type = (intra-page extra-page request)occurrence = amount of same links in the page
Once any individual page has been processed regarding toits own presentation model and regarding to its own vectorof links this vector is parsed to update the pool of candi-date web pages to analyze In the identification of links thisanalysis exploits positive and negative filters defined in theinitialization For example all external links or all extra-page links up to a certain depth level are omitted The ana-lyzed web page is then withdrawn from the pagesrsquo poolThe pool is examined to find a candidate recursively untilno candidate page still exists After exhausting the wholepagesrsquo pool the remainder of both dialog and presentationmodels is reverse engineered First vectors of links thathave been built for the collection of web pages in consid-eration are examined to identify LW transitions from eachpair of LWs Transformation rules include Any link with multiple occurrences within the same page
is reduced to a single link to avoid repetition All links with multiple sources within the same page
(eg a textual link a push button an icon) to the sametarget page are gathered Depending on parameters allinstances of such links can be kept or reduced to any par-ticular type For instance links can be restricted to onlyone occurrence of anchor and one occurrence of icon Inthis case the presentation model can combine both AIOsinto a single one of the ldquoiconrdquo type whose label is theanchor text Again positive and negative filters canautomatically expand or reduce the scope of AIOs inconcern for example considering only push buttons andforgetting all other types of AIOs for links
Any one-way link is transformed into a unidirectionaltransition Depending on the link type the operations onthe source and target LWs are specified1 When a link replaces one page by another the transi-
tion will contain a ldquocloserdquo operation on the source LWand a ldquomaximizerdquo operation on the target LW
2 When a link just pops up another page the transitionwill contain only a ldquodisplay with normal overlappingrdquooperation on the target LW
3 When a link pops up another page in a constrainedwindow the transition will contain a ldquodisplay withsystem-defined overlappingrdquo on the target LW
Any pair of reciprocal links between two pages is trans-formed into a single bi-directional transition between thetwo LWs
Any intra-page link is transformed into a control AIObranching to the related composite AIO belonging to thesame LW Normally intra-page links are intended tofasten visualization and avoid scrolling rather than pro-viding LW transitions This information is still re-cordedas in the future these AIOs may be replaced by some-thing else for example another LW In this design optionparts of a web page that are accessed by intra-page linksmay be replaced by separate LWs
Transitions between logical windows abstracting these linksare subsequently stored in a matrix Each matrix columndenotes a source LW while each matrix line denotes a tar-get LW Thus any cell denotes a window transition be-tween a pair of windowsThe resulting matrix is then passed to a pattern-matchingalgorithm that is in charge of identifying navigationalstructures among the LWs This algorithm begins byguessing the simplest navigational structures (such as se-quential or indexed navigation) and continues with morerich structures (such as guided or mixed navigations) Asthe window transition matrix can be rather sparse or nonregular finding the exact materialization of these naviga-tional structures can be infrequentThe next step is to generalize the existing structure to theclosest abstract structure type Of course each existingstructure can be generalized into a global navigation struc-ture But this network of LWs probably generates too manywindow transitions In order to moderate this generalizationprocess a threshold can be specified This parameter canfor example regulate how many transitions can be added inorder to match to the closest strict navigational structure Asthis algorithm may be incomplete the designer is allowed tocheck the resulting navigational structures The designermay accept reject or modify the structure produced the al-gorithm and she also may declare undiscovered structuresThis concludes the building of the dialog model
RE-ENGINEERING OF WEB PAGESNow that presentation and dialog models are available fromreverse engineering of web pages these two models cantheoretically benefit from forward engineering any model-based approach should be able to generate a new UI fromthese two models to be further re-integrated with the se-mantic core component Before reengineering web pagesinto a new UI according to the process depicted in fig 12 itis important to specify the context in which this new UI will
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and reengineering
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Fig 12 UI Reengineering
take place As indicated in the introduction technologicalpush and user pull are both increasing the demand for ac-cessing web sites in various contexts of use These varia-tions may depend on1 User several users will access the site each user be-
longing to a specific stereotype of the user populationEach stereotype ha its own set of parameters such aspreferences customized features interaction historycognitive profile typing vs selection skills motor- sight- or auditive-impairment and specific needs
2 Computing platform many different computing plat-forms can be used to access a site and each platformbrings its own set of constraints such as screen resolu-tion number of colors color palettes planes UI lan-guage (programming language script language ortab)based language are very different) network and in-teraction devices These two last constraints possessthemselves further constraints such as network band-width interaction devices capabilities and availability
3 Environment users access a site in different environ-ments Ambient conditions add a new set of constraintsincluding physical parameters (eg noise light workingspace room heat) and psychological social parameters(eg stressful time slot productivity task criticality un-stable environment collaborative or cooperative condi-tions home vs work)
Reengineering web pages to support all these variations ofthe context of use would be an ambitious task As we as-sumed in the introduction to mainly address cross-plat-formsupport this paper is focused on adding a computing plat-form model only We feel that techniques that are found tobe useful for addressing platform constraints will also beable to handle constraints that come from the user and theenvironment The rapid proliferation of various platformsfor web accessibility makes this a particularly timely con-cern Users increasingly want to compute while movingbetween locations thus moving from one platform (eg aWindows PC) to another (eg a Windows CE-compatibledevice) Other users do not want to move from one place toan-other but want to have multiple access from a singleplace (eg connecting a PDA to a PC)Many parameters vary across platforms VAQUITA onlysupport screen resolution variation for one UI language(ie HTML) as summarized in table 1 The concepts re-main the same without loss of generality
Context with no parameter varyingIn this context a new presentation model can be inferredfrom the existing models by considering at least 5 designs1 Redistribution of LWs within a single PU2 Redistribution of Composite AIOs within a single LW3 Redistribution of Simple AIOs within a single LW4 Redistribution of all AIOs within a single LW5 Redistribution of all AIOs within a single PU
Individual AIOs should remain unchanged but any redistri-bution of these elements can be imagined provided that thescreen constraints are satisfied To verify these constraintsthe presentation model needs to be enriched by two AIOabstract attributes typical height and length These attrib-utes are assigned to an average value in pixels for dimen-sion-fixed widgets (eg a check box a radio button)
These attributes are assigned to a value that equals a multi-plying coefficient in pixels times the character length fordimension-varying widgets (eg an edit box a list box)Other dimensions are inferred or estimated from the origi-nal CIOs themselves (eg an image thanks to theHEIGHT=x WIDTH=y tags) Analyzing all possible abovereconfigurations is beyond the scope of this paper We referto [225] for analyzing redistribution of LWs within a samePU (ie minimal maximal inputoutput functional userdefined grouped ungrouped)
Context with varying UI languageIn this context only the underlying UI language changesThis often occurs when multiple computing platforms run-ning different operating systems and presentation managersare considered Some languages are Internet-compatible(eg HTML Java WML) some are not (eg Visual Ba-sic C++) Beyond redistributions described in Section 41this context poses a new challenge all these languages donot share similar capabilities in terms of CIOs of the sourcecomputing platformFor instance an AIO may be mapped onto zero CIOs (thecorresponding CIO does not exist in the target language)one (the most frequent and easy case where a one-to-onerelationship exists between an AIO and its correspondingCIO) or many CIOs (several CIOs are required to repro-duce the same behavior of the intended AIO) To allowthese mappings a platform-specific transformation tableallows any AIO instance to be assigned to correspondingCIOsWhen no corresponding CIO exists an alternative CIO isproposed (ultimately the edit box is suggested) but can betailored by the designer Nevertheless most access devicesor specific computing platforms coming with their ownproprietary language also impose some screen resolutionchange For instance Wireless Markup Language (WML)can only be used on cellular phones Voice XML only onwith a screen reader A third context is then considered
Context with varying screen resolutionIt is a property of HTML that many HTML-compliantbrowsers on many types of screen can display it Althoughthis display is technically feasible (ie the browser adaptsthe presentation with respect to the platform) it may be un-usable to the end-user for one or another reason eg lossof information structure overcrowded screen too manyscroll bars image reduction) Therefore it makes sense toconsider how to re-engineer the UI for different screenresolutions This need is exacerbated by multiple HTMLbrowsers On different platforms equipped with different monitors
(ranging from workstation screen to small laptop) On different access devices with built-in browser (rang-
ing from the Internet ScreenPhone with 640x480 reso-lution to HTML-compatible PDA)
If the screen resolution is increasing the centered table withthree columns can be used to keep a presentation centeredon the screen with borders (fig 13) If the resolution is de-creasing the above redistributions are no longer applicableThree strategies have been explored so far
Fig 13 Presentation centered on the screen1 AIO replacement each AIO definition is augmented
with a list of potential replacements considered as eitherbehaviorally equivalent but consuming less space (ega list box to a drop-down list box) or degraded (eg anaccumulator to a multiple-value list box)
2 AIO size reduction typical height and length of AIOsare submitted to a possible reduction depending on us-ability constraints For example images and icons canbe reduced to a certain limit specified by a thresholdeg the minimally usable size of icons
3 AIO ungrouping when individual AIOs cannot be re-duced according to the two first strategies the distribu-tion of AIOs may be reconfigured AIOs can be groupedor ungrouped and new LWs can be created to contain alimited amount of AIOs
Again the development of these strategies bears further ex-planation which we hope to conduct in a later paper
Context with varying language and resolutionThis context is most likely to occur when a completely dif-ferent computing platform of access device is considered
For example a cellular phone only supports WML WebTVonly accepts a subset of HTML V3 and screen readersonly accept VoiceXML This context can be handled usinga combination of the approaches described in previouscontexts
RELATED WORKCross-platform development is not new as several environ-ments provide support for this purpose Galaxy [6] andOpen Interface [17] render the same UI on different plat-forms with their native look amp feel while SUIT [18] em-ploys a unique UI definition that can be processed on mul-tiple platforms However not one of these systems trulyadopts a model-based approach although SUITs commondefinition holds some presentation abstractions CT-UIMS[9] pioneered the platform model by supporting some AIO[15] redistribution for OSFMotif large screen and smallMacintosh screens AUIDL [101112] is probably the firstset of abstractions for reverse and re-engineering UIs frominternal hierarchical structures type and variable declara-tions a UI can be recovered in IDL with a presentationmodel (based on OO paradigm) and a dialog model (basedon Milnerrsquos process algebra) MORPH exploits productionrules [13] to infer AIOs [14] from CIOs and thus producesa graphical UI from a textual UI [15] This transformationalapproach is similar in principle to ours Forward engineer-ing can be executed by model composition [20] binding[21] or derivation thus proving that the approach is feasi-ble MORALE [1] is an extensive set of techniques andtools for reverse engineering legacy systems rather thanweb pages CELLEST [7] reverse engineers similar sys-tems but into DHTML thus adopting active models ratherthan passive models
CONCLUSIONThis paper has introduced a model-based approach sup-porting both reverse and re-engineering of web pages Thearchitecture outlined in fig 14 assumes that all models arestored in a model repository each model being specifiedwith the eXtensible Interface Markup Language (XIML)promoted by the XIML Consortium [28] This model tex-tual specification is declarative analyzable and editable[22]
HTMLpage
VAQUITA Web pagereverse engineering
User interfaceRe-engineering
XIMLPresentation
model
HTMLpage
WMLdeck
hellip
Model editor validator
Fig 14 Model-based approach architecture
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and redocumentation
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Platformmodel(1) (2)
Fig 15 UI Redocumentation
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
User interface reverse engineering
Taskmodel
Domainmodel
Usermodel
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
Application reverse engineering
Use
r ta
sk a
nd d
omai
nre
vers
e en
gine
erin
g
Log filesPredictive modelsDesign heuristics
Usability guidelinesFig 16 UI Design recovery
This ongoing work is just at the beginning as it will requirea significant amount of work to create a comprehensive en-vironment with tool-support for model-based reengineeringof web sites The described approach inevitably suffersfrom many restrictions and lack of support such as Non-standard HTML elements (eg embedded objects
browser-specific tags) or languages (eg JavaScriptfunctions ActiveX controls) are ignored as their supportwould require expensive development efforts
Dynamically created web pages are not supported sincetheir HTML code is generated on the fly The callbackroutines attached to AIOs producing dynamic pages canbe replaced by direct calls to the semantic core Activemodels are in this case too expensive to manage re-engineering on the fly However we can imagine to par-tially support applications that use explicit template-based page generation (eg XSLT JSP) in this case thestatic analysis should be able to differentiate variableparts from run-time provided parts
Style sheets are unsupported in the outlined algorithm asare other relevant presentation aspects For instancegraphical images designed as tabs (eg on www ama-zoncom) or links placed on top of such graphics are notrecognized and therefore cannot be mapped onto severalLWs of a same PU
On the other hand the environment we have described mayenvision other forms of UI reverse engineering [3] as repre-sented in table 1
Redocumentation (fig 15) this flow is similar to re-engineering except that the platform model is no longerneeded as it remains constant over time
Restructuring (fig 15 with arrow (1) used only) the UIremains basically the same for semantics and dialog butthe presentation model changes The scope of the plat-form model is restricted to re-engineering
Redesigning (fig 15 with arrows (1) and (2) used) theUI remains identical regarding its semantics but both thedialog (eg other interaction styles techniques or gen-res) and the presentation (eg any redistribution) modelscan change due to platform constraints This may consistsin a new category of UI re-engineering
Design recovery (fig 16) in this process it is expectedthat task and domain models could be recovered Guess-ing these models from HTML code seems impracticalbut other information sources might be investigated suchas log files produced by user traffic comparison withpredictive models design heuristics to identify usagepatterns and automated usability analysis based onguidelines
REFERENCES[1] Barclay PJ Griffiths T Mc Kirdy J Paton NW Coo-
per R Kennedy J ldquoThe Teallach Tool Using Models forFlexible User Interface Designrdquo Proc of 3rd Int Conf onComputer-Aided Design of User Interfaces CADUIrsquo99 (Lou-vain-la-Neuve 21-23 October 1999) Kluwer AcademicsDordrecht (1999) 139ndash158 Accessible at httpimgcsmanacukteallachpublicationsCadui99CADUI99ps
[2] G Abowd A Goel DF Jerding M McCracken MM
Moore JW Murdock C Potts S Rugaber and L WillsldquoMORALEndashMission Oriented Architectural Legacy Evolu-tionrdquo Proc of Int Conf on Software Maintenance (1997)
[3] F Bodart A-M Hennebert J-M Leheureux and JVanderdonckt ldquoComputer-Aided Window Identification inTRIDENTrdquo Proc of the 5th IFIP TC13 Conf on Human-Computer Interaction Interactrsquo95 (Lillehammer 25-29 June1995) Chapman amp Hall London 1995 pp 331-336 Acces-sible at httpwwwinfofundpacbecgi-publipub-spec-paperRP-95-021
[4] EJ Chikofsky and JH Cross ldquoReverse Engineering andDesign Recovery A Taxonomyrdquo IEEE Software Vol 1 No7 January 1990 pp 13-17
[5] J-M De Baud and S Rugaber ldquoA Software Re-engineeringMethod Using Domain Modelsrdquo Proc of Int Conf on Soft-ware Maintenance (October 1995) pp 204-213
[6] J Eisenstein and A Puerta ldquoAdaptation in Automated User-Interface Designrdquo Proc of ACM Int Conf on IntelligentUser Interfaces IUIrsquo2000 (New Orleans 9-12 January 2000)ACM Press New York 2000 pp 74-81
[8] L Kong E Strouila B Matichuk ldquoLegacy Interface Migra-tion A Task-Centered Approachrdquo Proc of 8th Int Conf onHuman-Computer Interaction HCI Internationalrsquo99 (Mu-nich 22-27 August 1999) H-J Bullinger and J Ziegler(eds) Lawrence Erlbaum Associates MahwahLondon1999 pp 1167-1171 Accessible at httpwwwcsualbertaca~strouliaPapershci99ps
[9] F Lonczewski and S Schreiber ldquoThe FUSE-System an In-tegrated User Interface Design Environmentrdquo Proc of 2nd
Int Workshop on Coputer-Aided Design of User InterfacesCADUIrsquo96 (Namur 5-7 June 1996) Presses Universitairesde Namur Namur 1996 pp 37-56 Accessible at ftphpeick7informatiktu-muenchendepubpaperssisfuse_cadui96psgz
[10] Ch Maumlrtin ldquoA UIMS for Knowledge Based Interface Tem-plate Generation and Interactionrdquo Proc of Interactrsquo90 El-sevier Science Pub Amsterdam 1990 pp 651-657
[11] E Merlo JF Girard K Kontogiannis P Panangaden andR De Mori ldquoReverse Engineering of User Interfacesrdquo Procof 1st Working Conference on Reverse EngineeringWCRErsquo93 (Baltimore 21-23 May 1993) RC Waters EJChikofsky (eds) IEEE Computer Society Press LosAlamitos 1993 pp 171-179
[12] E Merlo P-Y Gagneacute and A Thiboutocirct ldquoInference ofgraphical AUIDL specifications for the reverse engineeringof user interfacesrdquo Proc of Int Conf on Software Mainte-nance (19-23 September 1994) IEEE Computer SocietyPress Los Alamitos 1994 pp 80-88
[13] E Merlo P-Y Gagneacute J-F Girard K Kontogiannis LHendren P Panagaden and R De Mori ldquoReengineeringUser Interfacesrdquo IEEE Software Vol 12 No 1 January1995 pp 64-73
[14] MM Moore ldquoRule-Based Detection for Reverse Engineer-ing User Interfacesrdquo Proc of 3rd Working Conf on ReverseEngineering WCRErsquo96 (Monterey 8-10 November 1996) L
Wills I Baxter E Chikofsky (eds) IEEE Computer SocietyPress Los Alamitos 1996 pp 42-48 Accessible athttpwwwccgatechedufacMelodyMoorepapersWCRE96ps
[15] MM Moore ldquoRepresentation Issues for Reengineering In-teractive Systemsrdquo ACM Computing Surveys Vol 28 No 4December 1996 Article 199 Accessible at httpwwwacmorgpubsarticlesjournalssurveys1996-28-4esa199-moorea199-moorehtml
[16] MM Moore and S Rugaber ldquoUsing Knowledge Repre-sentation to Understand Interactive Systemsrdquo Proc of theFifth International Workshop on Program ComprehensionIWPC97 (Dearborn 28-30 May 1997) IEEE Computer So-ciety Press Los Alamitos 1997 Accessible at httpwwwccgatechedufacMelodyMoorepapersWPC97ps
[17] MM Moore and S Rugaber ldquoDomain Analysis for Trans-formational Reuserdquo Proc of 4th Working Conf on ReverseEngineering WCRErsquo97 (6-8 October 1997) IEEE ComputerSociety Press Los Alamitos 1997
[18] ldquoOpen Interfacetraderdquo Neuron Data 156 University AvenuePalo Alto CA 94301 1992
[19] R Pausch M Conway and R DeLine ldquoLessons Learnedfrom SUIT the Simple User Interface Toolkitrdquo ACM Transon Office Information Systems Vol 10 No 4 October1992 pp 320-344 Accessible at httpwwwcsvirginiaedu~uigroupdocspublicationsSuitlessonspaper ps
[20] AR Puerta ldquoThe MECANO Project Comprehensive andIntegrated Support for Model-Based Interface DevelopmentrdquoProc of the 2nd Int W on Computer-Aided Design of UserInterfaces CADUIrsquo96 (Namur 5-7 June 1996) Presses Uni-versitaires de Namur Namur 1996 pp 19-36
[21] REK Stirewalt and S Rugaber ldquoAutomating UI Genera-tion by Model Compositionrdquo Journal of Automated Soft-ware Engineering Vol 7 No 2 1998 pp 101-124 Acces-sible at httpwwwccgatechedureverserepositorygenerps
[22] REK Stirewalt ldquoMDL A Language for Binding User-Interface Modelsrdquo in [26] pp 159-184
[23] P Szekely P Luo and R Neches ldquoBeyond Interface Build-ers Model-Based Interface Toolsrdquo Proc of ACM Conf onHuman Aspects in Computing Systems InterCHIrsquo93 ACMPress New York 1993 pp 383-390
[24] P Szekely ldquoRetrospective and Challenges for Model-BasedInterface Developmentrdquo Proc of 3rd Int Workshop on Com-puter-Aided Design of User Interfaces CADUIrsquo96 (Namur5-7 June 1996) Presses Universitaires de Namur Namur1996 pp xxi-xliv
[25] J Vanderdonckt and F Bodart ldquoEncapsulating Knowledgefor Intelligent Interaction Objects Selectionrdquo Proc of In-terCHIrsquo93 ACM Press New York 1993 pp 424-429
[26] J Vanderdonckt and P Berquin ldquoTowards a Very LargeModel-based Approach for User Interface DevelopmentrdquoProc of 1st Int Workshop on User Interfaces to Data Inten-sive Systems UIDISrsquo99 IEEE Computer Society Press LosAlamitos 1999 pp 76-85
[27] J Vanderdonckt and A Puerta (eds) Proc of the 3rd IntConf on Computer-Aided Design of User Interfaces CA-DUIrsquo99 Kluwer Academics Publishers Dordrecht 1999
[28] The XIML Consortium httpwwwximlorg 2002
tic core component and (iii) submitting it to the staticanalysis algorithm for reverse engineering depicted in fig11 Thanks to the HTML code accessibility remote reverseengineering of Web pages is permitted in addition the de-sign of automated tools for reverse engineering should bemuch easier for web pages than for compiled program codein C or Pascal The phases of this algorithm are now de-tailed in the subsequent subsections as they are consideredin VAQUITA (httpwwwisysuclacbebchiresearchvaquitahtm) a tool that enables designers to reverse engineer aweb page into one or several presentation models Cur-rently only the presentation model is reverse engineered
Inclusionexclusion of presentation elementsInitialization of the Web pages pool
Web pages pool empty
Selection of an individual Web page
Identification of Concrete Interaction Objects
Transformation of Concrete Interaction Objectsinto Abstract Interaction Objects
Selection of Logical Window
Hierarchy building for LWs and PUs
Update vectors of links and Web pages pool
Identification of links
Abstraction of links into Windows Transitions
Identification of Navigation Structures
Buildingofpresentationmodel
Buildingofdialogmodel
yes
no
Fig 11 Reverse engineering algorithmInitialization To initiate the process the designer specifieswhether she wants to reverse engineer a whole web site (byproviding the home pagersquos URL) any portion of a site (byproviding a starting URL and the depth level up to whichpages will be considered) or a series of web pages (by en-tering their URLs locally andor from the Web) A first poolof candidate pagesrsquo URLs is formedTo allow more flexibility in the reverse engineering proc-ess the designer can include or exclude not only anyHTML tag or element but also any attribute of each tag Toavoid reparametrization this setting canbe saved in a con-figuration file for future usage
Presentation modeling An individual web page is firstselected in the pagesrsquo pool and then submitted to identifica-tion of CIOs The basic idea of the algorithm is to identifyindividual HTML elements to map them onto relevantAIO while building a hierarchy of AIOs The HTML codeof the web page is parsed to identify these HTML elementsThe mapping is based on a set of heuristics that depend onthe HTML element or tag analyzed Examples of such heu-ristics include Each web page is mapped onto a LW in the selection of
LW phase The ltTITLEgt if any is mapped onto the labelslot of this AIO Any HTML element can be mapped toone or many AIOs with one or many properties included
Heading levels (eg H1 H2 H3) are mapped onto re-spective composite presentation AIOs and to somestructure in the progressively forming hierarchy The textprovided within the ltHgtltHgt scope is mapped onto theLabel abstract attribute of the AIO
Each ltFORMgt tag is mapped onto a composite controlAIO Multiple or embedded forms generate multiplecomposite AIOs on corresponding hierarchy levelsltTRgt ltTDgt tags denoting tables are treated similarly
Web pagersquos frames are mapped onto separate LWs asthey are themselves partial views on other pages
Static banners icons graphics illustrations and GIF filesare listed in the ldquoimagesrdquo category of a ldquoGraphicsrdquo sim-ple presentation AIO animated GIFs or video are listedin its ldquoanimationrdquo category while JPEG files are believedto relate to the ldquophotographrdquo category
Portions of text are equally treated like labels with theirstatic properties being fed into the abstract attributes ofthe AIO eg the currently active BGCOLOR is mappedonto BackGroundColor and TEXT is mapped onto Fore-GroundColor When text is subject to an anchor an addi-tional control AIO is created
HTML proprietary CIOs are mapped onto their corre-sponding simple control AIOs Examples include theltSELECTgt and ltTEXTAREAgt tags and the TYPE at-tribute of the ltINPUTgt tag The NAME tag is mappedonto an identification label for the related AIO The con-tent of the VALUE tag is mapped onto the abstract attrib-ute DefaultValue and so forth
Dialog modeling To reverse engineer a dialog model andfinding out instances of transitions and navigational struc-tures a vector of links is created for each page according tothe following format
V(URL)=list of intra-page links list of extra-page linkswhere each link has the form
L = (AIO_name target_URL link_type occurrence)where AIO_name = identifier in the presentation model
of the AIO holding the link source consideredtarget_URL= URL of the page linkedlink_type = (intra-page extra-page request)occurrence = amount of same links in the page
Once any individual page has been processed regarding toits own presentation model and regarding to its own vectorof links this vector is parsed to update the pool of candi-date web pages to analyze In the identification of links thisanalysis exploits positive and negative filters defined in theinitialization For example all external links or all extra-page links up to a certain depth level are omitted The ana-lyzed web page is then withdrawn from the pagesrsquo poolThe pool is examined to find a candidate recursively untilno candidate page still exists After exhausting the wholepagesrsquo pool the remainder of both dialog and presentationmodels is reverse engineered First vectors of links thathave been built for the collection of web pages in consid-eration are examined to identify LW transitions from eachpair of LWs Transformation rules include Any link with multiple occurrences within the same page
is reduced to a single link to avoid repetition All links with multiple sources within the same page
(eg a textual link a push button an icon) to the sametarget page are gathered Depending on parameters allinstances of such links can be kept or reduced to any par-ticular type For instance links can be restricted to onlyone occurrence of anchor and one occurrence of icon Inthis case the presentation model can combine both AIOsinto a single one of the ldquoiconrdquo type whose label is theanchor text Again positive and negative filters canautomatically expand or reduce the scope of AIOs inconcern for example considering only push buttons andforgetting all other types of AIOs for links
Any one-way link is transformed into a unidirectionaltransition Depending on the link type the operations onthe source and target LWs are specified1 When a link replaces one page by another the transi-
tion will contain a ldquocloserdquo operation on the source LWand a ldquomaximizerdquo operation on the target LW
2 When a link just pops up another page the transitionwill contain only a ldquodisplay with normal overlappingrdquooperation on the target LW
3 When a link pops up another page in a constrainedwindow the transition will contain a ldquodisplay withsystem-defined overlappingrdquo on the target LW
Any pair of reciprocal links between two pages is trans-formed into a single bi-directional transition between thetwo LWs
Any intra-page link is transformed into a control AIObranching to the related composite AIO belonging to thesame LW Normally intra-page links are intended tofasten visualization and avoid scrolling rather than pro-viding LW transitions This information is still re-cordedas in the future these AIOs may be replaced by some-thing else for example another LW In this design optionparts of a web page that are accessed by intra-page linksmay be replaced by separate LWs
Transitions between logical windows abstracting these linksare subsequently stored in a matrix Each matrix columndenotes a source LW while each matrix line denotes a tar-get LW Thus any cell denotes a window transition be-tween a pair of windowsThe resulting matrix is then passed to a pattern-matchingalgorithm that is in charge of identifying navigationalstructures among the LWs This algorithm begins byguessing the simplest navigational structures (such as se-quential or indexed navigation) and continues with morerich structures (such as guided or mixed navigations) Asthe window transition matrix can be rather sparse or nonregular finding the exact materialization of these naviga-tional structures can be infrequentThe next step is to generalize the existing structure to theclosest abstract structure type Of course each existingstructure can be generalized into a global navigation struc-ture But this network of LWs probably generates too manywindow transitions In order to moderate this generalizationprocess a threshold can be specified This parameter canfor example regulate how many transitions can be added inorder to match to the closest strict navigational structure Asthis algorithm may be incomplete the designer is allowed tocheck the resulting navigational structures The designermay accept reject or modify the structure produced the al-gorithm and she also may declare undiscovered structuresThis concludes the building of the dialog model
RE-ENGINEERING OF WEB PAGESNow that presentation and dialog models are available fromreverse engineering of web pages these two models cantheoretically benefit from forward engineering any model-based approach should be able to generate a new UI fromthese two models to be further re-integrated with the se-mantic core component Before reengineering web pagesinto a new UI according to the process depicted in fig 12 itis important to specify the context in which this new UI will
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and reengineering
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Fig 12 UI Reengineering
take place As indicated in the introduction technologicalpush and user pull are both increasing the demand for ac-cessing web sites in various contexts of use These varia-tions may depend on1 User several users will access the site each user be-
longing to a specific stereotype of the user populationEach stereotype ha its own set of parameters such aspreferences customized features interaction historycognitive profile typing vs selection skills motor- sight- or auditive-impairment and specific needs
2 Computing platform many different computing plat-forms can be used to access a site and each platformbrings its own set of constraints such as screen resolu-tion number of colors color palettes planes UI lan-guage (programming language script language ortab)based language are very different) network and in-teraction devices These two last constraints possessthemselves further constraints such as network band-width interaction devices capabilities and availability
3 Environment users access a site in different environ-ments Ambient conditions add a new set of constraintsincluding physical parameters (eg noise light workingspace room heat) and psychological social parameters(eg stressful time slot productivity task criticality un-stable environment collaborative or cooperative condi-tions home vs work)
Reengineering web pages to support all these variations ofthe context of use would be an ambitious task As we as-sumed in the introduction to mainly address cross-plat-formsupport this paper is focused on adding a computing plat-form model only We feel that techniques that are found tobe useful for addressing platform constraints will also beable to handle constraints that come from the user and theenvironment The rapid proliferation of various platformsfor web accessibility makes this a particularly timely con-cern Users increasingly want to compute while movingbetween locations thus moving from one platform (eg aWindows PC) to another (eg a Windows CE-compatibledevice) Other users do not want to move from one place toan-other but want to have multiple access from a singleplace (eg connecting a PDA to a PC)Many parameters vary across platforms VAQUITA onlysupport screen resolution variation for one UI language(ie HTML) as summarized in table 1 The concepts re-main the same without loss of generality
Context with no parameter varyingIn this context a new presentation model can be inferredfrom the existing models by considering at least 5 designs1 Redistribution of LWs within a single PU2 Redistribution of Composite AIOs within a single LW3 Redistribution of Simple AIOs within a single LW4 Redistribution of all AIOs within a single LW5 Redistribution of all AIOs within a single PU
Individual AIOs should remain unchanged but any redistri-bution of these elements can be imagined provided that thescreen constraints are satisfied To verify these constraintsthe presentation model needs to be enriched by two AIOabstract attributes typical height and length These attrib-utes are assigned to an average value in pixels for dimen-sion-fixed widgets (eg a check box a radio button)
These attributes are assigned to a value that equals a multi-plying coefficient in pixels times the character length fordimension-varying widgets (eg an edit box a list box)Other dimensions are inferred or estimated from the origi-nal CIOs themselves (eg an image thanks to theHEIGHT=x WIDTH=y tags) Analyzing all possible abovereconfigurations is beyond the scope of this paper We referto [225] for analyzing redistribution of LWs within a samePU (ie minimal maximal inputoutput functional userdefined grouped ungrouped)
Context with varying UI languageIn this context only the underlying UI language changesThis often occurs when multiple computing platforms run-ning different operating systems and presentation managersare considered Some languages are Internet-compatible(eg HTML Java WML) some are not (eg Visual Ba-sic C++) Beyond redistributions described in Section 41this context poses a new challenge all these languages donot share similar capabilities in terms of CIOs of the sourcecomputing platformFor instance an AIO may be mapped onto zero CIOs (thecorresponding CIO does not exist in the target language)one (the most frequent and easy case where a one-to-onerelationship exists between an AIO and its correspondingCIO) or many CIOs (several CIOs are required to repro-duce the same behavior of the intended AIO) To allowthese mappings a platform-specific transformation tableallows any AIO instance to be assigned to correspondingCIOsWhen no corresponding CIO exists an alternative CIO isproposed (ultimately the edit box is suggested) but can betailored by the designer Nevertheless most access devicesor specific computing platforms coming with their ownproprietary language also impose some screen resolutionchange For instance Wireless Markup Language (WML)can only be used on cellular phones Voice XML only onwith a screen reader A third context is then considered
Context with varying screen resolutionIt is a property of HTML that many HTML-compliantbrowsers on many types of screen can display it Althoughthis display is technically feasible (ie the browser adaptsthe presentation with respect to the platform) it may be un-usable to the end-user for one or another reason eg lossof information structure overcrowded screen too manyscroll bars image reduction) Therefore it makes sense toconsider how to re-engineer the UI for different screenresolutions This need is exacerbated by multiple HTMLbrowsers On different platforms equipped with different monitors
(ranging from workstation screen to small laptop) On different access devices with built-in browser (rang-
ing from the Internet ScreenPhone with 640x480 reso-lution to HTML-compatible PDA)
If the screen resolution is increasing the centered table withthree columns can be used to keep a presentation centeredon the screen with borders (fig 13) If the resolution is de-creasing the above redistributions are no longer applicableThree strategies have been explored so far
Fig 13 Presentation centered on the screen1 AIO replacement each AIO definition is augmented
with a list of potential replacements considered as eitherbehaviorally equivalent but consuming less space (ega list box to a drop-down list box) or degraded (eg anaccumulator to a multiple-value list box)
2 AIO size reduction typical height and length of AIOsare submitted to a possible reduction depending on us-ability constraints For example images and icons canbe reduced to a certain limit specified by a thresholdeg the minimally usable size of icons
3 AIO ungrouping when individual AIOs cannot be re-duced according to the two first strategies the distribu-tion of AIOs may be reconfigured AIOs can be groupedor ungrouped and new LWs can be created to contain alimited amount of AIOs
Again the development of these strategies bears further ex-planation which we hope to conduct in a later paper
Context with varying language and resolutionThis context is most likely to occur when a completely dif-ferent computing platform of access device is considered
For example a cellular phone only supports WML WebTVonly accepts a subset of HTML V3 and screen readersonly accept VoiceXML This context can be handled usinga combination of the approaches described in previouscontexts
RELATED WORKCross-platform development is not new as several environ-ments provide support for this purpose Galaxy [6] andOpen Interface [17] render the same UI on different plat-forms with their native look amp feel while SUIT [18] em-ploys a unique UI definition that can be processed on mul-tiple platforms However not one of these systems trulyadopts a model-based approach although SUITs commondefinition holds some presentation abstractions CT-UIMS[9] pioneered the platform model by supporting some AIO[15] redistribution for OSFMotif large screen and smallMacintosh screens AUIDL [101112] is probably the firstset of abstractions for reverse and re-engineering UIs frominternal hierarchical structures type and variable declara-tions a UI can be recovered in IDL with a presentationmodel (based on OO paradigm) and a dialog model (basedon Milnerrsquos process algebra) MORPH exploits productionrules [13] to infer AIOs [14] from CIOs and thus producesa graphical UI from a textual UI [15] This transformationalapproach is similar in principle to ours Forward engineer-ing can be executed by model composition [20] binding[21] or derivation thus proving that the approach is feasi-ble MORALE [1] is an extensive set of techniques andtools for reverse engineering legacy systems rather thanweb pages CELLEST [7] reverse engineers similar sys-tems but into DHTML thus adopting active models ratherthan passive models
CONCLUSIONThis paper has introduced a model-based approach sup-porting both reverse and re-engineering of web pages Thearchitecture outlined in fig 14 assumes that all models arestored in a model repository each model being specifiedwith the eXtensible Interface Markup Language (XIML)promoted by the XIML Consortium [28] This model tex-tual specification is declarative analyzable and editable[22]
HTMLpage
VAQUITA Web pagereverse engineering
User interfaceRe-engineering
XIMLPresentation
model
HTMLpage
WMLdeck
hellip
Model editor validator
Fig 14 Model-based approach architecture
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and redocumentation
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Platformmodel(1) (2)
Fig 15 UI Redocumentation
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
User interface reverse engineering
Taskmodel
Domainmodel
Usermodel
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
Application reverse engineering
Use
r ta
sk a
nd d
omai
nre
vers
e en
gine
erin
g
Log filesPredictive modelsDesign heuristics
Usability guidelinesFig 16 UI Design recovery
This ongoing work is just at the beginning as it will requirea significant amount of work to create a comprehensive en-vironment with tool-support for model-based reengineeringof web sites The described approach inevitably suffersfrom many restrictions and lack of support such as Non-standard HTML elements (eg embedded objects
browser-specific tags) or languages (eg JavaScriptfunctions ActiveX controls) are ignored as their supportwould require expensive development efforts
Dynamically created web pages are not supported sincetheir HTML code is generated on the fly The callbackroutines attached to AIOs producing dynamic pages canbe replaced by direct calls to the semantic core Activemodels are in this case too expensive to manage re-engineering on the fly However we can imagine to par-tially support applications that use explicit template-based page generation (eg XSLT JSP) in this case thestatic analysis should be able to differentiate variableparts from run-time provided parts
Style sheets are unsupported in the outlined algorithm asare other relevant presentation aspects For instancegraphical images designed as tabs (eg on www ama-zoncom) or links placed on top of such graphics are notrecognized and therefore cannot be mapped onto severalLWs of a same PU
On the other hand the environment we have described mayenvision other forms of UI reverse engineering [3] as repre-sented in table 1
Redocumentation (fig 15) this flow is similar to re-engineering except that the platform model is no longerneeded as it remains constant over time
Restructuring (fig 15 with arrow (1) used only) the UIremains basically the same for semantics and dialog butthe presentation model changes The scope of the plat-form model is restricted to re-engineering
Redesigning (fig 15 with arrows (1) and (2) used) theUI remains identical regarding its semantics but both thedialog (eg other interaction styles techniques or gen-res) and the presentation (eg any redistribution) modelscan change due to platform constraints This may consistsin a new category of UI re-engineering
Design recovery (fig 16) in this process it is expectedthat task and domain models could be recovered Guess-ing these models from HTML code seems impracticalbut other information sources might be investigated suchas log files produced by user traffic comparison withpredictive models design heuristics to identify usagepatterns and automated usability analysis based onguidelines
REFERENCES[1] Barclay PJ Griffiths T Mc Kirdy J Paton NW Coo-
per R Kennedy J ldquoThe Teallach Tool Using Models forFlexible User Interface Designrdquo Proc of 3rd Int Conf onComputer-Aided Design of User Interfaces CADUIrsquo99 (Lou-vain-la-Neuve 21-23 October 1999) Kluwer AcademicsDordrecht (1999) 139ndash158 Accessible at httpimgcsmanacukteallachpublicationsCadui99CADUI99ps
[2] G Abowd A Goel DF Jerding M McCracken MM
Moore JW Murdock C Potts S Rugaber and L WillsldquoMORALEndashMission Oriented Architectural Legacy Evolu-tionrdquo Proc of Int Conf on Software Maintenance (1997)
[3] F Bodart A-M Hennebert J-M Leheureux and JVanderdonckt ldquoComputer-Aided Window Identification inTRIDENTrdquo Proc of the 5th IFIP TC13 Conf on Human-Computer Interaction Interactrsquo95 (Lillehammer 25-29 June1995) Chapman amp Hall London 1995 pp 331-336 Acces-sible at httpwwwinfofundpacbecgi-publipub-spec-paperRP-95-021
[4] EJ Chikofsky and JH Cross ldquoReverse Engineering andDesign Recovery A Taxonomyrdquo IEEE Software Vol 1 No7 January 1990 pp 13-17
[5] J-M De Baud and S Rugaber ldquoA Software Re-engineeringMethod Using Domain Modelsrdquo Proc of Int Conf on Soft-ware Maintenance (October 1995) pp 204-213
[6] J Eisenstein and A Puerta ldquoAdaptation in Automated User-Interface Designrdquo Proc of ACM Int Conf on IntelligentUser Interfaces IUIrsquo2000 (New Orleans 9-12 January 2000)ACM Press New York 2000 pp 74-81
[8] L Kong E Strouila B Matichuk ldquoLegacy Interface Migra-tion A Task-Centered Approachrdquo Proc of 8th Int Conf onHuman-Computer Interaction HCI Internationalrsquo99 (Mu-nich 22-27 August 1999) H-J Bullinger and J Ziegler(eds) Lawrence Erlbaum Associates MahwahLondon1999 pp 1167-1171 Accessible at httpwwwcsualbertaca~strouliaPapershci99ps
[9] F Lonczewski and S Schreiber ldquoThe FUSE-System an In-tegrated User Interface Design Environmentrdquo Proc of 2nd
Int Workshop on Coputer-Aided Design of User InterfacesCADUIrsquo96 (Namur 5-7 June 1996) Presses Universitairesde Namur Namur 1996 pp 37-56 Accessible at ftphpeick7informatiktu-muenchendepubpaperssisfuse_cadui96psgz
[10] Ch Maumlrtin ldquoA UIMS for Knowledge Based Interface Tem-plate Generation and Interactionrdquo Proc of Interactrsquo90 El-sevier Science Pub Amsterdam 1990 pp 651-657
[11] E Merlo JF Girard K Kontogiannis P Panangaden andR De Mori ldquoReverse Engineering of User Interfacesrdquo Procof 1st Working Conference on Reverse EngineeringWCRErsquo93 (Baltimore 21-23 May 1993) RC Waters EJChikofsky (eds) IEEE Computer Society Press LosAlamitos 1993 pp 171-179
[12] E Merlo P-Y Gagneacute and A Thiboutocirct ldquoInference ofgraphical AUIDL specifications for the reverse engineeringof user interfacesrdquo Proc of Int Conf on Software Mainte-nance (19-23 September 1994) IEEE Computer SocietyPress Los Alamitos 1994 pp 80-88
[13] E Merlo P-Y Gagneacute J-F Girard K Kontogiannis LHendren P Panagaden and R De Mori ldquoReengineeringUser Interfacesrdquo IEEE Software Vol 12 No 1 January1995 pp 64-73
[14] MM Moore ldquoRule-Based Detection for Reverse Engineer-ing User Interfacesrdquo Proc of 3rd Working Conf on ReverseEngineering WCRErsquo96 (Monterey 8-10 November 1996) L
Wills I Baxter E Chikofsky (eds) IEEE Computer SocietyPress Los Alamitos 1996 pp 42-48 Accessible athttpwwwccgatechedufacMelodyMoorepapersWCRE96ps
[15] MM Moore ldquoRepresentation Issues for Reengineering In-teractive Systemsrdquo ACM Computing Surveys Vol 28 No 4December 1996 Article 199 Accessible at httpwwwacmorgpubsarticlesjournalssurveys1996-28-4esa199-moorea199-moorehtml
[16] MM Moore and S Rugaber ldquoUsing Knowledge Repre-sentation to Understand Interactive Systemsrdquo Proc of theFifth International Workshop on Program ComprehensionIWPC97 (Dearborn 28-30 May 1997) IEEE Computer So-ciety Press Los Alamitos 1997 Accessible at httpwwwccgatechedufacMelodyMoorepapersWPC97ps
[17] MM Moore and S Rugaber ldquoDomain Analysis for Trans-formational Reuserdquo Proc of 4th Working Conf on ReverseEngineering WCRErsquo97 (6-8 October 1997) IEEE ComputerSociety Press Los Alamitos 1997
[18] ldquoOpen Interfacetraderdquo Neuron Data 156 University AvenuePalo Alto CA 94301 1992
[19] R Pausch M Conway and R DeLine ldquoLessons Learnedfrom SUIT the Simple User Interface Toolkitrdquo ACM Transon Office Information Systems Vol 10 No 4 October1992 pp 320-344 Accessible at httpwwwcsvirginiaedu~uigroupdocspublicationsSuitlessonspaper ps
[20] AR Puerta ldquoThe MECANO Project Comprehensive andIntegrated Support for Model-Based Interface DevelopmentrdquoProc of the 2nd Int W on Computer-Aided Design of UserInterfaces CADUIrsquo96 (Namur 5-7 June 1996) Presses Uni-versitaires de Namur Namur 1996 pp 19-36
[21] REK Stirewalt and S Rugaber ldquoAutomating UI Genera-tion by Model Compositionrdquo Journal of Automated Soft-ware Engineering Vol 7 No 2 1998 pp 101-124 Acces-sible at httpwwwccgatechedureverserepositorygenerps
[22] REK Stirewalt ldquoMDL A Language for Binding User-Interface Modelsrdquo in [26] pp 159-184
[23] P Szekely P Luo and R Neches ldquoBeyond Interface Build-ers Model-Based Interface Toolsrdquo Proc of ACM Conf onHuman Aspects in Computing Systems InterCHIrsquo93 ACMPress New York 1993 pp 383-390
[24] P Szekely ldquoRetrospective and Challenges for Model-BasedInterface Developmentrdquo Proc of 3rd Int Workshop on Com-puter-Aided Design of User Interfaces CADUIrsquo96 (Namur5-7 June 1996) Presses Universitaires de Namur Namur1996 pp xxi-xliv
[25] J Vanderdonckt and F Bodart ldquoEncapsulating Knowledgefor Intelligent Interaction Objects Selectionrdquo Proc of In-terCHIrsquo93 ACM Press New York 1993 pp 424-429
[26] J Vanderdonckt and P Berquin ldquoTowards a Very LargeModel-based Approach for User Interface DevelopmentrdquoProc of 1st Int Workshop on User Interfaces to Data Inten-sive Systems UIDISrsquo99 IEEE Computer Society Press LosAlamitos 1999 pp 76-85
[27] J Vanderdonckt and A Puerta (eds) Proc of the 3rd IntConf on Computer-Aided Design of User Interfaces CA-DUIrsquo99 Kluwer Academics Publishers Dordrecht 1999
[28] The XIML Consortium httpwwwximlorg 2002
Once any individual page has been processed regarding toits own presentation model and regarding to its own vectorof links this vector is parsed to update the pool of candi-date web pages to analyze In the identification of links thisanalysis exploits positive and negative filters defined in theinitialization For example all external links or all extra-page links up to a certain depth level are omitted The ana-lyzed web page is then withdrawn from the pagesrsquo poolThe pool is examined to find a candidate recursively untilno candidate page still exists After exhausting the wholepagesrsquo pool the remainder of both dialog and presentationmodels is reverse engineered First vectors of links thathave been built for the collection of web pages in consid-eration are examined to identify LW transitions from eachpair of LWs Transformation rules include Any link with multiple occurrences within the same page
is reduced to a single link to avoid repetition All links with multiple sources within the same page
(eg a textual link a push button an icon) to the sametarget page are gathered Depending on parameters allinstances of such links can be kept or reduced to any par-ticular type For instance links can be restricted to onlyone occurrence of anchor and one occurrence of icon Inthis case the presentation model can combine both AIOsinto a single one of the ldquoiconrdquo type whose label is theanchor text Again positive and negative filters canautomatically expand or reduce the scope of AIOs inconcern for example considering only push buttons andforgetting all other types of AIOs for links
Any one-way link is transformed into a unidirectionaltransition Depending on the link type the operations onthe source and target LWs are specified1 When a link replaces one page by another the transi-
tion will contain a ldquocloserdquo operation on the source LWand a ldquomaximizerdquo operation on the target LW
2 When a link just pops up another page the transitionwill contain only a ldquodisplay with normal overlappingrdquooperation on the target LW
3 When a link pops up another page in a constrainedwindow the transition will contain a ldquodisplay withsystem-defined overlappingrdquo on the target LW
Any pair of reciprocal links between two pages is trans-formed into a single bi-directional transition between thetwo LWs
Any intra-page link is transformed into a control AIObranching to the related composite AIO belonging to thesame LW Normally intra-page links are intended tofasten visualization and avoid scrolling rather than pro-viding LW transitions This information is still re-cordedas in the future these AIOs may be replaced by some-thing else for example another LW In this design optionparts of a web page that are accessed by intra-page linksmay be replaced by separate LWs
Transitions between logical windows abstracting these linksare subsequently stored in a matrix Each matrix columndenotes a source LW while each matrix line denotes a tar-get LW Thus any cell denotes a window transition be-tween a pair of windowsThe resulting matrix is then passed to a pattern-matchingalgorithm that is in charge of identifying navigationalstructures among the LWs This algorithm begins byguessing the simplest navigational structures (such as se-quential or indexed navigation) and continues with morerich structures (such as guided or mixed navigations) Asthe window transition matrix can be rather sparse or nonregular finding the exact materialization of these naviga-tional structures can be infrequentThe next step is to generalize the existing structure to theclosest abstract structure type Of course each existingstructure can be generalized into a global navigation struc-ture But this network of LWs probably generates too manywindow transitions In order to moderate this generalizationprocess a threshold can be specified This parameter canfor example regulate how many transitions can be added inorder to match to the closest strict navigational structure Asthis algorithm may be incomplete the designer is allowed tocheck the resulting navigational structures The designermay accept reject or modify the structure produced the al-gorithm and she also may declare undiscovered structuresThis concludes the building of the dialog model
RE-ENGINEERING OF WEB PAGESNow that presentation and dialog models are available fromreverse engineering of web pages these two models cantheoretically benefit from forward engineering any model-based approach should be able to generate a new UI fromthese two models to be further re-integrated with the se-mantic core component Before reengineering web pagesinto a new UI according to the process depicted in fig 12 itis important to specify the context in which this new UI will
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and reengineering
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Fig 12 UI Reengineering
take place As indicated in the introduction technologicalpush and user pull are both increasing the demand for ac-cessing web sites in various contexts of use These varia-tions may depend on1 User several users will access the site each user be-
longing to a specific stereotype of the user populationEach stereotype ha its own set of parameters such aspreferences customized features interaction historycognitive profile typing vs selection skills motor- sight- or auditive-impairment and specific needs
2 Computing platform many different computing plat-forms can be used to access a site and each platformbrings its own set of constraints such as screen resolu-tion number of colors color palettes planes UI lan-guage (programming language script language ortab)based language are very different) network and in-teraction devices These two last constraints possessthemselves further constraints such as network band-width interaction devices capabilities and availability
3 Environment users access a site in different environ-ments Ambient conditions add a new set of constraintsincluding physical parameters (eg noise light workingspace room heat) and psychological social parameters(eg stressful time slot productivity task criticality un-stable environment collaborative or cooperative condi-tions home vs work)
Reengineering web pages to support all these variations ofthe context of use would be an ambitious task As we as-sumed in the introduction to mainly address cross-plat-formsupport this paper is focused on adding a computing plat-form model only We feel that techniques that are found tobe useful for addressing platform constraints will also beable to handle constraints that come from the user and theenvironment The rapid proliferation of various platformsfor web accessibility makes this a particularly timely con-cern Users increasingly want to compute while movingbetween locations thus moving from one platform (eg aWindows PC) to another (eg a Windows CE-compatibledevice) Other users do not want to move from one place toan-other but want to have multiple access from a singleplace (eg connecting a PDA to a PC)Many parameters vary across platforms VAQUITA onlysupport screen resolution variation for one UI language(ie HTML) as summarized in table 1 The concepts re-main the same without loss of generality
Context with no parameter varyingIn this context a new presentation model can be inferredfrom the existing models by considering at least 5 designs1 Redistribution of LWs within a single PU2 Redistribution of Composite AIOs within a single LW3 Redistribution of Simple AIOs within a single LW4 Redistribution of all AIOs within a single LW5 Redistribution of all AIOs within a single PU
Individual AIOs should remain unchanged but any redistri-bution of these elements can be imagined provided that thescreen constraints are satisfied To verify these constraintsthe presentation model needs to be enriched by two AIOabstract attributes typical height and length These attrib-utes are assigned to an average value in pixels for dimen-sion-fixed widgets (eg a check box a radio button)
These attributes are assigned to a value that equals a multi-plying coefficient in pixels times the character length fordimension-varying widgets (eg an edit box a list box)Other dimensions are inferred or estimated from the origi-nal CIOs themselves (eg an image thanks to theHEIGHT=x WIDTH=y tags) Analyzing all possible abovereconfigurations is beyond the scope of this paper We referto [225] for analyzing redistribution of LWs within a samePU (ie minimal maximal inputoutput functional userdefined grouped ungrouped)
Context with varying UI languageIn this context only the underlying UI language changesThis often occurs when multiple computing platforms run-ning different operating systems and presentation managersare considered Some languages are Internet-compatible(eg HTML Java WML) some are not (eg Visual Ba-sic C++) Beyond redistributions described in Section 41this context poses a new challenge all these languages donot share similar capabilities in terms of CIOs of the sourcecomputing platformFor instance an AIO may be mapped onto zero CIOs (thecorresponding CIO does not exist in the target language)one (the most frequent and easy case where a one-to-onerelationship exists between an AIO and its correspondingCIO) or many CIOs (several CIOs are required to repro-duce the same behavior of the intended AIO) To allowthese mappings a platform-specific transformation tableallows any AIO instance to be assigned to correspondingCIOsWhen no corresponding CIO exists an alternative CIO isproposed (ultimately the edit box is suggested) but can betailored by the designer Nevertheless most access devicesor specific computing platforms coming with their ownproprietary language also impose some screen resolutionchange For instance Wireless Markup Language (WML)can only be used on cellular phones Voice XML only onwith a screen reader A third context is then considered
Context with varying screen resolutionIt is a property of HTML that many HTML-compliantbrowsers on many types of screen can display it Althoughthis display is technically feasible (ie the browser adaptsthe presentation with respect to the platform) it may be un-usable to the end-user for one or another reason eg lossof information structure overcrowded screen too manyscroll bars image reduction) Therefore it makes sense toconsider how to re-engineer the UI for different screenresolutions This need is exacerbated by multiple HTMLbrowsers On different platforms equipped with different monitors
(ranging from workstation screen to small laptop) On different access devices with built-in browser (rang-
ing from the Internet ScreenPhone with 640x480 reso-lution to HTML-compatible PDA)
If the screen resolution is increasing the centered table withthree columns can be used to keep a presentation centeredon the screen with borders (fig 13) If the resolution is de-creasing the above redistributions are no longer applicableThree strategies have been explored so far
Fig 13 Presentation centered on the screen1 AIO replacement each AIO definition is augmented
with a list of potential replacements considered as eitherbehaviorally equivalent but consuming less space (ega list box to a drop-down list box) or degraded (eg anaccumulator to a multiple-value list box)
2 AIO size reduction typical height and length of AIOsare submitted to a possible reduction depending on us-ability constraints For example images and icons canbe reduced to a certain limit specified by a thresholdeg the minimally usable size of icons
3 AIO ungrouping when individual AIOs cannot be re-duced according to the two first strategies the distribu-tion of AIOs may be reconfigured AIOs can be groupedor ungrouped and new LWs can be created to contain alimited amount of AIOs
Again the development of these strategies bears further ex-planation which we hope to conduct in a later paper
Context with varying language and resolutionThis context is most likely to occur when a completely dif-ferent computing platform of access device is considered
For example a cellular phone only supports WML WebTVonly accepts a subset of HTML V3 and screen readersonly accept VoiceXML This context can be handled usinga combination of the approaches described in previouscontexts
RELATED WORKCross-platform development is not new as several environ-ments provide support for this purpose Galaxy [6] andOpen Interface [17] render the same UI on different plat-forms with their native look amp feel while SUIT [18] em-ploys a unique UI definition that can be processed on mul-tiple platforms However not one of these systems trulyadopts a model-based approach although SUITs commondefinition holds some presentation abstractions CT-UIMS[9] pioneered the platform model by supporting some AIO[15] redistribution for OSFMotif large screen and smallMacintosh screens AUIDL [101112] is probably the firstset of abstractions for reverse and re-engineering UIs frominternal hierarchical structures type and variable declara-tions a UI can be recovered in IDL with a presentationmodel (based on OO paradigm) and a dialog model (basedon Milnerrsquos process algebra) MORPH exploits productionrules [13] to infer AIOs [14] from CIOs and thus producesa graphical UI from a textual UI [15] This transformationalapproach is similar in principle to ours Forward engineer-ing can be executed by model composition [20] binding[21] or derivation thus proving that the approach is feasi-ble MORALE [1] is an extensive set of techniques andtools for reverse engineering legacy systems rather thanweb pages CELLEST [7] reverse engineers similar sys-tems but into DHTML thus adopting active models ratherthan passive models
CONCLUSIONThis paper has introduced a model-based approach sup-porting both reverse and re-engineering of web pages Thearchitecture outlined in fig 14 assumes that all models arestored in a model repository each model being specifiedwith the eXtensible Interface Markup Language (XIML)promoted by the XIML Consortium [28] This model tex-tual specification is declarative analyzable and editable[22]
HTMLpage
VAQUITA Web pagereverse engineering
User interfaceRe-engineering
XIMLPresentation
model
HTMLpage
WMLdeck
hellip
Model editor validator
Fig 14 Model-based approach architecture
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and redocumentation
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Platformmodel(1) (2)
Fig 15 UI Redocumentation
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
User interface reverse engineering
Taskmodel
Domainmodel
Usermodel
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
Application reverse engineering
Use
r ta
sk a
nd d
omai
nre
vers
e en
gine
erin
g
Log filesPredictive modelsDesign heuristics
Usability guidelinesFig 16 UI Design recovery
This ongoing work is just at the beginning as it will requirea significant amount of work to create a comprehensive en-vironment with tool-support for model-based reengineeringof web sites The described approach inevitably suffersfrom many restrictions and lack of support such as Non-standard HTML elements (eg embedded objects
browser-specific tags) or languages (eg JavaScriptfunctions ActiveX controls) are ignored as their supportwould require expensive development efforts
Dynamically created web pages are not supported sincetheir HTML code is generated on the fly The callbackroutines attached to AIOs producing dynamic pages canbe replaced by direct calls to the semantic core Activemodels are in this case too expensive to manage re-engineering on the fly However we can imagine to par-tially support applications that use explicit template-based page generation (eg XSLT JSP) in this case thestatic analysis should be able to differentiate variableparts from run-time provided parts
Style sheets are unsupported in the outlined algorithm asare other relevant presentation aspects For instancegraphical images designed as tabs (eg on www ama-zoncom) or links placed on top of such graphics are notrecognized and therefore cannot be mapped onto severalLWs of a same PU
On the other hand the environment we have described mayenvision other forms of UI reverse engineering [3] as repre-sented in table 1
Redocumentation (fig 15) this flow is similar to re-engineering except that the platform model is no longerneeded as it remains constant over time
Restructuring (fig 15 with arrow (1) used only) the UIremains basically the same for semantics and dialog butthe presentation model changes The scope of the plat-form model is restricted to re-engineering
Redesigning (fig 15 with arrows (1) and (2) used) theUI remains identical regarding its semantics but both thedialog (eg other interaction styles techniques or gen-res) and the presentation (eg any redistribution) modelscan change due to platform constraints This may consistsin a new category of UI re-engineering
Design recovery (fig 16) in this process it is expectedthat task and domain models could be recovered Guess-ing these models from HTML code seems impracticalbut other information sources might be investigated suchas log files produced by user traffic comparison withpredictive models design heuristics to identify usagepatterns and automated usability analysis based onguidelines
REFERENCES[1] Barclay PJ Griffiths T Mc Kirdy J Paton NW Coo-
per R Kennedy J ldquoThe Teallach Tool Using Models forFlexible User Interface Designrdquo Proc of 3rd Int Conf onComputer-Aided Design of User Interfaces CADUIrsquo99 (Lou-vain-la-Neuve 21-23 October 1999) Kluwer AcademicsDordrecht (1999) 139ndash158 Accessible at httpimgcsmanacukteallachpublicationsCadui99CADUI99ps
[2] G Abowd A Goel DF Jerding M McCracken MM
Moore JW Murdock C Potts S Rugaber and L WillsldquoMORALEndashMission Oriented Architectural Legacy Evolu-tionrdquo Proc of Int Conf on Software Maintenance (1997)
[3] F Bodart A-M Hennebert J-M Leheureux and JVanderdonckt ldquoComputer-Aided Window Identification inTRIDENTrdquo Proc of the 5th IFIP TC13 Conf on Human-Computer Interaction Interactrsquo95 (Lillehammer 25-29 June1995) Chapman amp Hall London 1995 pp 331-336 Acces-sible at httpwwwinfofundpacbecgi-publipub-spec-paperRP-95-021
[4] EJ Chikofsky and JH Cross ldquoReverse Engineering andDesign Recovery A Taxonomyrdquo IEEE Software Vol 1 No7 January 1990 pp 13-17
[5] J-M De Baud and S Rugaber ldquoA Software Re-engineeringMethod Using Domain Modelsrdquo Proc of Int Conf on Soft-ware Maintenance (October 1995) pp 204-213
[6] J Eisenstein and A Puerta ldquoAdaptation in Automated User-Interface Designrdquo Proc of ACM Int Conf on IntelligentUser Interfaces IUIrsquo2000 (New Orleans 9-12 January 2000)ACM Press New York 2000 pp 74-81
[8] L Kong E Strouila B Matichuk ldquoLegacy Interface Migra-tion A Task-Centered Approachrdquo Proc of 8th Int Conf onHuman-Computer Interaction HCI Internationalrsquo99 (Mu-nich 22-27 August 1999) H-J Bullinger and J Ziegler(eds) Lawrence Erlbaum Associates MahwahLondon1999 pp 1167-1171 Accessible at httpwwwcsualbertaca~strouliaPapershci99ps
[9] F Lonczewski and S Schreiber ldquoThe FUSE-System an In-tegrated User Interface Design Environmentrdquo Proc of 2nd
Int Workshop on Coputer-Aided Design of User InterfacesCADUIrsquo96 (Namur 5-7 June 1996) Presses Universitairesde Namur Namur 1996 pp 37-56 Accessible at ftphpeick7informatiktu-muenchendepubpaperssisfuse_cadui96psgz
[10] Ch Maumlrtin ldquoA UIMS for Knowledge Based Interface Tem-plate Generation and Interactionrdquo Proc of Interactrsquo90 El-sevier Science Pub Amsterdam 1990 pp 651-657
[11] E Merlo JF Girard K Kontogiannis P Panangaden andR De Mori ldquoReverse Engineering of User Interfacesrdquo Procof 1st Working Conference on Reverse EngineeringWCRErsquo93 (Baltimore 21-23 May 1993) RC Waters EJChikofsky (eds) IEEE Computer Society Press LosAlamitos 1993 pp 171-179
[12] E Merlo P-Y Gagneacute and A Thiboutocirct ldquoInference ofgraphical AUIDL specifications for the reverse engineeringof user interfacesrdquo Proc of Int Conf on Software Mainte-nance (19-23 September 1994) IEEE Computer SocietyPress Los Alamitos 1994 pp 80-88
[13] E Merlo P-Y Gagneacute J-F Girard K Kontogiannis LHendren P Panagaden and R De Mori ldquoReengineeringUser Interfacesrdquo IEEE Software Vol 12 No 1 January1995 pp 64-73
[14] MM Moore ldquoRule-Based Detection for Reverse Engineer-ing User Interfacesrdquo Proc of 3rd Working Conf on ReverseEngineering WCRErsquo96 (Monterey 8-10 November 1996) L
Wills I Baxter E Chikofsky (eds) IEEE Computer SocietyPress Los Alamitos 1996 pp 42-48 Accessible athttpwwwccgatechedufacMelodyMoorepapersWCRE96ps
[15] MM Moore ldquoRepresentation Issues for Reengineering In-teractive Systemsrdquo ACM Computing Surveys Vol 28 No 4December 1996 Article 199 Accessible at httpwwwacmorgpubsarticlesjournalssurveys1996-28-4esa199-moorea199-moorehtml
[16] MM Moore and S Rugaber ldquoUsing Knowledge Repre-sentation to Understand Interactive Systemsrdquo Proc of theFifth International Workshop on Program ComprehensionIWPC97 (Dearborn 28-30 May 1997) IEEE Computer So-ciety Press Los Alamitos 1997 Accessible at httpwwwccgatechedufacMelodyMoorepapersWPC97ps
[17] MM Moore and S Rugaber ldquoDomain Analysis for Trans-formational Reuserdquo Proc of 4th Working Conf on ReverseEngineering WCRErsquo97 (6-8 October 1997) IEEE ComputerSociety Press Los Alamitos 1997
[18] ldquoOpen Interfacetraderdquo Neuron Data 156 University AvenuePalo Alto CA 94301 1992
[19] R Pausch M Conway and R DeLine ldquoLessons Learnedfrom SUIT the Simple User Interface Toolkitrdquo ACM Transon Office Information Systems Vol 10 No 4 October1992 pp 320-344 Accessible at httpwwwcsvirginiaedu~uigroupdocspublicationsSuitlessonspaper ps
[20] AR Puerta ldquoThe MECANO Project Comprehensive andIntegrated Support for Model-Based Interface DevelopmentrdquoProc of the 2nd Int W on Computer-Aided Design of UserInterfaces CADUIrsquo96 (Namur 5-7 June 1996) Presses Uni-versitaires de Namur Namur 1996 pp 19-36
[21] REK Stirewalt and S Rugaber ldquoAutomating UI Genera-tion by Model Compositionrdquo Journal of Automated Soft-ware Engineering Vol 7 No 2 1998 pp 101-124 Acces-sible at httpwwwccgatechedureverserepositorygenerps
[22] REK Stirewalt ldquoMDL A Language for Binding User-Interface Modelsrdquo in [26] pp 159-184
[23] P Szekely P Luo and R Neches ldquoBeyond Interface Build-ers Model-Based Interface Toolsrdquo Proc of ACM Conf onHuman Aspects in Computing Systems InterCHIrsquo93 ACMPress New York 1993 pp 383-390
[24] P Szekely ldquoRetrospective and Challenges for Model-BasedInterface Developmentrdquo Proc of 3rd Int Workshop on Com-puter-Aided Design of User Interfaces CADUIrsquo96 (Namur5-7 June 1996) Presses Universitaires de Namur Namur1996 pp xxi-xliv
[25] J Vanderdonckt and F Bodart ldquoEncapsulating Knowledgefor Intelligent Interaction Objects Selectionrdquo Proc of In-terCHIrsquo93 ACM Press New York 1993 pp 424-429
[26] J Vanderdonckt and P Berquin ldquoTowards a Very LargeModel-based Approach for User Interface DevelopmentrdquoProc of 1st Int Workshop on User Interfaces to Data Inten-sive Systems UIDISrsquo99 IEEE Computer Society Press LosAlamitos 1999 pp 76-85
[27] J Vanderdonckt and A Puerta (eds) Proc of the 3rd IntConf on Computer-Aided Design of User Interfaces CA-DUIrsquo99 Kluwer Academics Publishers Dordrecht 1999
[28] The XIML Consortium httpwwwximlorg 2002
take place As indicated in the introduction technologicalpush and user pull are both increasing the demand for ac-cessing web sites in various contexts of use These varia-tions may depend on1 User several users will access the site each user be-
longing to a specific stereotype of the user populationEach stereotype ha its own set of parameters such aspreferences customized features interaction historycognitive profile typing vs selection skills motor- sight- or auditive-impairment and specific needs
2 Computing platform many different computing plat-forms can be used to access a site and each platformbrings its own set of constraints such as screen resolu-tion number of colors color palettes planes UI lan-guage (programming language script language ortab)based language are very different) network and in-teraction devices These two last constraints possessthemselves further constraints such as network band-width interaction devices capabilities and availability
3 Environment users access a site in different environ-ments Ambient conditions add a new set of constraintsincluding physical parameters (eg noise light workingspace room heat) and psychological social parameters(eg stressful time slot productivity task criticality un-stable environment collaborative or cooperative condi-tions home vs work)
Reengineering web pages to support all these variations ofthe context of use would be an ambitious task As we as-sumed in the introduction to mainly address cross-plat-formsupport this paper is focused on adding a computing plat-form model only We feel that techniques that are found tobe useful for addressing platform constraints will also beable to handle constraints that come from the user and theenvironment The rapid proliferation of various platformsfor web accessibility makes this a particularly timely con-cern Users increasingly want to compute while movingbetween locations thus moving from one platform (eg aWindows PC) to another (eg a Windows CE-compatibledevice) Other users do not want to move from one place toan-other but want to have multiple access from a singleplace (eg connecting a PDA to a PC)Many parameters vary across platforms VAQUITA onlysupport screen resolution variation for one UI language(ie HTML) as summarized in table 1 The concepts re-main the same without loss of generality
Context with no parameter varyingIn this context a new presentation model can be inferredfrom the existing models by considering at least 5 designs1 Redistribution of LWs within a single PU2 Redistribution of Composite AIOs within a single LW3 Redistribution of Simple AIOs within a single LW4 Redistribution of all AIOs within a single LW5 Redistribution of all AIOs within a single PU
Individual AIOs should remain unchanged but any redistri-bution of these elements can be imagined provided that thescreen constraints are satisfied To verify these constraintsthe presentation model needs to be enriched by two AIOabstract attributes typical height and length These attrib-utes are assigned to an average value in pixels for dimen-sion-fixed widgets (eg a check box a radio button)
These attributes are assigned to a value that equals a multi-plying coefficient in pixels times the character length fordimension-varying widgets (eg an edit box a list box)Other dimensions are inferred or estimated from the origi-nal CIOs themselves (eg an image thanks to theHEIGHT=x WIDTH=y tags) Analyzing all possible abovereconfigurations is beyond the scope of this paper We referto [225] for analyzing redistribution of LWs within a samePU (ie minimal maximal inputoutput functional userdefined grouped ungrouped)
Context with varying UI languageIn this context only the underlying UI language changesThis often occurs when multiple computing platforms run-ning different operating systems and presentation managersare considered Some languages are Internet-compatible(eg HTML Java WML) some are not (eg Visual Ba-sic C++) Beyond redistributions described in Section 41this context poses a new challenge all these languages donot share similar capabilities in terms of CIOs of the sourcecomputing platformFor instance an AIO may be mapped onto zero CIOs (thecorresponding CIO does not exist in the target language)one (the most frequent and easy case where a one-to-onerelationship exists between an AIO and its correspondingCIO) or many CIOs (several CIOs are required to repro-duce the same behavior of the intended AIO) To allowthese mappings a platform-specific transformation tableallows any AIO instance to be assigned to correspondingCIOsWhen no corresponding CIO exists an alternative CIO isproposed (ultimately the edit box is suggested) but can betailored by the designer Nevertheless most access devicesor specific computing platforms coming with their ownproprietary language also impose some screen resolutionchange For instance Wireless Markup Language (WML)can only be used on cellular phones Voice XML only onwith a screen reader A third context is then considered
Context with varying screen resolutionIt is a property of HTML that many HTML-compliantbrowsers on many types of screen can display it Althoughthis display is technically feasible (ie the browser adaptsthe presentation with respect to the platform) it may be un-usable to the end-user for one or another reason eg lossof information structure overcrowded screen too manyscroll bars image reduction) Therefore it makes sense toconsider how to re-engineer the UI for different screenresolutions This need is exacerbated by multiple HTMLbrowsers On different platforms equipped with different monitors
(ranging from workstation screen to small laptop) On different access devices with built-in browser (rang-
ing from the Internet ScreenPhone with 640x480 reso-lution to HTML-compatible PDA)
If the screen resolution is increasing the centered table withthree columns can be used to keep a presentation centeredon the screen with borders (fig 13) If the resolution is de-creasing the above redistributions are no longer applicableThree strategies have been explored so far
Fig 13 Presentation centered on the screen1 AIO replacement each AIO definition is augmented
with a list of potential replacements considered as eitherbehaviorally equivalent but consuming less space (ega list box to a drop-down list box) or degraded (eg anaccumulator to a multiple-value list box)
2 AIO size reduction typical height and length of AIOsare submitted to a possible reduction depending on us-ability constraints For example images and icons canbe reduced to a certain limit specified by a thresholdeg the minimally usable size of icons
3 AIO ungrouping when individual AIOs cannot be re-duced according to the two first strategies the distribu-tion of AIOs may be reconfigured AIOs can be groupedor ungrouped and new LWs can be created to contain alimited amount of AIOs
Again the development of these strategies bears further ex-planation which we hope to conduct in a later paper
Context with varying language and resolutionThis context is most likely to occur when a completely dif-ferent computing platform of access device is considered
For example a cellular phone only supports WML WebTVonly accepts a subset of HTML V3 and screen readersonly accept VoiceXML This context can be handled usinga combination of the approaches described in previouscontexts
RELATED WORKCross-platform development is not new as several environ-ments provide support for this purpose Galaxy [6] andOpen Interface [17] render the same UI on different plat-forms with their native look amp feel while SUIT [18] em-ploys a unique UI definition that can be processed on mul-tiple platforms However not one of these systems trulyadopts a model-based approach although SUITs commondefinition holds some presentation abstractions CT-UIMS[9] pioneered the platform model by supporting some AIO[15] redistribution for OSFMotif large screen and smallMacintosh screens AUIDL [101112] is probably the firstset of abstractions for reverse and re-engineering UIs frominternal hierarchical structures type and variable declara-tions a UI can be recovered in IDL with a presentationmodel (based on OO paradigm) and a dialog model (basedon Milnerrsquos process algebra) MORPH exploits productionrules [13] to infer AIOs [14] from CIOs and thus producesa graphical UI from a textual UI [15] This transformationalapproach is similar in principle to ours Forward engineer-ing can be executed by model composition [20] binding[21] or derivation thus proving that the approach is feasi-ble MORALE [1] is an extensive set of techniques andtools for reverse engineering legacy systems rather thanweb pages CELLEST [7] reverse engineers similar sys-tems but into DHTML thus adopting active models ratherthan passive models
CONCLUSIONThis paper has introduced a model-based approach sup-porting both reverse and re-engineering of web pages Thearchitecture outlined in fig 14 assumes that all models arestored in a model repository each model being specifiedwith the eXtensible Interface Markup Language (XIML)promoted by the XIML Consortium [28] This model tex-tual specification is declarative analyzable and editable[22]
HTMLpage
VAQUITA Web pagereverse engineering
User interfaceRe-engineering
XIMLPresentation
model
HTMLpage
WMLdeck
hellip
Model editor validator
Fig 14 Model-based approach architecture
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and redocumentation
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Platformmodel(1) (2)
Fig 15 UI Redocumentation
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
User interface reverse engineering
Taskmodel
Domainmodel
Usermodel
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
Application reverse engineering
Use
r ta
sk a
nd d
omai
nre
vers
e en
gine
erin
g
Log filesPredictive modelsDesign heuristics
Usability guidelinesFig 16 UI Design recovery
This ongoing work is just at the beginning as it will requirea significant amount of work to create a comprehensive en-vironment with tool-support for model-based reengineeringof web sites The described approach inevitably suffersfrom many restrictions and lack of support such as Non-standard HTML elements (eg embedded objects
browser-specific tags) or languages (eg JavaScriptfunctions ActiveX controls) are ignored as their supportwould require expensive development efforts
Dynamically created web pages are not supported sincetheir HTML code is generated on the fly The callbackroutines attached to AIOs producing dynamic pages canbe replaced by direct calls to the semantic core Activemodels are in this case too expensive to manage re-engineering on the fly However we can imagine to par-tially support applications that use explicit template-based page generation (eg XSLT JSP) in this case thestatic analysis should be able to differentiate variableparts from run-time provided parts
Style sheets are unsupported in the outlined algorithm asare other relevant presentation aspects For instancegraphical images designed as tabs (eg on www ama-zoncom) or links placed on top of such graphics are notrecognized and therefore cannot be mapped onto severalLWs of a same PU
On the other hand the environment we have described mayenvision other forms of UI reverse engineering [3] as repre-sented in table 1
Redocumentation (fig 15) this flow is similar to re-engineering except that the platform model is no longerneeded as it remains constant over time
Restructuring (fig 15 with arrow (1) used only) the UIremains basically the same for semantics and dialog butthe presentation model changes The scope of the plat-form model is restricted to re-engineering
Redesigning (fig 15 with arrows (1) and (2) used) theUI remains identical regarding its semantics but both thedialog (eg other interaction styles techniques or gen-res) and the presentation (eg any redistribution) modelscan change due to platform constraints This may consistsin a new category of UI re-engineering
Design recovery (fig 16) in this process it is expectedthat task and domain models could be recovered Guess-ing these models from HTML code seems impracticalbut other information sources might be investigated suchas log files produced by user traffic comparison withpredictive models design heuristics to identify usagepatterns and automated usability analysis based onguidelines
REFERENCES[1] Barclay PJ Griffiths T Mc Kirdy J Paton NW Coo-
per R Kennedy J ldquoThe Teallach Tool Using Models forFlexible User Interface Designrdquo Proc of 3rd Int Conf onComputer-Aided Design of User Interfaces CADUIrsquo99 (Lou-vain-la-Neuve 21-23 October 1999) Kluwer AcademicsDordrecht (1999) 139ndash158 Accessible at httpimgcsmanacukteallachpublicationsCadui99CADUI99ps
[2] G Abowd A Goel DF Jerding M McCracken MM
Moore JW Murdock C Potts S Rugaber and L WillsldquoMORALEndashMission Oriented Architectural Legacy Evolu-tionrdquo Proc of Int Conf on Software Maintenance (1997)
[3] F Bodart A-M Hennebert J-M Leheureux and JVanderdonckt ldquoComputer-Aided Window Identification inTRIDENTrdquo Proc of the 5th IFIP TC13 Conf on Human-Computer Interaction Interactrsquo95 (Lillehammer 25-29 June1995) Chapman amp Hall London 1995 pp 331-336 Acces-sible at httpwwwinfofundpacbecgi-publipub-spec-paperRP-95-021
[4] EJ Chikofsky and JH Cross ldquoReverse Engineering andDesign Recovery A Taxonomyrdquo IEEE Software Vol 1 No7 January 1990 pp 13-17
[5] J-M De Baud and S Rugaber ldquoA Software Re-engineeringMethod Using Domain Modelsrdquo Proc of Int Conf on Soft-ware Maintenance (October 1995) pp 204-213
[6] J Eisenstein and A Puerta ldquoAdaptation in Automated User-Interface Designrdquo Proc of ACM Int Conf on IntelligentUser Interfaces IUIrsquo2000 (New Orleans 9-12 January 2000)ACM Press New York 2000 pp 74-81
[8] L Kong E Strouila B Matichuk ldquoLegacy Interface Migra-tion A Task-Centered Approachrdquo Proc of 8th Int Conf onHuman-Computer Interaction HCI Internationalrsquo99 (Mu-nich 22-27 August 1999) H-J Bullinger and J Ziegler(eds) Lawrence Erlbaum Associates MahwahLondon1999 pp 1167-1171 Accessible at httpwwwcsualbertaca~strouliaPapershci99ps
[9] F Lonczewski and S Schreiber ldquoThe FUSE-System an In-tegrated User Interface Design Environmentrdquo Proc of 2nd
Int Workshop on Coputer-Aided Design of User InterfacesCADUIrsquo96 (Namur 5-7 June 1996) Presses Universitairesde Namur Namur 1996 pp 37-56 Accessible at ftphpeick7informatiktu-muenchendepubpaperssisfuse_cadui96psgz
[10] Ch Maumlrtin ldquoA UIMS for Knowledge Based Interface Tem-plate Generation and Interactionrdquo Proc of Interactrsquo90 El-sevier Science Pub Amsterdam 1990 pp 651-657
[11] E Merlo JF Girard K Kontogiannis P Panangaden andR De Mori ldquoReverse Engineering of User Interfacesrdquo Procof 1st Working Conference on Reverse EngineeringWCRErsquo93 (Baltimore 21-23 May 1993) RC Waters EJChikofsky (eds) IEEE Computer Society Press LosAlamitos 1993 pp 171-179
[12] E Merlo P-Y Gagneacute and A Thiboutocirct ldquoInference ofgraphical AUIDL specifications for the reverse engineeringof user interfacesrdquo Proc of Int Conf on Software Mainte-nance (19-23 September 1994) IEEE Computer SocietyPress Los Alamitos 1994 pp 80-88
[13] E Merlo P-Y Gagneacute J-F Girard K Kontogiannis LHendren P Panagaden and R De Mori ldquoReengineeringUser Interfacesrdquo IEEE Software Vol 12 No 1 January1995 pp 64-73
[14] MM Moore ldquoRule-Based Detection for Reverse Engineer-ing User Interfacesrdquo Proc of 3rd Working Conf on ReverseEngineering WCRErsquo96 (Monterey 8-10 November 1996) L
Wills I Baxter E Chikofsky (eds) IEEE Computer SocietyPress Los Alamitos 1996 pp 42-48 Accessible athttpwwwccgatechedufacMelodyMoorepapersWCRE96ps
[15] MM Moore ldquoRepresentation Issues for Reengineering In-teractive Systemsrdquo ACM Computing Surveys Vol 28 No 4December 1996 Article 199 Accessible at httpwwwacmorgpubsarticlesjournalssurveys1996-28-4esa199-moorea199-moorehtml
[16] MM Moore and S Rugaber ldquoUsing Knowledge Repre-sentation to Understand Interactive Systemsrdquo Proc of theFifth International Workshop on Program ComprehensionIWPC97 (Dearborn 28-30 May 1997) IEEE Computer So-ciety Press Los Alamitos 1997 Accessible at httpwwwccgatechedufacMelodyMoorepapersWPC97ps
[17] MM Moore and S Rugaber ldquoDomain Analysis for Trans-formational Reuserdquo Proc of 4th Working Conf on ReverseEngineering WCRErsquo97 (6-8 October 1997) IEEE ComputerSociety Press Los Alamitos 1997
[18] ldquoOpen Interfacetraderdquo Neuron Data 156 University AvenuePalo Alto CA 94301 1992
[19] R Pausch M Conway and R DeLine ldquoLessons Learnedfrom SUIT the Simple User Interface Toolkitrdquo ACM Transon Office Information Systems Vol 10 No 4 October1992 pp 320-344 Accessible at httpwwwcsvirginiaedu~uigroupdocspublicationsSuitlessonspaper ps
[20] AR Puerta ldquoThe MECANO Project Comprehensive andIntegrated Support for Model-Based Interface DevelopmentrdquoProc of the 2nd Int W on Computer-Aided Design of UserInterfaces CADUIrsquo96 (Namur 5-7 June 1996) Presses Uni-versitaires de Namur Namur 1996 pp 19-36
[21] REK Stirewalt and S Rugaber ldquoAutomating UI Genera-tion by Model Compositionrdquo Journal of Automated Soft-ware Engineering Vol 7 No 2 1998 pp 101-124 Acces-sible at httpwwwccgatechedureverserepositorygenerps
[22] REK Stirewalt ldquoMDL A Language for Binding User-Interface Modelsrdquo in [26] pp 159-184
[23] P Szekely P Luo and R Neches ldquoBeyond Interface Build-ers Model-Based Interface Toolsrdquo Proc of ACM Conf onHuman Aspects in Computing Systems InterCHIrsquo93 ACMPress New York 1993 pp 383-390
[24] P Szekely ldquoRetrospective and Challenges for Model-BasedInterface Developmentrdquo Proc of 3rd Int Workshop on Com-puter-Aided Design of User Interfaces CADUIrsquo96 (Namur5-7 June 1996) Presses Universitaires de Namur Namur1996 pp xxi-xliv
[25] J Vanderdonckt and F Bodart ldquoEncapsulating Knowledgefor Intelligent Interaction Objects Selectionrdquo Proc of In-terCHIrsquo93 ACM Press New York 1993 pp 424-429
[26] J Vanderdonckt and P Berquin ldquoTowards a Very LargeModel-based Approach for User Interface DevelopmentrdquoProc of 1st Int Workshop on User Interfaces to Data Inten-sive Systems UIDISrsquo99 IEEE Computer Society Press LosAlamitos 1999 pp 76-85
[27] J Vanderdonckt and A Puerta (eds) Proc of the 3rd IntConf on Computer-Aided Design of User Interfaces CA-DUIrsquo99 Kluwer Academics Publishers Dordrecht 1999
[28] The XIML Consortium httpwwwximlorg 2002
Context with varying screen resolutionIt is a property of HTML that many HTML-compliantbrowsers on many types of screen can display it Althoughthis display is technically feasible (ie the browser adaptsthe presentation with respect to the platform) it may be un-usable to the end-user for one or another reason eg lossof information structure overcrowded screen too manyscroll bars image reduction) Therefore it makes sense toconsider how to re-engineer the UI for different screenresolutions This need is exacerbated by multiple HTMLbrowsers On different platforms equipped with different monitors
(ranging from workstation screen to small laptop) On different access devices with built-in browser (rang-
ing from the Internet ScreenPhone with 640x480 reso-lution to HTML-compatible PDA)
If the screen resolution is increasing the centered table withthree columns can be used to keep a presentation centeredon the screen with borders (fig 13) If the resolution is de-creasing the above redistributions are no longer applicableThree strategies have been explored so far
Fig 13 Presentation centered on the screen1 AIO replacement each AIO definition is augmented
with a list of potential replacements considered as eitherbehaviorally equivalent but consuming less space (ega list box to a drop-down list box) or degraded (eg anaccumulator to a multiple-value list box)
2 AIO size reduction typical height and length of AIOsare submitted to a possible reduction depending on us-ability constraints For example images and icons canbe reduced to a certain limit specified by a thresholdeg the minimally usable size of icons
3 AIO ungrouping when individual AIOs cannot be re-duced according to the two first strategies the distribu-tion of AIOs may be reconfigured AIOs can be groupedor ungrouped and new LWs can be created to contain alimited amount of AIOs
Again the development of these strategies bears further ex-planation which we hope to conduct in a later paper
Context with varying language and resolutionThis context is most likely to occur when a completely dif-ferent computing platform of access device is considered
For example a cellular phone only supports WML WebTVonly accepts a subset of HTML V3 and screen readersonly accept VoiceXML This context can be handled usinga combination of the approaches described in previouscontexts
RELATED WORKCross-platform development is not new as several environ-ments provide support for this purpose Galaxy [6] andOpen Interface [17] render the same UI on different plat-forms with their native look amp feel while SUIT [18] em-ploys a unique UI definition that can be processed on mul-tiple platforms However not one of these systems trulyadopts a model-based approach although SUITs commondefinition holds some presentation abstractions CT-UIMS[9] pioneered the platform model by supporting some AIO[15] redistribution for OSFMotif large screen and smallMacintosh screens AUIDL [101112] is probably the firstset of abstractions for reverse and re-engineering UIs frominternal hierarchical structures type and variable declara-tions a UI can be recovered in IDL with a presentationmodel (based on OO paradigm) and a dialog model (basedon Milnerrsquos process algebra) MORPH exploits productionrules [13] to infer AIOs [14] from CIOs and thus producesa graphical UI from a textual UI [15] This transformationalapproach is similar in principle to ours Forward engineer-ing can be executed by model composition [20] binding[21] or derivation thus proving that the approach is feasi-ble MORALE [1] is an extensive set of techniques andtools for reverse engineering legacy systems rather thanweb pages CELLEST [7] reverse engineers similar sys-tems but into DHTML thus adopting active models ratherthan passive models
CONCLUSIONThis paper has introduced a model-based approach sup-porting both reverse and re-engineering of web pages Thearchitecture outlined in fig 14 assumes that all models arestored in a model repository each model being specifiedwith the eXtensible Interface Markup Language (XIML)promoted by the XIML Consortium [28] This model tex-tual specification is declarative analyzable and editable[22]
HTMLpage
VAQUITA Web pagereverse engineering
User interfaceRe-engineering
XIMLPresentation
model
HTMLpage
WMLdeck
hellip
Model editor validator
Fig 14 Model-based approach architecture
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and redocumentation
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Platformmodel(1) (2)
Fig 15 UI Redocumentation
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
User interface reverse engineering
Taskmodel
Domainmodel
Usermodel
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
Application reverse engineering
Use
r ta
sk a
nd d
omai
nre
vers
e en
gine
erin
g
Log filesPredictive modelsDesign heuristics
Usability guidelinesFig 16 UI Design recovery
This ongoing work is just at the beginning as it will requirea significant amount of work to create a comprehensive en-vironment with tool-support for model-based reengineeringof web sites The described approach inevitably suffersfrom many restrictions and lack of support such as Non-standard HTML elements (eg embedded objects
browser-specific tags) or languages (eg JavaScriptfunctions ActiveX controls) are ignored as their supportwould require expensive development efforts
Dynamically created web pages are not supported sincetheir HTML code is generated on the fly The callbackroutines attached to AIOs producing dynamic pages canbe replaced by direct calls to the semantic core Activemodels are in this case too expensive to manage re-engineering on the fly However we can imagine to par-tially support applications that use explicit template-based page generation (eg XSLT JSP) in this case thestatic analysis should be able to differentiate variableparts from run-time provided parts
Style sheets are unsupported in the outlined algorithm asare other relevant presentation aspects For instancegraphical images designed as tabs (eg on www ama-zoncom) or links placed on top of such graphics are notrecognized and therefore cannot be mapped onto severalLWs of a same PU
On the other hand the environment we have described mayenvision other forms of UI reverse engineering [3] as repre-sented in table 1
Redocumentation (fig 15) this flow is similar to re-engineering except that the platform model is no longerneeded as it remains constant over time
Restructuring (fig 15 with arrow (1) used only) the UIremains basically the same for semantics and dialog butthe presentation model changes The scope of the plat-form model is restricted to re-engineering
Redesigning (fig 15 with arrows (1) and (2) used) theUI remains identical regarding its semantics but both thedialog (eg other interaction styles techniques or gen-res) and the presentation (eg any redistribution) modelscan change due to platform constraints This may consistsin a new category of UI re-engineering
Design recovery (fig 16) in this process it is expectedthat task and domain models could be recovered Guess-ing these models from HTML code seems impracticalbut other information sources might be investigated suchas log files produced by user traffic comparison withpredictive models design heuristics to identify usagepatterns and automated usability analysis based onguidelines
REFERENCES[1] Barclay PJ Griffiths T Mc Kirdy J Paton NW Coo-
per R Kennedy J ldquoThe Teallach Tool Using Models forFlexible User Interface Designrdquo Proc of 3rd Int Conf onComputer-Aided Design of User Interfaces CADUIrsquo99 (Lou-vain-la-Neuve 21-23 October 1999) Kluwer AcademicsDordrecht (1999) 139ndash158 Accessible at httpimgcsmanacukteallachpublicationsCadui99CADUI99ps
[2] G Abowd A Goel DF Jerding M McCracken MM
Moore JW Murdock C Potts S Rugaber and L WillsldquoMORALEndashMission Oriented Architectural Legacy Evolu-tionrdquo Proc of Int Conf on Software Maintenance (1997)
[3] F Bodart A-M Hennebert J-M Leheureux and JVanderdonckt ldquoComputer-Aided Window Identification inTRIDENTrdquo Proc of the 5th IFIP TC13 Conf on Human-Computer Interaction Interactrsquo95 (Lillehammer 25-29 June1995) Chapman amp Hall London 1995 pp 331-336 Acces-sible at httpwwwinfofundpacbecgi-publipub-spec-paperRP-95-021
[4] EJ Chikofsky and JH Cross ldquoReverse Engineering andDesign Recovery A Taxonomyrdquo IEEE Software Vol 1 No7 January 1990 pp 13-17
[5] J-M De Baud and S Rugaber ldquoA Software Re-engineeringMethod Using Domain Modelsrdquo Proc of Int Conf on Soft-ware Maintenance (October 1995) pp 204-213
[6] J Eisenstein and A Puerta ldquoAdaptation in Automated User-Interface Designrdquo Proc of ACM Int Conf on IntelligentUser Interfaces IUIrsquo2000 (New Orleans 9-12 January 2000)ACM Press New York 2000 pp 74-81
[8] L Kong E Strouila B Matichuk ldquoLegacy Interface Migra-tion A Task-Centered Approachrdquo Proc of 8th Int Conf onHuman-Computer Interaction HCI Internationalrsquo99 (Mu-nich 22-27 August 1999) H-J Bullinger and J Ziegler(eds) Lawrence Erlbaum Associates MahwahLondon1999 pp 1167-1171 Accessible at httpwwwcsualbertaca~strouliaPapershci99ps
[9] F Lonczewski and S Schreiber ldquoThe FUSE-System an In-tegrated User Interface Design Environmentrdquo Proc of 2nd
Int Workshop on Coputer-Aided Design of User InterfacesCADUIrsquo96 (Namur 5-7 June 1996) Presses Universitairesde Namur Namur 1996 pp 37-56 Accessible at ftphpeick7informatiktu-muenchendepubpaperssisfuse_cadui96psgz
[10] Ch Maumlrtin ldquoA UIMS for Knowledge Based Interface Tem-plate Generation and Interactionrdquo Proc of Interactrsquo90 El-sevier Science Pub Amsterdam 1990 pp 651-657
[11] E Merlo JF Girard K Kontogiannis P Panangaden andR De Mori ldquoReverse Engineering of User Interfacesrdquo Procof 1st Working Conference on Reverse EngineeringWCRErsquo93 (Baltimore 21-23 May 1993) RC Waters EJChikofsky (eds) IEEE Computer Society Press LosAlamitos 1993 pp 171-179
[12] E Merlo P-Y Gagneacute and A Thiboutocirct ldquoInference ofgraphical AUIDL specifications for the reverse engineeringof user interfacesrdquo Proc of Int Conf on Software Mainte-nance (19-23 September 1994) IEEE Computer SocietyPress Los Alamitos 1994 pp 80-88
[13] E Merlo P-Y Gagneacute J-F Girard K Kontogiannis LHendren P Panagaden and R De Mori ldquoReengineeringUser Interfacesrdquo IEEE Software Vol 12 No 1 January1995 pp 64-73
[14] MM Moore ldquoRule-Based Detection for Reverse Engineer-ing User Interfacesrdquo Proc of 3rd Working Conf on ReverseEngineering WCRErsquo96 (Monterey 8-10 November 1996) L
Wills I Baxter E Chikofsky (eds) IEEE Computer SocietyPress Los Alamitos 1996 pp 42-48 Accessible athttpwwwccgatechedufacMelodyMoorepapersWCRE96ps
[15] MM Moore ldquoRepresentation Issues for Reengineering In-teractive Systemsrdquo ACM Computing Surveys Vol 28 No 4December 1996 Article 199 Accessible at httpwwwacmorgpubsarticlesjournalssurveys1996-28-4esa199-moorea199-moorehtml
[16] MM Moore and S Rugaber ldquoUsing Knowledge Repre-sentation to Understand Interactive Systemsrdquo Proc of theFifth International Workshop on Program ComprehensionIWPC97 (Dearborn 28-30 May 1997) IEEE Computer So-ciety Press Los Alamitos 1997 Accessible at httpwwwccgatechedufacMelodyMoorepapersWPC97ps
[17] MM Moore and S Rugaber ldquoDomain Analysis for Trans-formational Reuserdquo Proc of 4th Working Conf on ReverseEngineering WCRErsquo97 (6-8 October 1997) IEEE ComputerSociety Press Los Alamitos 1997
[18] ldquoOpen Interfacetraderdquo Neuron Data 156 University AvenuePalo Alto CA 94301 1992
[19] R Pausch M Conway and R DeLine ldquoLessons Learnedfrom SUIT the Simple User Interface Toolkitrdquo ACM Transon Office Information Systems Vol 10 No 4 October1992 pp 320-344 Accessible at httpwwwcsvirginiaedu~uigroupdocspublicationsSuitlessonspaper ps
[20] AR Puerta ldquoThe MECANO Project Comprehensive andIntegrated Support for Model-Based Interface DevelopmentrdquoProc of the 2nd Int W on Computer-Aided Design of UserInterfaces CADUIrsquo96 (Namur 5-7 June 1996) Presses Uni-versitaires de Namur Namur 1996 pp 19-36
[21] REK Stirewalt and S Rugaber ldquoAutomating UI Genera-tion by Model Compositionrdquo Journal of Automated Soft-ware Engineering Vol 7 No 2 1998 pp 101-124 Acces-sible at httpwwwccgatechedureverserepositorygenerps
[22] REK Stirewalt ldquoMDL A Language for Binding User-Interface Modelsrdquo in [26] pp 159-184
[23] P Szekely P Luo and R Neches ldquoBeyond Interface Build-ers Model-Based Interface Toolsrdquo Proc of ACM Conf onHuman Aspects in Computing Systems InterCHIrsquo93 ACMPress New York 1993 pp 383-390
[24] P Szekely ldquoRetrospective and Challenges for Model-BasedInterface Developmentrdquo Proc of 3rd Int Workshop on Com-puter-Aided Design of User Interfaces CADUIrsquo96 (Namur5-7 June 1996) Presses Universitaires de Namur Namur1996 pp xxi-xliv
[25] J Vanderdonckt and F Bodart ldquoEncapsulating Knowledgefor Intelligent Interaction Objects Selectionrdquo Proc of In-terCHIrsquo93 ACM Press New York 1993 pp 424-429
[26] J Vanderdonckt and P Berquin ldquoTowards a Very LargeModel-based Approach for User Interface DevelopmentrdquoProc of 1st Int Workshop on User Interfaces to Data Inten-sive Systems UIDISrsquo99 IEEE Computer Society Press LosAlamitos 1999 pp 76-85
[27] J Vanderdonckt and A Puerta (eds) Proc of the 3rd IntConf on Computer-Aided Design of User Interfaces CA-DUIrsquo99 Kluwer Academics Publishers Dordrecht 1999
[28] The XIML Consortium httpwwwximlorg 2002
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
Presentationmodel
Dialogmodel
User interface reverse engineering and redocumentation
New UIcode N
ew U
I Int
egra
tion
New interactiveapplication
Platformmodel(1) (2)
Fig 15 UI Redocumentation
Interactiveapplication
Semanticcore code
UI E
xtra
ctio
n
User inter-face code
User interface reverse engineering
Taskmodel
Domainmodel
Usermodel
Applicationmodel
Datamodel
Semantic corecomponent
Presentationmodel
Dialogmodel
Application reverse engineering
Use
r ta
sk a
nd d
omai
nre
vers
e en
gine
erin
g
Log filesPredictive modelsDesign heuristics
Usability guidelinesFig 16 UI Design recovery
This ongoing work is just at the beginning as it will requirea significant amount of work to create a comprehensive en-vironment with tool-support for model-based reengineeringof web sites The described approach inevitably suffersfrom many restrictions and lack of support such as Non-standard HTML elements (eg embedded objects
browser-specific tags) or languages (eg JavaScriptfunctions ActiveX controls) are ignored as their supportwould require expensive development efforts
Dynamically created web pages are not supported sincetheir HTML code is generated on the fly The callbackroutines attached to AIOs producing dynamic pages canbe replaced by direct calls to the semantic core Activemodels are in this case too expensive to manage re-engineering on the fly However we can imagine to par-tially support applications that use explicit template-based page generation (eg XSLT JSP) in this case thestatic analysis should be able to differentiate variableparts from run-time provided parts
Style sheets are unsupported in the outlined algorithm asare other relevant presentation aspects For instancegraphical images designed as tabs (eg on www ama-zoncom) or links placed on top of such graphics are notrecognized and therefore cannot be mapped onto severalLWs of a same PU
On the other hand the environment we have described mayenvision other forms of UI reverse engineering [3] as repre-sented in table 1
Redocumentation (fig 15) this flow is similar to re-engineering except that the platform model is no longerneeded as it remains constant over time
Restructuring (fig 15 with arrow (1) used only) the UIremains basically the same for semantics and dialog butthe presentation model changes The scope of the plat-form model is restricted to re-engineering
Redesigning (fig 15 with arrows (1) and (2) used) theUI remains identical regarding its semantics but both thedialog (eg other interaction styles techniques or gen-res) and the presentation (eg any redistribution) modelscan change due to platform constraints This may consistsin a new category of UI re-engineering
Design recovery (fig 16) in this process it is expectedthat task and domain models could be recovered Guess-ing these models from HTML code seems impracticalbut other information sources might be investigated suchas log files produced by user traffic comparison withpredictive models design heuristics to identify usagepatterns and automated usability analysis based onguidelines
REFERENCES[1] Barclay PJ Griffiths T Mc Kirdy J Paton NW Coo-
per R Kennedy J ldquoThe Teallach Tool Using Models forFlexible User Interface Designrdquo Proc of 3rd Int Conf onComputer-Aided Design of User Interfaces CADUIrsquo99 (Lou-vain-la-Neuve 21-23 October 1999) Kluwer AcademicsDordrecht (1999) 139ndash158 Accessible at httpimgcsmanacukteallachpublicationsCadui99CADUI99ps
[2] G Abowd A Goel DF Jerding M McCracken MM
Moore JW Murdock C Potts S Rugaber and L WillsldquoMORALEndashMission Oriented Architectural Legacy Evolu-tionrdquo Proc of Int Conf on Software Maintenance (1997)
[3] F Bodart A-M Hennebert J-M Leheureux and JVanderdonckt ldquoComputer-Aided Window Identification inTRIDENTrdquo Proc of the 5th IFIP TC13 Conf on Human-Computer Interaction Interactrsquo95 (Lillehammer 25-29 June1995) Chapman amp Hall London 1995 pp 331-336 Acces-sible at httpwwwinfofundpacbecgi-publipub-spec-paperRP-95-021
[4] EJ Chikofsky and JH Cross ldquoReverse Engineering andDesign Recovery A Taxonomyrdquo IEEE Software Vol 1 No7 January 1990 pp 13-17
[5] J-M De Baud and S Rugaber ldquoA Software Re-engineeringMethod Using Domain Modelsrdquo Proc of Int Conf on Soft-ware Maintenance (October 1995) pp 204-213
[6] J Eisenstein and A Puerta ldquoAdaptation in Automated User-Interface Designrdquo Proc of ACM Int Conf on IntelligentUser Interfaces IUIrsquo2000 (New Orleans 9-12 January 2000)ACM Press New York 2000 pp 74-81
[8] L Kong E Strouila B Matichuk ldquoLegacy Interface Migra-tion A Task-Centered Approachrdquo Proc of 8th Int Conf onHuman-Computer Interaction HCI Internationalrsquo99 (Mu-nich 22-27 August 1999) H-J Bullinger and J Ziegler(eds) Lawrence Erlbaum Associates MahwahLondon1999 pp 1167-1171 Accessible at httpwwwcsualbertaca~strouliaPapershci99ps
[9] F Lonczewski and S Schreiber ldquoThe FUSE-System an In-tegrated User Interface Design Environmentrdquo Proc of 2nd
Int Workshop on Coputer-Aided Design of User InterfacesCADUIrsquo96 (Namur 5-7 June 1996) Presses Universitairesde Namur Namur 1996 pp 37-56 Accessible at ftphpeick7informatiktu-muenchendepubpaperssisfuse_cadui96psgz
[10] Ch Maumlrtin ldquoA UIMS for Knowledge Based Interface Tem-plate Generation and Interactionrdquo Proc of Interactrsquo90 El-sevier Science Pub Amsterdam 1990 pp 651-657
[11] E Merlo JF Girard K Kontogiannis P Panangaden andR De Mori ldquoReverse Engineering of User Interfacesrdquo Procof 1st Working Conference on Reverse EngineeringWCRErsquo93 (Baltimore 21-23 May 1993) RC Waters EJChikofsky (eds) IEEE Computer Society Press LosAlamitos 1993 pp 171-179
[12] E Merlo P-Y Gagneacute and A Thiboutocirct ldquoInference ofgraphical AUIDL specifications for the reverse engineeringof user interfacesrdquo Proc of Int Conf on Software Mainte-nance (19-23 September 1994) IEEE Computer SocietyPress Los Alamitos 1994 pp 80-88
[13] E Merlo P-Y Gagneacute J-F Girard K Kontogiannis LHendren P Panagaden and R De Mori ldquoReengineeringUser Interfacesrdquo IEEE Software Vol 12 No 1 January1995 pp 64-73
[14] MM Moore ldquoRule-Based Detection for Reverse Engineer-ing User Interfacesrdquo Proc of 3rd Working Conf on ReverseEngineering WCRErsquo96 (Monterey 8-10 November 1996) L
Wills I Baxter E Chikofsky (eds) IEEE Computer SocietyPress Los Alamitos 1996 pp 42-48 Accessible athttpwwwccgatechedufacMelodyMoorepapersWCRE96ps
[15] MM Moore ldquoRepresentation Issues for Reengineering In-teractive Systemsrdquo ACM Computing Surveys Vol 28 No 4December 1996 Article 199 Accessible at httpwwwacmorgpubsarticlesjournalssurveys1996-28-4esa199-moorea199-moorehtml
[16] MM Moore and S Rugaber ldquoUsing Knowledge Repre-sentation to Understand Interactive Systemsrdquo Proc of theFifth International Workshop on Program ComprehensionIWPC97 (Dearborn 28-30 May 1997) IEEE Computer So-ciety Press Los Alamitos 1997 Accessible at httpwwwccgatechedufacMelodyMoorepapersWPC97ps
[17] MM Moore and S Rugaber ldquoDomain Analysis for Trans-formational Reuserdquo Proc of 4th Working Conf on ReverseEngineering WCRErsquo97 (6-8 October 1997) IEEE ComputerSociety Press Los Alamitos 1997
[18] ldquoOpen Interfacetraderdquo Neuron Data 156 University AvenuePalo Alto CA 94301 1992
[19] R Pausch M Conway and R DeLine ldquoLessons Learnedfrom SUIT the Simple User Interface Toolkitrdquo ACM Transon Office Information Systems Vol 10 No 4 October1992 pp 320-344 Accessible at httpwwwcsvirginiaedu~uigroupdocspublicationsSuitlessonspaper ps
[20] AR Puerta ldquoThe MECANO Project Comprehensive andIntegrated Support for Model-Based Interface DevelopmentrdquoProc of the 2nd Int W on Computer-Aided Design of UserInterfaces CADUIrsquo96 (Namur 5-7 June 1996) Presses Uni-versitaires de Namur Namur 1996 pp 19-36
[21] REK Stirewalt and S Rugaber ldquoAutomating UI Genera-tion by Model Compositionrdquo Journal of Automated Soft-ware Engineering Vol 7 No 2 1998 pp 101-124 Acces-sible at httpwwwccgatechedureverserepositorygenerps
[22] REK Stirewalt ldquoMDL A Language for Binding User-Interface Modelsrdquo in [26] pp 159-184
[23] P Szekely P Luo and R Neches ldquoBeyond Interface Build-ers Model-Based Interface Toolsrdquo Proc of ACM Conf onHuman Aspects in Computing Systems InterCHIrsquo93 ACMPress New York 1993 pp 383-390
[24] P Szekely ldquoRetrospective and Challenges for Model-BasedInterface Developmentrdquo Proc of 3rd Int Workshop on Com-puter-Aided Design of User Interfaces CADUIrsquo96 (Namur5-7 June 1996) Presses Universitaires de Namur Namur1996 pp xxi-xliv
[25] J Vanderdonckt and F Bodart ldquoEncapsulating Knowledgefor Intelligent Interaction Objects Selectionrdquo Proc of In-terCHIrsquo93 ACM Press New York 1993 pp 424-429
[26] J Vanderdonckt and P Berquin ldquoTowards a Very LargeModel-based Approach for User Interface DevelopmentrdquoProc of 1st Int Workshop on User Interfaces to Data Inten-sive Systems UIDISrsquo99 IEEE Computer Society Press LosAlamitos 1999 pp 76-85
[27] J Vanderdonckt and A Puerta (eds) Proc of the 3rd IntConf on Computer-Aided Design of User Interfaces CA-DUIrsquo99 Kluwer Academics Publishers Dordrecht 1999
[28] The XIML Consortium httpwwwximlorg 2002
Moore JW Murdock C Potts S Rugaber and L WillsldquoMORALEndashMission Oriented Architectural Legacy Evolu-tionrdquo Proc of Int Conf on Software Maintenance (1997)
[3] F Bodart A-M Hennebert J-M Leheureux and JVanderdonckt ldquoComputer-Aided Window Identification inTRIDENTrdquo Proc of the 5th IFIP TC13 Conf on Human-Computer Interaction Interactrsquo95 (Lillehammer 25-29 June1995) Chapman amp Hall London 1995 pp 331-336 Acces-sible at httpwwwinfofundpacbecgi-publipub-spec-paperRP-95-021
[4] EJ Chikofsky and JH Cross ldquoReverse Engineering andDesign Recovery A Taxonomyrdquo IEEE Software Vol 1 No7 January 1990 pp 13-17
[5] J-M De Baud and S Rugaber ldquoA Software Re-engineeringMethod Using Domain Modelsrdquo Proc of Int Conf on Soft-ware Maintenance (October 1995) pp 204-213
[6] J Eisenstein and A Puerta ldquoAdaptation in Automated User-Interface Designrdquo Proc of ACM Int Conf on IntelligentUser Interfaces IUIrsquo2000 (New Orleans 9-12 January 2000)ACM Press New York 2000 pp 74-81
[8] L Kong E Strouila B Matichuk ldquoLegacy Interface Migra-tion A Task-Centered Approachrdquo Proc of 8th Int Conf onHuman-Computer Interaction HCI Internationalrsquo99 (Mu-nich 22-27 August 1999) H-J Bullinger and J Ziegler(eds) Lawrence Erlbaum Associates MahwahLondon1999 pp 1167-1171 Accessible at httpwwwcsualbertaca~strouliaPapershci99ps
[9] F Lonczewski and S Schreiber ldquoThe FUSE-System an In-tegrated User Interface Design Environmentrdquo Proc of 2nd
Int Workshop on Coputer-Aided Design of User InterfacesCADUIrsquo96 (Namur 5-7 June 1996) Presses Universitairesde Namur Namur 1996 pp 37-56 Accessible at ftphpeick7informatiktu-muenchendepubpaperssisfuse_cadui96psgz
[10] Ch Maumlrtin ldquoA UIMS for Knowledge Based Interface Tem-plate Generation and Interactionrdquo Proc of Interactrsquo90 El-sevier Science Pub Amsterdam 1990 pp 651-657
[11] E Merlo JF Girard K Kontogiannis P Panangaden andR De Mori ldquoReverse Engineering of User Interfacesrdquo Procof 1st Working Conference on Reverse EngineeringWCRErsquo93 (Baltimore 21-23 May 1993) RC Waters EJChikofsky (eds) IEEE Computer Society Press LosAlamitos 1993 pp 171-179
[12] E Merlo P-Y Gagneacute and A Thiboutocirct ldquoInference ofgraphical AUIDL specifications for the reverse engineeringof user interfacesrdquo Proc of Int Conf on Software Mainte-nance (19-23 September 1994) IEEE Computer SocietyPress Los Alamitos 1994 pp 80-88
[13] E Merlo P-Y Gagneacute J-F Girard K Kontogiannis LHendren P Panagaden and R De Mori ldquoReengineeringUser Interfacesrdquo IEEE Software Vol 12 No 1 January1995 pp 64-73
[14] MM Moore ldquoRule-Based Detection for Reverse Engineer-ing User Interfacesrdquo Proc of 3rd Working Conf on ReverseEngineering WCRErsquo96 (Monterey 8-10 November 1996) L
Wills I Baxter E Chikofsky (eds) IEEE Computer SocietyPress Los Alamitos 1996 pp 42-48 Accessible athttpwwwccgatechedufacMelodyMoorepapersWCRE96ps
[15] MM Moore ldquoRepresentation Issues for Reengineering In-teractive Systemsrdquo ACM Computing Surveys Vol 28 No 4December 1996 Article 199 Accessible at httpwwwacmorgpubsarticlesjournalssurveys1996-28-4esa199-moorea199-moorehtml
[16] MM Moore and S Rugaber ldquoUsing Knowledge Repre-sentation to Understand Interactive Systemsrdquo Proc of theFifth International Workshop on Program ComprehensionIWPC97 (Dearborn 28-30 May 1997) IEEE Computer So-ciety Press Los Alamitos 1997 Accessible at httpwwwccgatechedufacMelodyMoorepapersWPC97ps
[17] MM Moore and S Rugaber ldquoDomain Analysis for Trans-formational Reuserdquo Proc of 4th Working Conf on ReverseEngineering WCRErsquo97 (6-8 October 1997) IEEE ComputerSociety Press Los Alamitos 1997
[18] ldquoOpen Interfacetraderdquo Neuron Data 156 University AvenuePalo Alto CA 94301 1992
[19] R Pausch M Conway and R DeLine ldquoLessons Learnedfrom SUIT the Simple User Interface Toolkitrdquo ACM Transon Office Information Systems Vol 10 No 4 October1992 pp 320-344 Accessible at httpwwwcsvirginiaedu~uigroupdocspublicationsSuitlessonspaper ps
[20] AR Puerta ldquoThe MECANO Project Comprehensive andIntegrated Support for Model-Based Interface DevelopmentrdquoProc of the 2nd Int W on Computer-Aided Design of UserInterfaces CADUIrsquo96 (Namur 5-7 June 1996) Presses Uni-versitaires de Namur Namur 1996 pp 19-36
[21] REK Stirewalt and S Rugaber ldquoAutomating UI Genera-tion by Model Compositionrdquo Journal of Automated Soft-ware Engineering Vol 7 No 2 1998 pp 101-124 Acces-sible at httpwwwccgatechedureverserepositorygenerps
[22] REK Stirewalt ldquoMDL A Language for Binding User-Interface Modelsrdquo in [26] pp 159-184
[23] P Szekely P Luo and R Neches ldquoBeyond Interface Build-ers Model-Based Interface Toolsrdquo Proc of ACM Conf onHuman Aspects in Computing Systems InterCHIrsquo93 ACMPress New York 1993 pp 383-390
[24] P Szekely ldquoRetrospective and Challenges for Model-BasedInterface Developmentrdquo Proc of 3rd Int Workshop on Com-puter-Aided Design of User Interfaces CADUIrsquo96 (Namur5-7 June 1996) Presses Universitaires de Namur Namur1996 pp xxi-xliv
[25] J Vanderdonckt and F Bodart ldquoEncapsulating Knowledgefor Intelligent Interaction Objects Selectionrdquo Proc of In-terCHIrsquo93 ACM Press New York 1993 pp 424-429
[26] J Vanderdonckt and P Berquin ldquoTowards a Very LargeModel-based Approach for User Interface DevelopmentrdquoProc of 1st Int Workshop on User Interfaces to Data Inten-sive Systems UIDISrsquo99 IEEE Computer Society Press LosAlamitos 1999 pp 76-85
[27] J Vanderdonckt and A Puerta (eds) Proc of the 3rd IntConf on Computer-Aided Design of User Interfaces CA-DUIrsquo99 Kluwer Academics Publishers Dordrecht 1999
