Storing Efficiently Bioinformatics Workflows

Michel Kinsy and Zoe LacroixArizona State University

PO Box 875706 Tempe AZ 85287-5706, USA

AbstractWe propose an efficient storage strategy to record

bioinformatics workflows. Our approach presents sci-enttists with a flexible design model that distinguishesthe scientific aim as a design protocol expressed againstan ontology from the implementations(s), scientificworkflows composed of bioirnformatics services, andtheir execution. The storage strategy presented in thispaper allows efficient access and constitutes the frame-work for reasoning on scientific protocols and experi-mental data.

1 IntroductionThe field of bioinformatics has grown immensely in

the last 20 years; this growth is due to the intenseresearch efforts invested in analyzing protein struc-tures and sequencing of genes. Those efforts have pro-duced vast amounts of data and raised different storageand query challenges not typically encountered withbusiness data. Various works have been done to ad-dress the functionality and the capacity of scientificdatabases [7]. These databases and repositories typ-ically do not record the processes producing the datathey are storing, leaving scientists with no access to in-formation relevant to data provenance needed to eval-uate the quality of the results produced at execution.Moreover, complete protocol information including thedescription of the process and the selection of the re-sources used to implement it is critical to reproducethe experiment thus validates the scientific soundnessof the results. In the recent years different softwareprojects have been initiated to assist scientists to de-sign, implement, and store their workflows and the datacollected by the execution of these workflows [12, 11, 2].But these systems in their attempts to be general scien-tific workflow design tools, have failed to provide pro-tocol storage and to optimize the characteristics mostneeded and useful in bioinformatics workflow design,implementation, storage, and querying.

ProtocolDB is designed to provide scientists witha software tool allowing them to design and manage

their workflows exploiting a domain consistent with thesemantics of the experiments, store, and query bothworkflow design and implementation as well as the datacollected at workflow execution. In ProtocolDB, scien-tific protocols record both the scientific aim of each taskand the description of its implementation. To offer flex-ibility, we decompose each scientific protocol into twocomponents: design and implementation. Both the de-sign and the implementation of a scientific protocol arecomposed of coordinated tasks. Each task of the proto-col design is defined by its input, output, and descrip-tion. When an ontology is available to describe thescientific objects and tasks involved in a scientific pro-tocol, the input and output of each protocol design taskare defined by their respective concept classes. Thedescription of the task may be a relationship definedbetween the input and output conceptual classes or adescription of a relationship not defined in the ontology.The protocol implementation describes the selection ofresources used to implement each task of the protocol.The description of an implementation task is the appli-cation, or service (e.g., Web service [1], BioMOBY [13])selected to implement the corresponding design task.The input (respectively output) of a protocol imple-mentation task is the description of the data collectioninput (respectively output) format of the service. In-put data are instances of the input conceptual class ofthe corresponding design task. A similar distinction ofthe scientific aim from its implementation was noted in[5] with workflow conceptualization and specification,and in [3] with abstract and concrete workflows.

2 Motivating ExampleAlternative Splicing (AS) is a splicing process of a

pre-mRNA sequence transcribed from one gene thatleads to different mature mRNA molecules thus to dif-ferent functional proteins. Alternative splicing eventsare produced by different arrangements of the exons ofa given gene. The Alternative Splicing Protocol (ASP)we present in this section is currently supporting theBiolnformatics Pipeline Alternative Splicing Services

1-4244-1509-8/07/$25.00 02007 IEEE 1328

BIPASS [9].The Alternative Splicing Protocol (ASP) takes a set

of transcripts as input and returns clusters of tran-scripts aligned to a gene. The process of alignmentconsists of an alignment of each transcript sequenceagainst each genomic sequence of a whole genome ofone or more organisms. This step is executed withall known transcripts extracted from different publicdatabases. A clustering step immediately follows thealignment step. That step allows delimiting the tran-script region of a gene excluding its regulation region.A cluster normally represents or may be representativeof all intermediate transcripts (from the pre-messenger-RNA(s) to the mature messenger-RNA(s)) required toobtain one or several functional translated proteinsfrom the same gene.A single protocol design step such as the alignment

step of ASP may be mapped to a complex implemen-tation protocol. The motivations for such a compleximplementation protocol are detailed and analyzed in[8]. The protocol implementation corresponding to thealignment task of ASP is composed of seven tasks il-lustrated in Figure 1.

-..1 Si ~~~~~~~~~~~~siAT21/Vs251 A1S1s2 F I 1Fat iS'2~

_2 _2

s2

Figure 1. Initial Design Task Decomposition

First the input sequences are retrieved from lo-cal or public data sources. Transcript Data Fil-tering/Queries (TAFQ) and Genomic Data Filter-ing/Queries (GAFQ) denote queries against local orexternal databases to retrieve the input sequence queryand the sequence database against which it is aligned.The input sequences are submitted to BLAT, an align-ment tool that eliminates roughly all sequences notlikely to produce a fine alignment with the input.Si and S2 denote the Transcript and Genome dataflows respectively, and BLAT is selected to implementAT1. Once the first alignment step is performed,the first 10% of the output are selected (F). Thenthe aligned transcripts are extracted from the previoussteps (branch 1) and the aligned genomic sequences areextracted from the previous step (branch 2). Fat de-notes the filtering function that removes all transcriptsaligned but failed under a given threshold percentagevalue. EGS extracts the exact part of the matchinggenomic sequences and adds a defined number of basesin upstream and downstream of the extracted genomicsequence. The resulting transcript sequences Si andgenomic sequences S2 are submitted to a second align-

ment tool that performs a fine alignment step. AT2 isimplemented with SIM4 while S1'/S2' represents thefinal output of the whole alignment.

3 Protocol Model Analysis

Scientific workflows expressed in workflow systemsuch as Taverna [12], Kepler [11], or SemanticBio [10]are typically driven by interoperability. They are com-posed of bioinformatics services that are connected se-quentially or in parallel for execution. These systemsdo not offer a protocol analysis that could generatean equivalent protocol that could be more efficient forstorage purposes, execution, or data provenance anal-ysis. In this section we analyze scientific protocols andshow that Petri nets offer a valuable model to representand optimize scientific protocols.An equivalent representation of the protocol shown

in Figure 1 with a Finite State Machine (FSM) with aSource state and a Sink state (introduced for complete-ness) is shown in Figure 2. The FSM representation ofthe protocol may easily be converted into a Petri netby representing each state or edge of the FSM by afiring transition or process.

Figure 2. FSM of Design Task

The Petri net obtained from the transformation ofthe above automaton is shown in Figure 3. Such rep-resentation is an abstraction of the dataflow that cap-tures the internal structure resulting from the sequen-tial or parallel connectors used to define the protocol.

Figure 3. Petri Net of Design Task

The incidence matrix of the protocol is defined byA = (aiu), where aiu= e(ti, pi) - e(pj, tj), and whereti is a transition (e.g., AT1), pi is a process (e.g., p4),and e(n, m) is the number of incoming edges from n tom.

1-4244-1509-8/07/$25.00 02007 IEEE 1329

Example The incidence matrix A of the alignmentprotocol illustrated in Figure 2 is computed as follows.Each column corresponds to a process po, ,Pio andeach raw corresponds to a workflow transition Source,TDFQ, GDFQ, AT1, F, Fat, EGS, AT2, and Sink.The incidence matrix obtained from the Petri net ofthe design task decomposition is shown in Figure 4.

( 11

0000000

10-1000000

010-100000

001-100000

0001-10000

001000-100

00001-1000

0000010-10

000001-100

0000001-10

00000001-1

Figure 4. ASP incidence matrix

The Petri net simulates the workflow as follows. Theinitial state is represented by a vector zO and each firingvector identifies the transition fired in the Petri net.Given a state 'i and a firing vector v'i, i+1 = i +viA.

Example Firing the transition Fat corresponds to thefiring vector [0,0,0,0,0,1,0,0,0]. From the initial statez0 A[00,0,°,0,0, ,0, 0, 00 0], the successive applica-tion of firing rules generates the covering tree of theworkflow illustrated in Figure 5. The coverability treeis the set of all states of the system reachable from theinitial state and thus represents all valid implementa-tions that match the design task. There are variousalgorithms proposed to efficiently and automaticallydetermine the covering tree [4, 6].

The approach allows the representation (thus thestorage) of complex implementation workflows in theterms of a simple mathematical model which conservessemantics and use syntaxes that can be traced back tothe original design. Furthermore, although the graphonly represents data dependencies, the model is robustenough to support other types of dependencies. For ex-ample, in addition to the data dependencies we couldassociate with each task a time constraint as follows.For each task T, there is a couple (te, ti) where te repre-sents the earliest time the task can be executed and t, isthe latest time the task can be executed to preserve theover dependencies in the workflow. The time constraintcapacity of the model allows us to not only optimize adesign in terms of steps but also in term of schedulingfor query planning. Other paramaters may be asso-ciated to tasks to capture various measures relevantto workflow execution. Such analysis so far has beengreatly ignored by other bioinformatics workflow data

Initial State xb [o,o,o,o,°o°oo,oopq]

Source

[I 1 7°01°1°n°>°l°ol°ld

X[X,1, 11oo1o1,1oXoXoI [1 I,I0 1 1,o1 XoTDFO DFO

GDFQ TDFQ

[ao,i 0oio0oo0,0]

AT1

101OOsO'1J01X1O1Q0OI10]F

[0 0,0 0s'0"1's1 s°s°1°0°]

Fat

[0o 0,0 O,O' ,I,O Ii 0,0]ES

AT2

[0'~0'0s° °O'O 0'0011]s nk

Initial State xo [o0o,ooo0o0oo0o0o0o]

Figure 5. Coverability Tree of the Petri Net

analysis and storage approaches. It is crucial whendealing with these vast amounts of data and the sortof data intensive analyses over large distributed sys-tems as encountered in bioinformatics to have a modelthat also addresses workflow performance.

4 Workflow Implementation ModelIn our approach a workflow implementation is

achieved in two steps: an implementation specificationstep where the workflow is expressed in terms of tan-gible, available resources: such as input and outputformat or data type, laboratory tools and applications;and a design/implementation mapping.

4.1 Implementation SpecificationThe implementation protocol that represents the ex-

ecutable workflow is consistent with the design pro-tocol which captures the semantics of the workflow.If a given design task translates into a compositionof tasks in the implementation model, a local refine-ment of the design task is created and attached to thespecific implementation such that each implementationtask is mapped directly to a design task. The fact

1-4244-1509-8/07/$25.00 02007 IEEE 1330

that the design is abstracted and represented by a ma-trix automatically promotes modularity of the designthrough a matrix partitioning1 and fully supports re-decomposition at the implementation stage. Matrixpartitioning is a mature technique for splitting a verylarge matrix into smaller, easier to store and more man-ageable sub-matrices for which key characteristics areeasily determined. In this model as in any graph matrixrepresentation, depending on the complexity and thesize of the workflow, incidence matrix can be large andfilled with zeros, therefore matrix partitioning allowsconformity in storage, reduction in redundant data andblock grouping for modularity.

Table 1. Graph Storage TableSource Destination Weight of the EdgeSource TDFQ 1Source GDFQ 1TDFQ AT1 1GDFQ AT1 1AT1 F 1F AT2 1Fat EGS 1Fat AT2 1EGS AT2 1AT2 Sink 1

Figure 6. Matrix Partitioning

Example We partition the ASP incidence matrix (seeFigure 6) and although the matrix partitioning was, inthis example, arbitrary the resulting sub-matrices are

still coherent sub-graphs of the Petri net illustrated inFigure 7, demonstrating the consistency of the designand the robustness and the degree of modularity of themodel.

A=4 zz.\ps/A, p8 EGS ,

Po TDFQ p2

Source TI F

A ~~~~~~~ ~I p4F

op1 GDFQp3

a_.

Figure 7. Sub-workflows

Such modeling supports add-ons and updates whichcan be easily performed without the overhead of refor-matting the whole design graph. For example in our

1Experiments on Sparse Matrix Partitioning by S. Riyavongy.CERFACS Working Note WN/PA/03/32 CERFACS, 42 AvenueG. Coriolis, 31057 Toulouse Cedex, France.

current illustration design we could replace the pro-cess p4 by some complex sub-graph without modify-ing the rest of the design and only be concerned withthe consistency of the data representing the edge com-ing and leaving p4. It is worth noticing that to fullysupport and reinforce data dependencies or other con-strains during decomposition, uniformity and unique-ness in the naming (IDs) of tasks must be adopted bythe designer, but this requirement is not an additionaloverhead because the storage and the query of tasksalready imposed such requirement on the overall work-flow design. This technique also allows the system tosearch for certain patterns in the design and to up-date a given design by searching for the most commondecomposition of a design task at the implementationstage.

4.2 Workflow Storage Schema

The storage of the design workflow is done at twodifferent levels of abstraction, although any one of themfully expresses the design. First the graph is storedin a table (source, destination, weight of edge). Forany given two connected tasks in the workflow, thesource attribute is the preceding task's name or ID, thedestination attribute is the successor task of the source,and the weight of edge is the number of data edgesfrom the source's output to the destination's input. Forexample the graph storage corresponding to Figure 2is displayed in Table 1.

The graph table could be the only record of theworkflow, but the computation and analysis of thePetri net and its matrix would have to be computedfor each implementation of the workflow. To eliminatethis calculation we propose to store a compressed ver-sion of its incidence matrix. This solution will avoidmassive matrix data storage because. Indeed as theworkflow gets largeer the matrix size increases, evenwith the matrix partitioning. The compression matrixstorage has the following schema: (ti, process pjm

1-4244-1509-8/07/$25.00 02007 IEEE

0 0 0 0 0 0 -i 1 i 0 00 ( 0 0 0 -1 0 0 -1 I 00 0 0 0 0 0 0 -1, 0 -1 i

. 0 ( 0 0 0 0 0 0 0 0 .l )

0 00 01 0.l I0-1 I00 0

00 000 000 000 0

1331

A2 A2

A3 A4

process pjn, Wm,...,Wn), where Wmi,..., Wn are the nu-

meric values of ati,pjm.. ati,pjn found in the matrix,with ati,pjx non zero. For example, A will be repre-sented with (Source, pO, p1, 1, 1), (TDFQ, pO, p2, -1,1), ..., (sink, plO, -1) and recorded as follows.

5 Conclusion and Future Work

ProtocolDB currently under development at Ari-zona State University is a system for scientific proto-col management, including creating, storing, querying,and analyzing scientific workflows. The key compo-nents of the system consist of a friendly user interfaceto design and implement workflows, a robust relationaldatabase to store workflows and their associated data.In this paper we present preliminary results related tostrategies for optimal protocol representation to opti-mize protocol access. Future work will focus on exploit-ing workflow equivalences to improve workflow execu-tion, workflow storage, and reasoning on data prove-nance.Acknowledgments This research was partially supportedby the National Science Foundation2 (grants IIS 0223042,IIS 0431174, IIS 0612273, and IIS 0738906). Michel Kinsyconducted the work while completing his undergraduatestudies at ASU. The authors would like to thank Piotr Wlo-darczyk and Christophe Legendre for their valuable input,and Natalia Kwasnikowska and Jan Van den Bussche formultiple discussions on scientific workflows.

References[1] David Booth and Canyang Kevin Liu. Web Ser-

vices Description Language (WSDL) Version 2.0Part 0: Primer. W3C Working Draft, Decem-ber 2004. http://www.w3.org/TR/2004/WD-wsdl20-primer-20041221/.

2Any opinion, finding, and conclusion or recommendation ex-pressed in this material are those of the authors and do notnecessarily reflect the views of the National Science Foundation.

[2] David Churches, Gabor Gombas, Andrew Harrison,Jason Maassen, Craig Robinson, Matthew Shields, IanTaylor, and Ian Wang. Programming Scientific andDistributed Workflow with Triana Services. Inter-national Journal on Concurrency and Computation:Practice and Experience, 18(10):1021-1037, 2006.

[3] Ewa Deelman, James Blythe, Yolanda Gil, CarlKesselman, Gaurang Mehta, Sonal Patil, Mei-Hui Su,Karan Vahi, and Miron Livny. Pegasus: Mapping Sci-entific Workflows onto the Grid. In European AcrossGrids Conference, pages 11-20, 2004.

[4] Alain Finkel. Decidability of the termination problemfor completely specified protocols. Distributed Com-puting, 7(3):129-135, 1994.

[5] Nada Hashmi, Sung Lee, and Michael P. Cummings.Abstracting workflows: unifying bioinformatics taskconceptualization and specification through semanticweb services. In W3C Workshop on Semantic Web forLife Sciences, Cambridge, Massachusetts, USA, 2004.

[6] Didier Cristani Alain Jean-Marie and Christine Coves.Petri net analysis: Complexity and finite coverabilitygraph in modular design. Studies in Informatics andControl, 14(1):54-64, 2005.

[7] Zoe Lacroix and Terence Critchlow, editors. Bioinfor-matics: Managing Scientific Data. Morgan KaufmannPublishing, 2003.

[8] Zoe Lacroix and Christophe Legendre. Analysis of aScientific Protocol: Selecting Suitable Resources. InFirst IEEE International Workshop on Service Ori-ented Technologies for Biological Databases and Tools,In conjunction with ICWS SCC, pages 130-137, 2007.

[9] Zoe Lacroix, Christophe Legendre, Louiqa Raschid,and Ben Snyder. BIPASS: Biolnformatics PipelinesAlternative Splicing Services. Nucleic Acids Research,Web Server Issue: W292-6, July 2007.

[10] Zoe Lacroix and Herve Menager. SemanticBio: Build-ing Conceptual Scientific Workflows over Web Ser-vices. In Data Integration in the Life Sciences, volume3615 of Lecture Notes in Computer Science, pages 296-299. Springer, 2005.

[11] Bertram Ludascher, Ilkay Altintas, Chad Berkley, DanHiggins, Efrat Jaeger, Matthew Jones, Edward A. Lee,Jing Tao, and Yang Zhao. Scientific Workflow Man-agement and the KEPLER System. Concurrency andComputation: Practice and Experience, Special Issueon Scientific Workflows, 18(10):1039-1065, 2005.

[12] Thomas M. Oinn, Matthew Addis, Justin Ferris, Dar-ren Marvin, Martin Senger, R. Mark Greenwood,Tim Carver, Kevin Glover, Matthew R. Pocock, AnilWipat, and Peter Li. Taverna: a tool for the com-position and enactment of bioinformatics workflows.Bioinformatics, 20(17):3045-3054, 2004.

[13] Mark D. Wilkinson and Matthew Links. BioMOBY: anopen-source biological web services proposal. Briefingsin Bioinformatics, 3(4):331-341, December 2002.

1-4244-1509-8/07/$25.00 02007 IEEE

- -

1332