Modelling Clinical Guidelines and Protocols in MHB.
A Tutorial
Katharina Kaiser, Andreas Seyfang Institute for Software Technology & Interactive Systems Vienna University of Technology Favoritenstrasse 9-‐11/188-‐1 1040 Vienna, Austria {lastname}@ifs.tuwien.ac.at
MHB-‐Tutorial Kaiser, Seyfang
I
version 1.0 March 18, 2010 Technical Report Asgaard-‐TR-‐2010-‐1
MHB-‐Tutorial Kaiser, Seyfang
II
Contents
Abstract .................................................................................................................................................III
Chapter 1 Introduction.....................................................................................................................1
Chapter 2 The MHB Ontology ........................................................................................................3
2.1 Control Flow Dimension ..................................................................................................3
2.1.1 clinical-‐activity ............................................................................................................5
2.1.2 if-‐then ..............................................................................................................................5
2.1.3 option-‐group ................................................................................................................6
2.1.4 decomposition .............................................................................................................6
2.1.5 synchronization ..........................................................................................................7
2.1.6 repetition .......................................................................................................................9
2.2 Data Dimension ................................................................................................................ 10
2.2.1 input.............................................................................................................................. 10
2.2.2 abstraction ................................................................................................................. 10
2.2.3 usage ............................................................................................................................. 11
2.2.4 definition..................................................................................................................... 12
2.3 Temporal Dimension...................................................................................................... 12
2.4 Evidence Dimension ....................................................................................................... 14
2.5 Background Information .............................................................................................. 16
2.6 Resources ............................................................................................................................ 16
2.7 Patient Aspects.................................................................................................................. 17
Chapter 3 Modelling a Guideline in MHB Using the DELT/A Tool ............................ 18
Bibliography....................................................................................................................................... 20
MHB-‐Tutorial Kaiser, Seyfang
III
Abstract
Modelling a clinical guideline or protocol in a computer-‐interpretable representation is a very complex task. It requires familiarity with the medical subject, ability to transform the medical task knowledge into rules, and knowledge of the data flow associated with the medical task. Performing this using one of the common computer-‐interpretable representation formalisms can be hard and error-‐prone. Thus, MHB – the many-‐headed bridge between guideline formats – was introduced to provide an intermediate step to abstract the medical knowledge before generating the final formalism. However, abstracting guideline knowledge in MHB is still a task that has to be trained – by both physicians and knowledge engineers. This tutorial provides material for training MHB modelling using example models.
MHB-‐Tutorial Kaiser, Seyfang
1
Chapter 1 Introduction
Clinical guidelines are “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” [1]. A guideline describes the optimal care for patients and therefore, when properly applied, it is assumed that they improve the quality of care.
Translating guidelines into a computer-‐processable form brings several advantages. It makes them more accessible to browsing, it allows their execution, i.e., the selection of the appropriate treatment steps based on the patient condition, and it is a precondition to various quality assurance techniques.
Producing a formal model of a guideline is difficult and expensive. In addition, the resulting model is often difficult to compare to the original. If a guideline is revised, the modelling effort is lost and it is not easy to detect which changes in the formal model are required by the changes in the original text. The main reason for this is that there is a large gap between natural language and the currently available formal representations. To close this gap in a versatile manner we designed an intermediate representation called MHB (Many-‐Headed Bridge) [2]. It can be seen as a small and versatile ontology of guideline components. It groups the statements in the guideline into chunks with predefined dimensions such as control flow, data flow, temporal aspects, and evidence. The aspects of each dimension are described using natural language.
MHB is designed as a versatile device to improve guideline quality. Modelling a guideline in MHB makes important aspects such as control and data flow, resources, and patient aspects explicit. They can easily be grouped in various overview lists. Using the MHB model helps in locating and acquiring knowledge, which is missing in the guideline text, and pointing out inconsistencies such as contradicting definitions or recommendations.
The tool, which is used to generate and maintain the guideline’s version in MHB, forms an important background for the design of the representation itself. This tool is called Document Exploration and Linking Tools with Add-‐ons (DELT/A) [3]. DELT/A allows for explicit linking between pairs of guideline parts in textual and formal representations.
DELT/A supports a multi-‐step modelling process. E.g., in a first step, the original guideline text is displayed at the left-‐hand side and the MHB model at the right-‐hand side. In a second step, the (then complete) MHB model is displayed at the left and a newly created translation of MHB to Asbru, GLIF, or ProForma is shown at the right-‐hand side. Tools similar to DELT/A are GEM Cutter [4], Degel [5], and Stepper [6]. DELT/A differs from GEM Cutter and Degel in maintaining
MHB-‐Tutorial Kaiser, Seyfang
2
explicit links between the original text and the formal representation. In contrast to Stepper it does not prescribe a fixed number of modelling steps from the informal text to the formal model.
In this tutorial we describe how a guideline is modeled in MHB and how the DELT/A tool is used therefore. Each MHB dimension is described using examples.
MHB-‐Tutorial Kaiser, Seyfang
3
Chapter 2 The MHB Ontology
MHB is an XML-‐based representation. The overall structure of an MHB file is very flexible. It is a series of chunks. Each chunk corresponds to a certain bit of information in the natural language guideline text, e.g., a sentence, part of a sentence, or more than one sentence. Initially, the order of the chunks reflects the order of building blocks in the original version of the guideline. However, they can be moved freely in the MHB file. Such regrouping eases the construction of a more formal representation based on the MHB model.
Below see the minimal MHB model. A set of chunks can be grouped in a chunk-‐group. When modelling a guideline or protocol you should make a chunk group for each section, subsection, etc. <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE root SYSTEM "~/xmlLanguages/MHB/MHB_1.03.dtd"> <!--MHB document created by k.kaiser using DELT/A on 5/28/09--> <root> <chunk-group title="chapter"> <chunk-group title="subheading"> <chunk chunk-id="#CHUNK-00001"> ... </chunk> </chunk-group> </chunk-group> </root>
Each chunk is then decomposed in its kind of information. There are eight different kinds of information, so called dimensions (see Figure 1 for an overview). We will now give examples for each kind of dimension.
2.1 Control Flow Dimension
A control flow consists of various activities that are connected in a given order. In order to model the control flow of a guideline or protocol, MHB provides various models (see Figure 2 for an overview).
When modelling, the most important aspect is to detect the action or activity within a text. For instance, see the following sentence:
Placenta should be carefully examined.
Here, the activity can be named „examination of placenta“.
MHB-‐Tutorial Kaiser, Seyfang
4
Figure 1. Dimensions in MHB. A chunk can be described by up to eight different dimensions.
Figure 2. Description of the control element in MHB. Its various child elements describe specific aspects of the control flow dimension of a chunk.
Another example is this:
Intravenous administration of hydrating or glucose solutions should be reserved for those patients who refuse to eat with a protracted labour.
Thereby, the activity is “intravenous administration of hydrating or glucose solutions”.
MHB-‐Tutorial Kaiser, Seyfang
5
Furthermore, we should be able to determine conditions that must be fulfilled in order to perform the action. In this case we have two conditions that have to be fulfilled: “refuse to eat” and “protracted labour”.
As we now know the most important parts of these sentences, we can now proceed with modelling them in MHB. MHB provides various models in order to do so. We will describe each of them with an example.
2.1.1 clinical-‐activity
With this model we can describe an activity for which only a description is given. No further decomposition is done. Here is an example for a clinical activity1:
Placenta should be carefully examined.
We can model such a sentence using clinical-activity within the control dimension:
<control> <clinical-activity name="examination of placenta" description="Placenta should be carefully examined." /> </control>
2.1.2 if-‐then
If an action or activity is only performed when a specific condition is fulfilled, we use the if-then model.
The following example sentence can be presented with this model2.
Intravenous administration of hydrating or glucose solutions should be reserved for those patients who refuse to eat with a protracted labour.
The if-then model contains a condition attribute, whereas conditions can be combined with AND and OR and can be negated with NOT. If we model a condition we have to remove any “if”, “when”, “in case of”, etc. from the original text. We will probably alter the condition’s text we have copied from the original text. Furthermore, the model contains a result attribute, which is used to present the action or activity. Other attributes are modifiers for the condition and the result as well as a degree-of-certainty attribute.
<control> <if-then condition="refuse to eat AND protracted labour" result="intravenous administration of hydrating or glucose solutions" degree-of-certainty="should" /> </control>
1 Actions or activities are shown with blue background. 2 Conditions are presented with red background.
MHB-‐Tutorial Kaiser, Seyfang
6
2.1.3 option-‐group
If a set of alternative actions or activities is given, we can use the option-group to specify the alternatives, the selection type (single-‐choice, multiple-‐choice), or the number of options to select, etc. Each alternative action is presented by an if-then model. In case the condition is not explicitly stated we leave it empty.
Figure 3. Option-‐group element in control dimension. It is used to describe alternative actions.
MEPERIDINE ADMINISTRATION ROUTE DOSAGE Intramuscular 1mg/kg Intravenous 1 fl of 100 mg in 20 ml of physiological solution up to 0.5
mg/kg repeatable after 3-‐4 hours
<control> <option-group parent-task="meperidine administration" selection-type="single-choice" > <if-then condition="" result="intramuscular meperidine administrat." /> <if-then condition="" result="intravenous meperidine administration" /> </option> </control>
2.1.4 decomposition
This model is used to describe an action that consists of multiple sub-‐actions. The difference between a decomposition and an option-group is that in a decomposition the sub-‐actions can be synchronized according to a specific ordering, such as parallel, sequential, any-‐order, or unordered (see Figure 4).
MHB-‐Tutorial Kaiser, Seyfang
7
Figure 4. Decomposition element in control dimension. It is used if a task is split in more than one subtasks.
At admission: (a) collect case history (b) measure blood pressure (c) perform vaginal examination (d) perform fetal heart rate evaluation and an ultrasound assessment of amiotic
fluid index
In the example above, the task to be performed at admission consists of five sub-‐actions. We therefore model a parent-task “admission examination” containing five sub-‐tasks.
<control> <decomposition parent-task="admission examination" description="admission of a woman who fit admission criteria to the hospital" > <child-task name="collection of case history" /> <child-task name="measurement of blood pressure" /> <child-task name="examination of vagina" /> <child-task name="evaluation of fetal heart rate" /> <child-task name="ultrasound assessment of amiotic fluid index" /> </decomposition> </control>
All these child-tasks are performed without synchronization, which means that no ordering of the tasks is stated.
2.1.5 synchronization
When several tasks are performed in parallel or otherwise independent from each other, the question arises when to pursue the rest of the guideline. For this purpose, we use the synchronization model (see Figure 5 for details). Many guideline representation formalisms (e.g., Asbru, GLIF) define those subtasks
MHB-‐Tutorial Kaiser, Seyfang
8
(”children”, awaited-subtasks in MHB) which must be completed before the next step is taken in a logical expression.
Figure 5. Synchronization element in control dimension. It is used if one or more tasks have to be completed or aborted in order to continue another task.
In the following example, we see the dependencies of the last activity „send the blood samples“ on various activities. Sending the blood samples can only be performed, when all other activities have already been performed (and are finished).
The nurse will cannulate a vein and collect 3 test tubes of blood for urgent blood examination.
The neurologist will fill the request form for blood examinations, call the Urgency Lab to order an immediate analysis of the blood, and send the blood samples.
Furthermore, the task “collect 3 test tubes of blood” can also only be performed if the task “cannulate a vein” has already been performed.
<chunk chunk-id=”CHUNK-0012”> <control> <synchronization
waiting-task="collect 3 test tubes of blood"> <awaited-subtask name="cannulate a vein" /> </synchronization> </control> </chunk> <chunk chunk-id=”CHUNK-0013”> <control> <synchronization waiting-task="sending blood samples" > <awaited-subtask name="collect 3 test tubes of blood"/> <awaited-subtask name="fill request form for blood examinations" /> <awaited-subtask name="call Urgency Lab" /> </synchronization> </control> </chunk>
MHB-‐Tutorial Kaiser, Seyfang
9
2.1.6 repetition
Repeated actions or activities can be modeled using the repetition element. Thereby, a parent task is named as well as a child task, which describes the repeated action (see Figure 6).
Figure 6. Repetition element in control dimension. It is used if a task is performed more than once.
Look again at the example we have already seen in Section 2.1.3 (option-‐group). We focus on the intravenous administration of Meperidine, which is repeated after 3 to 4 hours:
MEPERIDINE ADMINISTRATION ROUTE DOSAGE Intramuscular 1mg/kg Intravenous 1 fl of 100 mg in 20 ml of physiological solution up to 0.5
mg/kg repeatable after 3-‐4 hours
<control> ... <repetition envelope-task=”intravenous meperidine administr.” repeated-task=”intravenous meperidine dose” repeat-specification=”after 3 to 4 hours” /> </control>
Thus, we use the parent task “intravenous meperidine administration” from the option-group as envelope-task. The task that is repeated is then “intravenous meperidine dose”, which is the activity that is repeated according to the repeat-specification. The repeat-specification can describe the number of repetitions, the period between repetitions, or even the constraints for ending the repetition cycle.
MHB-‐Tutorial Kaiser, Seyfang
10
2.2 Data Dimension
During diagnosis and treatment of the patient, data is processed. As control flow describes the gathering of information, data flow describes how one piece of information is abstracted from other ones, but also where data is used and where it is obtained.
The data dimension represents the definition of data, data input, data usage, and abstraction rules to calculate data.
Figure 7. Data dimension.
2.2.1 input
If data is entered into the patient record during patient interview or diagnosis or data is asked by the system, we use the input model. We only have to define the input model once the data is entered. If the data is used multiple times, we do not need to use the input model again – except when the data is changing over time and re-‐entered into the system (e.g., repeated measuring of blood pressure, measuring labour contractions).
Most of the data will be entered during initial examination (e.g., date of birth, height, weight):
<data> <input name ="date-of-birth" /> <input name ="body-weight" /> <input name ="body-height" /> ... </data>
2.2.2 abstraction
Often data is used that is calculated from other data. In order to define the calculation rule we use the abstraction model. See our example to calculate the correct dosage for intramuscular administration of meperidine:
MHB-‐Tutorial Kaiser, Seyfang
11
MEPERIDINE ADMINISTRATION ROUTE DOSAGE Intramuscular 1mg/kg Intravenous 1 fl of 100 mg in 20 ml of physiological solution up to 0.5
mg/kg repeatable after 3-‐4 hours
<data> <abstraction abstraction-rule="body-weight * 1mg Meperidine" result="meperidine-dosage-im" /> ... </data>
The name of the variable is defined using the result attribute.
Another example is the calculation of the BMI (body mass index). Thereby, the person’s height and weight are used (kg/m2). As in most cases the height of a person is stated in cm we divide it through 100 to receive the height in meters.
<data> <abstraction abstraction-rule="body-weight / (body-height/100 * body-height/100)" result="bmi" /> ... </data>
2.2.3 usage
All variables used in a chunk (in an abstraction-rule or in a condition of an if-then model), have to be modeled with the usage model3.
<data> <abstraction abstraction-rule="body-weight / (body-height/100 * body-height/100)" result="bmi" /> <usage name="body-weight” /> <usage name="body-height” /> </data>
Here, we have again the example of Section 2.1.2 (if-‐then).
Intravenous administration of hydrating or glucose solutions should be reserved for those patients who refuse to eat with a protracted labour .
<control> <if-then condition="refuse to eat AND protracted labour" result="intravenous administration of hydrating or glucose solutions" degree-of-certainty="should" /> </control> <data> <usage name="refusal to eat” /> <usage name="protracted labour” /> </data>
3 Data aspects are displayed in italic font and blue double borders.
MHB-‐Tutorial Kaiser, Seyfang
12
2.2.4 definition
Of course, variables have to be defined, too. The definition of a data item is rarely found in the guideline in explicit form. Still, it is necessary for the formal version of the guideline.
Figure 8. Definition element in data dimension. It is used to define specifications of a parameter.
Similar to the data definition in programming languages, we use the definition model in MHB (see Figure 8). It consists of a name, additional descriptions, a type, and often a range of plausible values and a preferred unit.
<data> <definition name ="bmi" description="body mass index; body-weight (kg)/body-height (m)^2" technical-specification=”numeric value; unit: kg/m2”/> <definition name ="body-weight" description="weight of a person’s body" technical-specification=”numeric value; unit: kg” /> <definition name ="body-height" description="height of a person’s body" technical-specification=”numeric value; unit: cm” /> </data>
2.3 Temporal Dimension
Time is an important dimension in treatment processes. Both data and control flow may have temporal aspects. They can be qualitative or quantitative. Qualitative temporal relations between time points are before, after, and equal. For two intervals, Allen defined 13 relations based on all combinations of the relations of time points [7].
In order to be capable of specifications about time in many guideline representation formalisms, MHB’s temporal dimension covers at least Asbru’s temporal statements (start, end, duration, and qualitative relation). See Figure 9 for an overview.
MHB-‐Tutorial Kaiser, Seyfang
13
Figure 9. Temporal dimension.
The most frequently used elements are start, end, and duration. They can be used to describe the starting and ending time of an interval as well as its duration (see Figure 10).
Figure 10. Start, end, and duration element of time dimension.
MHB-‐Tutorial Kaiser, Seyfang
14
The following example shows a temporal aspect modelled with the start element4.
Women with pain but no cervical changes should be re-‐examined after two hours.
<chunk chunk-id=”CHUNK-0005”> <control> <if-then condition="pain AND NOT cervical changes " result="re-examination after two hours" degree-of-certainty="should" /> </control> … <time subject=”re-examination after two hours” > <start reference-point="previous examination" estimate="2 hours" /> </time> </chunk>
In order to describe qualitative relations between time points or between intervals (according to Allen), qualitative-‐relation can be used. Figure 11 shows its details.
Figure 11. Qualitative relation to describe the temporal dimension between time points or
intervals.
2.4 Evidence Dimension
An evidence-‐based guideline builds a bridge from carefully examined pieces of evidence which are obtained for certain particular parts of the problem to generally applicable recommendations for diagnosis, treatment, screening, etc.
For explicit references with defined format (statements of evidence, literature references) MHB provides the attributes grade, level, importance and
4 Temporal aspects are displayed in green background.
MHB-‐Tutorial Kaiser, Seyfang
15
literature-reference. However, only some of the many theoretically possible combinations are used in practice.
• Literature references in the scientific explanation are not rated or graded; they are only represented by storing the reference (number or name of first author) in literature-reference.
• Sometimes literature references in evidence conclusions have a level, which is stored in the attribute of this name while the reference is again stored in literature-reference.
• Evidence conclusions often have a grade.
• Evidence conclusions and/or recommendations may have an importance attached to them by the guideline authors independent of the grade of evidence. In these cases, the attribute importance is used in parallel to grade.
Note that the usage of grade, level and importance depends on the organization issuing the guideline.
For implicit references in the guideline, it can help to improve the quality of the guideline to make them explicit in the MHB file. The attribute is-based-on contains a reference to another MHB element. The ID used is that of the chunk on which the claim in this chunk is based on.
In practice, making implicit links explicit will be limited by a trade-‐off between the work investment into this task and the expected gain in quality.
Figure 12. Evidence dimension. If any kind of evidence information is given in a guideline or protocol, it can be described by this dimension. It makes evidence information explicit. However, most guideline representation formalisms cannot deal with it.
MHB-‐Tutorial Kaiser, Seyfang
16
The example below shows an extract of a SIGN guideline. SIGN is an institution indicating both level of evidence and grades of recommendations. Furthermore, they also specify literature references.
An HTA explored the optimum timing of brain imaging for patients in the acute phase of stroke.48 A decision analysis model was developed comparing a ‘scan all patients within 48 hours’ pathway of care in acute stroke against alternative scan strategies. The most cost-‐effective strategy, in terms of least overall cost and most quality adjusted life years (QAlys) after adjusting for different age ranges, proportions of infarcts and accuracy of CT, was to scan all patients immediately.
1++
A All patients with suspected stroke should have brain imaging immediately on presentation.
<chunk chunk-id=”CHUNK-0008”> … <evidence grade=”A" level=”1++” literature-reference=”48” /> </chunk>
2.5 Background Information
Background information describes various aspects of the topic. Some refer to a particular statement or group of statements while others are only loosely coupled to particular statements or recommendations. Also their potential to be formally encoded largely varies. Figure 13 shows the background element and its aspects that can be described by it.
Figure 13. Background dimension. If the guideline or protocol contains information such as intentions, effects, relations, educational information, explanations, or indicators, we can use this dimension to describe it explicitly.
2.6 Resources
Resources are important aspects in managing treatment processes. With this dimension we are able to define personnel, devices, costs (also drugs), etc.
MHB-‐Tutorial Kaiser, Seyfang
17
2.7 Patient Aspects
Using this dimension, we can define aspects that predominantly concern patients. For instance, we use it to mark risks, discomfort, and so on.
MHB-‐Tutorial Kaiser, Seyfang
18
Chapter 3 Modelling a Guideline in MHB Using the DELT/A Tool
The easiest way to generate a MHB model of a guideline is by using the DELT/A5 tool.
The tool is a Java application and can be downloaded here: http://www.asgaard.tuwien.ac.at/~peter/DELTA/download/delta.zip
or
http://www.asgaard.tuwien.ac.at/~peter/DELTA/download/delta.tar.gz.
You need Java (version 1.4 or higher). Extract the archive to a new folder and run it by either using one of the start scripts or by double-‐clicking on delta.jar.
Figure 14. The DELT/A tool.
The tool consists of three panes (see Figure 14). In the left upper pane the HTML file is loaded. In the right upper pane the MHB model is generated. It is displayed in XML tree view. In the bottom pane the ‘macros’ are loaded. Macros combine
5 Document Exploration and Linking Tool / with Addons
MHB-‐Tutorial Kaiser, Seyfang
19
multiple XML elements together with their attributes and can be used for the simple construction of new XML documents. There are specific macros defined for the MHB language.
The typical proceeding is the following:
1. Define a chunk by marking up text in the left pane. A chunk should not contain too much text. Rule of thumb: one sentence = one chunk.
2. In the right pane, select the position of the MHB element to be inserted in MHB’s XML tree.
3. Select the MHB macro from the macros pane. On the right side of the pane you can select how the macro is inserted: ‘Insert into’, ‘Insert below’, ‘Insert above’, ‘Replace’.
4. After the content of the macro is inserted in the right pane, you can add additional information (e.g., additional optional attributes).
Always stick on these four steps. Macros that cannot be inserted are greyed out (depending on your position in MHB’s XML tree).
As an XML file is not very comprehensible and clearly represented to humans, you can also use the OMA tool6 to merge the information from the HTML file and the MHB file. Its output is a HTML file displaying the MHB output in HTML tables next to the text of the original document. Thus, it is easier to see which text has been modeled by which kind of MHB models. Furthermore, the hierarchy of the tasks is displayed, the task cross reference, and the data cross reference.
Additional hints:
• Create a DELT/A project containing the HTML file and the MHB file.
• Save your files frequently. Some users reported that after “Save all” only the HTML file was saved, but not the MHB file. Save your files separately to avoid loss of your work.7
• Practice, practice, practice, …
Acknowledgements. This work is partially supported by “Fonds zur Förderung der wissenschaftlichen Forschung FWF” (Austrian Science Fund), grant L290-‐N04. The research leading to these results has received funding from the European Community's Seventh Framework Programme (FP7/2007-‐2013) under grant agreement n°216134 and the European Commission’s IST program, under contract number IST-‐FP6-‐508794.
6 The OMA tool is a Perl script. It uses the Treebuilder library, which is already included in some distributions (e.g., ActiveState). If your Perl installation doesn’t contain the library, you have to install it from http://search.cpan.org/~petek/HTML-‐Tree-‐3.23/lib/HTML/TreeBuilder.pm. Note for Mac users: it may be necessary to install the library as superuser. 7 That’s embarassing and we know it.
MHB-‐Tutorial Kaiser, Seyfang
20
Bibliography
[1] M. J. Field and K. N. Lohr, editors. Clinical Practice Guidelines: Directions for a New Program. National Academies Press, Institute of Medicine, Washington DC, 1990.
[2] A. Seyfang, S. Miksch, M. Marcos, J. Wittenberg, C. Polo-‐Conde, and K. Rosenbrand. Bridging the gap between informal and formal guideline representations. In G. Brewka, S. Coradeschi, A. Perini, and P. Traverso, editors, European Conference on Artificial Intelligence (ECAI-2006), volume 141 of Frontiers in Artificial Intelligence and Applications, pages 447–451, Riva del Garda, Italy, 2006. IOS Press.
[3] P. Votruba, S. Miksch, and R. Kosara. Facilitating knowledge maintenance of clinical guidelines and protocols. In M. Fieschi, E. Coiera, and Y.-‐C. J. Li, editors, Proceedings from the Medinfo 2004 World Congress on Medical Informatics, pages 57–61. AMIA, IOS Press, 2004.
[4] R. N. Shiffman, B. T. Karras, A. Agrawal, R. Chen, L. Marenco, and S. Nath. GEM: a proposal for a more comprehensive guideline document model using XML. Journal of the American Medical Informatics Association (JAMIA), 7(5):488–498, 2000.
[5] Y. Shahar, O. Young, E. Shalom, A. Mayaffit, R. Moskovitch, A. Hessing, and M. Galperin. DEGEL: A hybrid, multiple-‐ontology framework for specification and retrieval of clinical guidelines. In M. Dojat, E. Keravnou, and P. Barahona, editors, Proceedings of the 9th Conference on Artificial Intelligence in Medicine in Europe, AIME 2003, volume 2780 of LNAI, pages 122–131, Protaras, Cyprus, 2003. Springer Verlag.
[6] V. Svátek and M. Ružička. Step-‐by-‐step mark-‐up of medical guideline documents. International Journal of Medical Informatics, 70(2-‐3):319–335, July 2003.
[7] J. F. Allen. Maintaining knowledge about temporal intervals. Communications of the ACM, 26(11), November 1983.