Universität Würzburg
Continuous Knowledge Engineeringwith Semantic Wikis
Dr. Joachim Baumeister
Kumulative Habilitationsschriftzur Erlangung der Lehrbefähigung für Informatik
Fachmentorat:Prof. Dr. Frank Puppe, Universität Würzburg
Prof. Dr. Klaus-Dieter Althoff, Universität HildesheimProf. Dr. Dietmar Seipel, Universität Würzburg
Würzburg, 25. Februar 2010
Continuous Knowledge Engineeringwith Semantic Wikis(Habilitation Summary)
Joachim Baumeister (University of Würzburg)
This work summarizes the research body of the following publications:
1. Joachim Baumeister. Advanced measures for empirical testing. In FLAIRS’09: Proceed-ings of the 22th International Florida Artificial Intelligence Research Society Conference,pages 378–383. AAAI Press, 2009 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2. Joachim Baumeister and Grzegorz J. Nalepa. Verification of distributed knowledge insemantic knowledge wikis. In FLAIRS’09: Proceedings of the 22th International FloridaArtificial Intelligence Research Society Conference, pages 384–389. AAAI Press, 2009. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3. Joachim Baumeister and Frank Puppe. Web-based knowledge engineering with knowl-edge wikis. In Proceedings of Symbiotic Relationships between Semantic Web and Knowl-edge Engineering (AAAI 2008 Spring Symposium), 2008 . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4. Joachim Baumeister and Dietmar Seipel. Verification and refactoring of ontologies withrules. In EKAW’06: Proceedings of the 15th International Conference on KnowledgeEngineering and Knowledge Management, pages 82–95, Berlin, 2006. Springer . . . . . 51
5. Joachim Baumeister and Dietmar Seipel. Anomalies in ontologies with rules. Web Se-mantics: Science, Services and Agents on the World Wide Web, 8(1):55–68, 2010 . . . . 67
6. Joachim Baumeister, Jürgen Bregenzer, and Frank Puppe. Gray box robustness testing ofrule systems. In KI’06: Proceedings of the 29th Annual German Conference on ArtificialIntelligence, LNAI 4314, pages 346–360. Springer, 2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
7. Joachim Baumeister, Thomas Kleemann, and Dietmar Seipel. Towards the verificationof ontologies with rules. In FLAIRS’07: Proceedings of the 20th International FloridaArtificial Intelligence Research Society Conference, pages 524–529, 2007 . . . . . . . . . . . 97
8. Joachim Baumeister, Martina Menge, and Frank Puppe. Visualization techniques for theevaluation of knowledge systems. In FLAIRS’08: Proceedings of the 21th InternationalFlorida Artificial Intelligence Research Society Conference, pages 329–334. AAAI Press,2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3
9. Joachim Baumeister, Jochen Reutelshoefer, and Frank Puppe. Continuous knowledgeengineering with semantic wikis. In CMS’09: Proceedings of 7th Conference on ComputerMethods and Systems (Knowledge Engineering and Intelligent Systems), pages 163–168.Oprogramowanie Naukowo-Techniczne, 2009 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
10. Joachim Baumeister, Jochen Reutelshoefer, and Frank Puppe. KnowWE: A semantic wikifor knowledge engineering. Applied Intelligence, to appear, 2010 . . . . . . . . . . . . . . . . . 115
11. Jochen Reutelshoefer, Florian Lemmerich, Fabian Haupt, and Joachim Baumeister. Anextensible semantic wiki architecture. In SemWiki’09: Fourth Workshop on Semantic Wikis– The Semantic Wiki Web (CEUR proceedings 464), 2009b . . . . . . . . . . . . . . . . . . . . . . . 145
12. Sebastian Schaffert, François Bry, Joachim Baumeister, and Malte Kiesel. Semantic wikis.IEEE Software, 25(4):8–11, 2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
1 Introduction
With the commercial success of knowledge-based technologies in the nineties, we see an in-creasing relevance in industry, nowadays. Intelligent systems are integrated and fully appliedin industry: For example, systems cover service-support tasks in telecommunication domainsas it is implemented by companies like IISY AG1. In the technical domain, Bosch GmbH2 pro-vides knowledge-based systems to support the diagnosis task in the automotive industry (Nghiaand Puppe, 2009), whereas TIGER is an intelligent system for the fault diagnosis of gas tur-bines (Milne and Nicol, 2000). Traditionally, intelligent systens are well-known in the medicaldomain, see for example the system SmartCare for the automated ventilation of patients (Mers-mann and Dojat, 2004), SonoConsult for the adaptive documentation and consultation of sono-graphical examinations (Hüttig et al., 2004), or GIDEON for the treatment of febrile travel-ers (Kimura et al., 2005).
The prejudicial cost-benefit ratio of knowledge-based systems, however, hinders its wide-spread success. With the cost-benefit ratio of a knowledge-based system we refer to the relationbetween the costs of developing and maintaining a system and the resulting benefit of its applica-tion when put into daily use. Unfortunately, the costs are often the critical element, since the for-malization of domain knowledge is a very time-consuming and specialized task. To accomplishthis task, development processes and tools are proposed in literature, see for instance (Angeleet al., 1998; Schreiber et al., 2001), but these are rarely adapted to the specific needs and require-ments of the particular application domain. Moreover, todays knowledge engineering projectsoften face the challenge that knowledge is present at different levels of formalization, but pro-cesses and tools are mostly not sufficiently prepared to meet this challenge. Knowledge appearsin different representations ranging from technical documents, construction plans, sheets, andexperiences of human experts, but also in the explicit form of rules and models. In summary, the
1http://www.iisy.com2http://www.bosch.com
4
knowledge acquisition bottleneck still exists today. In various knowledge engineering projectswe frequently experienced the following dilemmas:
1. The Single/Multiple Experts Dilemma.
The motivation and sophistication of single domain specialists is often the driving force ofsuccessful knowledge acquisition and evolution. Although, high-quality experts can guar-antee the construction of high-quality knowledge bases, these persons are often short intime and motivational endurance. Distributing the workload over a number of specialistswould decrease this problem, but at the same time would increase the risk of reducing theoverall quality of the formalized knowledge.
In addition, the collaboration of a group of specialists is hardly supported by (academicand industrial) authoring tools. Here, the dilemma exists of favoring a distributed over amonolithic development process—involving multiple specialists instead of a single spe-cialist.
2. The Flexibility/Productivity Dilemma.
Current process models and state–of–the–art tools are often tailored to capture the domainknowledge in a specific knowledge representation and knowledge granularity. The mentalmodel of domain specialists, however, often differs from the given representation. Toolsoften lack the flexibility to map the mental models to the acquisition interfaces of thetools and thus complicate the development process even more. Besides the mental modelof the specialists, knowledge additionally is often already present in various forms, suchas textual and tabular data, but also as explicit rules.
On the one hand, mapping the particular mental model of the specialists to the providedknowledge representation and interfaces, respectively, often turned out to be difficult andtime-consuming. On the other hand, a tool, that offers the maximal flexibility regardingthe user interfaces and the provided knowledge representations, typically would increasethe complexity of its use and therefore decreases the productivity of the developers; thisprinciple was described as the Flexibility-Usability Tradeoff (Lidwell et al., 2003, p. 86).In consequence, we face the dilemma of demanding a tool with maximal flexibility vs. atool with maximal productivity.
3. Sophisticated Inference vs. Pragmatic Knowledge Formalization.
At the beginning of a project it is often very difficult to determine the appropriate levelof formalization. Here, we face the dilemma of using expressive knowledge for sophisti-cated inferences vs. a broad but pragmatic knowledge formalization. When modeling thedomain knowledge on a very precise level using an expressive knowledge representation,we are able to perform sophisticated inferences. As a drawback, knowledge elicitationat such a detailed level is a time-consuming and complex task; sometimes, it is actuallynot possible to formalize parts of the domain knowledge that precisely. In contrast, thepragmatic knowledge formalization process propagates to start with a shallow but broadknowledge acquisition at the beginning of the project. Later, parts of the knowledge arerefined when necessary. For example, it is sometimes sufficient that parts of the knowl-edge base are simply specified as natural text and remain at this formalization level. Other
5
parts of the knowledge base are required for automated inferences, so a specialization ofthe knowledge becomes necessary.
In the presented work, we contribute novel concepts and tools in order to lighten these dilem-mas. Certainly, we are not aiming for solving the dilemmas as a whole, but we claim to helpsimplifying the corresponding challenges.
In the following, we introduce the knowledge formalization continuum (Baumeister et al.,2009) as a mental metaphor, that intends to help the domain specialist in understanding theconcept of knowledge and its gradual formalization. In summary, the knowledge formalizationcontinuum proposes a flexible interpretation of knowledge, where the domain knowledge is notrestricted to a specific formalization representation, but can range from multimedia and text toexpressive logic formulae. With the introduction of the knowledge formalization continuum wealleviate Dilemma 2 and Dilemma 3. For the practical application, we see that current state–of–the–art tools are not capable to work on the knowledge formalization continuum. We con-sequently propose and demonstrate the implementation of a new generation of knowledge engi-neering tools, that are derived from Semantic Wikis (Schaffert et al., 2008). This new generationof tools offers a flexible formalization process and provides the means to capture and reason withvarious sources of knowledge. Additionally, it scales with the number of users and domain spe-cialists, respectively. We introduce the extensible Semantic Wiki KnowWE (Baumeister et al.,2010) by describing its architecture and knowledge engineering approach. The availability of anappropriate and flexible tool such as KnowWE relieves Dilemma 1 and Dilemma 2.
The introduction of a new conceptual model and a corresponding tool, however, does nottackle the question about the quality of the formalized knowledge. Having the conceptual modelof multimodal knowledge sources, traditional evaluation techniques cannot be fully applied any-more. We discuss advanced evaluation methods that are applicable and practically relevant inthe context of the presented approach. Evaluation is often defined as the upper class of thevalidation and verification tasks (Ayel and Laurent, 1991). More recently, the additional taskassessment is added as a third subdiscipline. Regarding evaluation, we distinguish the followingsub-tasks:
1. Validation as a black-box test investigates the reasonable behavior of the system, i.e., bychecking whether the system yields the expected outputs for given inputs.
2. Verification as a white-box test analyzes the knowledge base in order to detect anomaliesor further deficiencies that may affect the correct and maintainable implementation of thesystem. Here, the system is often tested against a previously defined specification.
3. Assessment as a soft category of tests mostly considering the utility and the effectivity ofthe built system.
Advanced knowledge engineering tools—such as the Semantic Wiki KnowWE—representthe fundamental concepts of the domain knowledge in an ontology. We motivated above, thatadvanced processes and tools should be capable to combine different knowledge formalizations.As a popular example, the Semantic Web initiative currently proposes the extension of ontolo-gies by a rule-based layer, e.g., see (Horrocks et al., 2005). First, we discuss implications of the
6
verification process when ontologies are mixed with rules. The implications of traditional ver-ification methods, when applied to Semantic Wikis, were discussed in (Baumeister and Seipel,2010). Second, the validation of knowledge bases is an important aspect of knowledge engi-neering. We introduce the notion of a sequential test case as an extension of classic test cases,that proved to be appropriate for the validation of industrial knowledge bases. Consequently, wediscuss corresponding adaptations of the precision and recall measures (Baumeister, 2009), thatare applied to the results of empirical testing runs. Besides the plain validation of knowledgebases by empirical testing methods, the robustness of the built system is an interesting feature.We introduce the concept of grey-box robustness testing (Baumeister et al., 2006), that is anextension of the previously introduced robustness testing by degradation studies (Groot et al.,2003, 2000). Also, adapted visualization methods (Baumeister et al., 2008) showed a significantimprovement during the evaluation and manual inspection of knowledge bases.
Each of the following sections summarizes and highlights the important aspects of the specifictopics covered in this thesis.
2 The Knowledge Formalization Continuum
As we motivated in the introduction, traditional knowledge engineering approaches require theformalization of knowledge at an early stage and—more importantly—at a fixed level of formal-ization. Experiences in many real-world applications, however, showed that both requirementsare often not necessary but rather hinder the successful development of the project. We con-tribute the concept of the knowledge formalization continuum in order to give domain special-ists a flexible mental model of the knowledge to be used in the application project. This mentalmodel frees all involved parties to commit to a particular kind of knowledge formalization at anearly stage, but offers a versatile understanding of the formalization process.
A continuum can be seen as “a nonspatial whole in which no part or portion is distinct ordistinguishable from adjacent parts”; alternatively a continuum can be understood as “anythingthat goes through a gradual transition from one condition, to a different condition, without anyabrupt changes”3.
We use these definitions of a continuum to explain the idea of the knowledge formalizationcontinuum, where gradual transitions on formalization degrees of the same knowledge are possi-ble, but where the knowledge to be modelled experiences no abrupt changes or “discontinuities”.It is important to notice that the knowledge formalization continuum is neither a physical modelnor a methodology for developing knowledge bases. Rather, it should be seen as a metaphor ofthe knowledge development process in order to help the domain specialists to see even raw data,such as text and multimedia, as first-class knowledge. In the extreme cases, domain knowledgeis provided as very informal data (images, text), or is represented by formal representations suchas decision trees or functional models. See Figure 1 for a (non-exhaustive) depiction of thedifferent knowledge representations possible in the knowledge formalization continuum. Eachformalization alternative has its own advantages and drawbacks. For example, textual knowl-edge can be easily elicited and often is already available in the domain. No prior knowledge with
3see WordNet (Wordnet, 2010)/Wikipedia (Wikipedia, 2010) for full definitions/explanations
7
Knowledge Formalization Continuum
TextTags
Semantic annotations
Fault models
Functional modelsDecision
trees
Cases
Segmented text
Tabular data
Semantically equivalent transitions
Images
Mindmaps /Flow charts
LogicRules
Figure 1: Possible knowledge transitions within the knowledge formalization continuum.
respect to tools or knowledge representation is necessary. However, automated reasoning usingtextual knowledge is not possible with current state–of–the–art methods: The knowledge canbe retrieved only by using string-based matching methods but not by semantic queries. Logicrules or models are well-suited for automated reasoning, and queries can be processed on the se-mantic level. In contrast to textual knowledge, the acquisition of rules and models is a complexand time-consuming task. The transition between two representations on the knowledge for-malization continuum is often possible by using established methods, such as natural languageprocessing, text mining, visualization methods, refactorings, and manual elicitation methods.
The core ideas of the knowledge formalization continuum and its application within a Seman-tic Wiki are described in (Baumeister et al., 2009).
3 The eXtensible Semantic Wiki KnowWE
In recent years the advent and success of Web 2.0 application, such as wikis, blogs, and socialnetworks, has changed the way people are using the Internet. When compared to traditional websites, Web 2.0 applications explicitly involve the users as primary contributors to the system.Thus, the value of the particular systems usually grows with the increasing contribution of theusers. Not only private life but also daily business is influenced by the success of Web 2.0approaches. One prominent example is the wide-spread use of wikis as flexible knowledgemanagement tools, both in personal life and business environments.
Standard wiki systems, however, show limitations when the included content is intended tobe used as explicit knowledge. For the retrieval of the content, only a simple full-text searchis possible, and knowledge connected across different articles cannot be aggregated in a unifiedmanner. This issue motivated the development of Semantic Wikis that extend standard wikisby an explicit ontological layer defined by semantic annotions of the wiki content. Thanks to
8
semantic annotations, knowledge reuse is improved by semantic search and semantic naviga-tion (Schaffert et al., 2008). At the same time, Semantic Wikis successfully serve as ontologydevelopment tools, that provide a simple, web-based interface to build semantic applications.Typically, the expressiveness of an ontology developed by a Semantic Wiki corresponds to asubset of the web ontology language OWL, which is sufficient for many applications.
The development of (diagnostic) knowledge systems, however, commonly requires the inte-gration of strong problem-solving knowledge, for example (production) rules, decision trees, andfault models. We consequently developed the system KnowWE as an extensible Semantic Wiki,that is able to capture and share strong problem-solving knowledge of various types. KnowWEis the first implementation of a Semantic Wiki, that explicitly integrates strong problem-solvingknowledge into the wiki context. The extension by strong problem-solving knowledge motivatesthe following conceptual changes to the Semantic Wiki architecture:
(a) The representation of problem-solving knowledge and its alignment with the ontology layer.
(b) The design of a knowledge base that is distributed over the wiki.
(c) The appropriate interoperability of explicit knowledge with the surrounding tacit knowledgesuch as text and multimedia.
(d) The tailored interfaces to capture and use the knowledge.
A task ontology interweaves standard OWL ontologies with the problem-solving knowledge.That way, the knowledge of the wiki can be transparently accessed by SPARQL queries. Ad-ditionally, more expressive knowledge—rules, decision trees, and fault models—is used forautomated reasoning.
For larger application projects, we introduce the concept of wiki masters as a unified knowl-edge engineering metaphor that helps building (large) knowledge bases within a wiki. Here, theknowledge is distributed over the wiki system in servant articles, but is (virtually) joined by asmall number of master articles. Alternative variants of masters can be defined declaratively bythe wiki users.
In the last years, we gained experience in developing knowledge bases using the sketchedSemantic Wiki approach. We discuss the most relevant reflections in the following:
1. Flexible organization of the knowledge: Semantic Wikis free the users from a predefinedorganization of the content. As the only requirement, a Semantic Wiki requires the articleas a logical organization unit, i.e., content is structured by distributing it over wiki articles.In consequence, a project is in no way restricted in terms of dividing the knowledge intological units, but rather permits any structure as long as it fits into the partitioning ofseparate articles.
In a medical project, for example, the application knowledge was structured according totheir cardinal symptoms, for example neurological problems, chest pain, etc. This parti-tioning seemed reasonable with respect to the applied knowledge representation, becausefor each cardinal symptom one or more heuristic decision trees were defined subsequently.In other projects, the wiki implemented a more solution-oriented organization, i.e., for
9
each solution a separate article was defined, containing the tacit knowledge as well as thecorresponding problem-solving knowledge for the particular solution.
2. Continuous representation of multimodal knowledge: When compared to classic knowl-edge engineering tools, a Semantic Wiki offers a flexible integration of various types ofknowledge. That way, we are able to simply combine tacit knowledge—such as text andmultimedia—with more explicit forms of knowledge—such as rules and fault models.Therefore, a Semantic Wiki provides an appropriate technical basis for engineering onthe knowledge formalization continuum. We experienced the combined representation oftacit and explicit knowledge to be very beneficial, since tacit knowledge can serve withinthe project in various ways: (a) as startup documents at the beginning of a project toinformally collect knowledge about the domain, (b) as documentation of the knowledgeengineering process including the comments on the design decisions taken and notes forfuture enhancements, (c) as explanation for the formal counter-part defined in problem-solving markup, and (d) as pursuing information for concepts represented by the article.
For example, in a medical project the knowledge was originally formalized in a graph-based notion by MS-Visio documents. These documents were attached as underlyingtacit knowledge explaining the decision tree representation, that was used during the for-malization phase. Due to the version control of the wiki, older versions of the attacheddocuments as well as of the wiki articles can be reviewed and compared to the currentstate at any time. Moreover, the articles incorporated an implicit documentation of thedevelopment process.
3. Simple administration and rights management: In the past, development tools usu-ally required the installation of proprietary software on the client-side. Web-based soft-ware, such as wikis, only require the availability of a standard web-browser and an inter-net/intranet connection on the client side. As an additional benefit of web-based software,knowledge engineers are not limited to the particular computer that has the software in-stalled, but are able to start and continue the development process on any computer witha browser and an (internet) connection to the wiki server.
Due to the built-in rights management, wikis allow for a fine-grained setting of the read/writeaccess of the articles. In this manner, some parts of the wiki can be closed for public ac-cess, for example, when the development process is not finished for these parts or forarticles, that document administrative content. Finally, any content stored in the wiki—knowledge as well as data—is held under version control, and changes and revisions canbe safely performed.
The conceptual model of Semantic Wikis combining ontologies and problem-solving knowl-edge is described in (Baumeister et al., 2010). Appropriate markups for the effective and ex-tensible knowledge acquisition are introduced in (Baumeister and Puppe, 2008; Reutelshoeferet al., 2009b).
10
4 Quality in Intelligent Systems
Application-driven domain ontologies build the knowledge backbone of advanced intelligentsystems. In the previous section, we briefly introduced the Semantic Wiki KnowWE as anexample of an advanced knowledge management system. Here, the fundamental concepts arealso linked by an underlying ontology. Additionally, we see that ontologies are combined withfurther types of knowledge. The most prominent example is the combination of ontologies withlogic-based rules, as proposed by recent developments of the Semantic Web Stack.
When putting developed systems into practice, it is essential to evaluate their quality. Knowl-edge engineering research provides verification and validation methods as objective measuresfor the quality evaluation. We investigated advanced verification and validation methods thattake heterogenous knowledge and practical requirements into account. By verification, we de-note the detection of anomalies that disagree with a (logical) specification.
4.1 Verification of Ontologies with Rules
In general, when OWL ontologies and rules are combined, the detection of all anomalies is anundecidable task. Therefore, we investigate methods that rely on a pattern-based approach, thustrying to find occurrences of known anomalies in the knowledge. By verification, we under-stand the syntactic analysis of ontologies at the symbolic level for detecting anomalies. Thework is based on prior research on the evaluation of ontologies introduced by (Gómez-Pérez,2001) and research on the verification of rules, for instance described by (Preece et al., 1992).However, the combination of taxonomic and other ontological knowledge with a rule exten-sion induces new evaluation issues, that can cause redundant or even inconsistent behavior. Forexample, an obvious redundancy may be due to the coexistence of the taxonomic relation Asub-class-of B and the rule A → B. We contribute to this work by extending classic measuresby novel anomalies that result from the combination of rule-based and ontological knowledge.It is important to notice that there exists no final enumeration of anomalies, but new anomaliesarise due to application-specific requirements. Therefore, we introduce the declarative spec-ification of anomalies by the new language Datalog*, that allows for flexibly including newand application-relevant anomalies. In detail, we investigate the implications and problems thatemerge from rule definitions in combination with some of the following ontological descriptions:(a) class relations like subclass, complement, disjointness and (b) basic property characteristicslike transitivity, symmetry, ranges and domains, and cardinality restrictions.
The detection of anomalies in ontologies with rules is discussed in (Baumeister and Seipel,2010) and (Baumeister and Seipel, 2006), whereas (Baumeister et al., 2007) also considers rea-sonable refactoring methods to eliminate found anomalies.
4.2 Advanced Validation and Visualization
Critical application domains require the elaborate and thoughtful validation of the knowledgebases before deployment. Empirical testing denotes the most popular validation technique,where predefined test cases are used to simulate and review the correct behavior of the sys-
11
tem. We motivate that the classic notions of a test case and corresponding measures are notsufficient in many (industrial) application scenarios. Today, typical knowledge systems are• interactive, i.e., providing an adaptive interview with the user to allow for effective problem–
solving with a minimal amount of user input, and• anytime, i.e., from an (early) point of the problem–solving process the systems are able
to provide (preliminary) solutions to the user’s problem; the quality/detail of the solutionsusually improves with further inputs.
We contribute enhanced notions of a test case, that generalize the standard test case to a ratedtest case, where competing solutions can be ordered by their rating state. We also introducethe notion of a sequential test case as a further generalization of the rated test case, wheredistinct episodes of a particular test case can explicitly be represented. For those extensionswe introduce appropriate adaptations of the measures precision and recall. Whenever a testcase fails, the effectiveness of inspecting the problem becomes an important issue. The novelvisualization technique DDTree (dialog/derivation tree) is introduced, that combines strategicand derivation knowledge in a graphical manner, and that is successfully applied in a number ofknowledge engineering projects. DDTrees allow for an intuitive and compact depiction of testcases and they support the manual inspection of erroneous cases by coloring metaphors. Thevisualization demonstrated its usefulness during debugging sessions of knowledge bases andtest cases, respectively. Moreover, it provides an intuitive overview of the validity of the entiretest suite. Advanced empirical testing methods are described in (Baumeister, 2009).
4.3 Grey-Box Robustness Testing
As described in the previous section, empirical testing checks the correct behavior of a knowl-edge base by running test cases. Only little research is available, however, that considers the va-lidity of the knowledge in noisy environments. The validation of knowledge bases with respectto noisy or incomplete knowledge is called robustness testing (Groot et al., 2000). Robustnesstesting performs a series of empirical test runs having a varying input or knowledge base quality.In the context of intelligent systems on the web, e.g., Semantic Web applications, the robustnessis a very important issue, since these applications are intended to be used by random users.
We contributed to the research on robustness testing by introducing grey-box testing tech-niques (Baumeister et al., 2006). Grey-box testing incorporates background knowledge to makethe results of the tests more realistic with respect to the expected application environment. Withthese extensions, we are able to process knowledge concerning the ambivalence of user inputsand the dependency between different inputs. In consequence, more realistic degradation studiesare conducted even for knowledge bases with an interactive interview structure.
5 Applications and Outlook
The introduction of the knowledge formalization continuum and extensible Semantic Wikis de-notes a significant improvement when compared to previous knowledge engineering approaches.Due to the flexible approach of the knowledge formalization and the availability of an appro-priate tool, we are now able to model and combine problem-solving knowledge at different
12
formalization levels. The conceptual approach of the knowledge formalization continuum andits implementation by the Semantic Wiki KnowWE is currently used in a number of (partlyindustrial, partly academic) projects, ranging from simple recommender systems to complexdecision-support systems for technical and medical devices. For example, KnowWE providesa technical platform to support a biological community within the BIOLOG Wissen4 project(formerly LaDy). BIOLOG Wissen serves as a web-based application for the collaborative con-struction and use of a decision-support system for landscape diversity. It aims to integrate knowl-edge on causal dependencies of stakeholders, relevant statistical data, and multimedia content.We refer the interested reader to (Nadrowski et al., 2008) for more details. In another recentproject, KnowWE is extended by diagnostic workflow knowledge in the context of the CliWEproject5. By this extension, the wiki is used to collaboratively develop clinical guidelines, thatare integrated as compiled knowledge bases into next-generation medical devices. A first pro-totype of this extension is reported in Hatko et al. (Hatko et al., 2009). Further applications ofthe presented approach have been developed in the technical domain for the diagnosis of spe-cial purpose vehicles. In the historical domain, KnowWE serves as a e-Learning platform forrepresenting and teaching knowledge about ancient greek history (Reutelshoefer et al., 2010).
In the future, current methods for the evolution of ontologies and problem-solving knowl-edge need to be re-considered in the light of the different faces of knowledge. Especially ina distributed environment like a Semantic Wiki, the included knowledge is likely to undergocontinuous modifications. Adapted evaluation methods and corresponding refactorings of theknowledge across the entire repository will help to understand and manage even complex tasks.First steps into this direction (Baumeister and Nalepa, 2009; Reutelshoefer et al., 2009a) showpromising results.
References
Jürgen Angele, Dieter Fensel, Dieter Landes, and Rudi Studer. Developing knowledge-basedsystems with MIKE. Automated Software Engineering: An International Journal, 5(4):389–418, October 1998.
Marc Ayel and Jean-Pierre Laurent. Validation, Verification and Test of Knowledge-Based Sys-tems. Wiley, 1991.
Joachim Baumeister. Advanced measures for empirical testing. In FLAIRS’09: Proceedingsof the 22th International Florida Artificial Intelligence Research Society Conference, pages378–383. AAAI Press, 2009.
Joachim Baumeister and Grzegorz J. Nalepa. Verification of distributed knowledge in semanticknowledge wikis. In FLAIRS’09: Proceedings of the 22th International Florida ArtificialIntelligence Research Society Conference, pages 384–389. AAAI Press, 2009.
4BIOLOG is funded by the German Federal Ministry of Education and Research from 2007-2009 (final fundingphase).
5CliWE (Clinical Wiki Environment) is funded by Drägerwerk, Germany and runs from 2009-2012.
13
Joachim Baumeister and Frank Puppe. Web-based knowledge engineering with knowledgewikis. In Proceedings of Symbiotic Relationships between Semantic Web and KnowledgeEngineering (AAAI 2008 Spring Symposium), 2008.
Joachim Baumeister and Dietmar Seipel. Verification and refactoring of ontologies with rules.In EKAW’06: Proceedings of the 15th International Conference on Knowledge Engineeringand Knowledge Management, pages 82–95, Berlin, 2006. Springer.
Joachim Baumeister and Dietmar Seipel. Anomalies in ontologies with rules. Web Semantics:Science, Services and Agents on the World Wide Web, 8(1):55–68, 2010.
Joachim Baumeister, Jürgen Bregenzer, and Frank Puppe. Gray box robustness testing of rulesystems. In KI’06: Proceedings of the 29th Annual German Conference on Artificial Intelli-gence, LNAI 4314, pages 346–360. Springer, 2006.
Joachim Baumeister, Thomas Kleemann, and Dietmar Seipel. Towards the verification of on-tologies with rules. In FLAIRS’07: Proceedings of the 20th International Florida ArtificialIntelligence Research Society Conference, pages 524–529, 2007.
Joachim Baumeister, Martina Menge, and Frank Puppe. Visualization techniques for the eval-uation of knowledge systems. In FLAIRS’08: Proceedings of the 21th International FloridaArtificial Intelligence Research Society Conference, pages 329–334. AAAI Press, 2008.
Joachim Baumeister, Jochen Reutelshoefer, and Frank Puppe. Continuous knowledge engineer-ing with semantic wikis. In CMS’09: Proceedings of 7th Conference on Computer Meth-ods and Systems (Knowledge Engineering and Intelligent Systems), pages 163–168. Opro-gramowanie Naukowo-Techniczne, 2009.
Joachim Baumeister, Jochen Reutelshoefer, and Frank Puppe. KnowWE: A semantic wiki forknowledge engineering. Applied Intelligence, to appear, 2010.
Asunción Gómez-Pérez. Evaluation of ontologies. International Journal of Intelligent Systems,16(3):391–409, 2001.
Perry Groot, Frank van Harmelen, and Annette ten Teije. Torture tests: A quantitative analysisfor the robustness of knowledge-based systems. In Knowledge Acquisition, Modeling andManagement, LNAI 1319, pages 403–418, Berlin, 2000. Springer.
Perry Groot, Annette ten Teije, and Frank van Harmelen. A quantitative analysis of the robust-ness of knowledge-based systems through degradation studies. Knowledge and InformationSystems, 7(2):224–245, 2003.
Reinhard Hatko, Joachim Baumeister, and Frank Puppe. Diaflux: Diagnostic flows in wikis. InFGWM’09: Proceedings of German Workshop of Knowledge and Experience Management(at LWA’09), 2009.
Ian Horrocks, Bijan Parsia, Peter Patel-Schneider, and James Hendler. Semantic web archi-tecture: Stack or two towers? In Francois Fages and Sylvain Soliman, editors, Principles
14
and Practice of Semantic Web Reasoning (PPSWR), number 3703 in LNCS, pages 37–41.Springer, 2005.
Matthias Hüttig, Georg Buscher, Thomas Menzel, Wolfgang Scheppach, Frank Puppe, andHans-Peter Buscher. A diagnostic expert system for structured reports, quality assessment,and training of residents in sonography. Medizinische Klinik, 3:117–22, 2004.
Mikio Kimura, Mitsuo Sakamoto, Takuya Adachi, and Hiroko Sagara. Diagnosis of febrileillnesses in returned travelers using the PC software GIDEON. Travel Medicine and InfectiousDisease, 3(3):157–160, 2005.
William Lidwell, Kritina Holden, and Jill Butler. Universal Principles of Design. RockportPublishers, October 2003. ISBN 1592530079.
Stefan Mersmann and Michel Dojat. SmartCaretm - automated clinical guidelines in criticalcare. In ECAI’04/PAIS’04: Proceedings of the 16th European Conference on Artificial Intel-ligence, including Prestigious Applications of Intelligent Systems, pages 745–749, Valencia,Spain, 2004. IOS Press.
Robert Milne and Charlie Nicol. TIGER: Continuous diagnosis of gas turbines. In ECAI’00:Proceedings of the 14th European Conference on Artificial Intelligence, Berlin, Germany,2000.
Karin Nadrowski, Joachim Baumeister, and Volkmar Wolters. LaDy: Knowledge Wiki zurkollaborativen und wissensbasierten Entscheidungshilfe zu Umweltveränderung und Biodi-versität. Naturschutz und Biologische Vielfalt, 60:171–176, 2008.
Dang Duc Nghia and Frank Puppe. Hybrides, skalierbares Diagnosesystem für freie Kfz-Werkstätten. KI, 23(2):31–37, 2009.
Alun Preece, Rajjan Shinghal, and Aida Batarekh. Principles and practice in verifying rule-basedsystems. The Knowledge Engineering Review, 7 (2):115–141, 1992.
Jochen Reutelshoefer, Joachim Baumeister, and Frank Puppe. A data structure for the refactor-ing of multimodal knowledge. In KESE’09: 5th Workshop on Knowledge Engineering andSoftware Engineering (CEUR proceedings 486), Paderborn, 2009a.
Jochen Reutelshoefer, Florian Lemmerich, Fabian Haupt, and Joachim Baumeister. An exten-sible semantic wiki architecture. In SemWiki’09: Fourth Workshop on Semantic Wikis – TheSemantic Wiki Web (CEUR proceedings 464), 2009b.
Jochen Reutelshoefer, Florian Lemmerich, Joachim Baumeister, Jorit Wintjes, and Lorenz Haas.Taking OWL to athens – semantic web technology takes ancient greek history to students. toappear, 2010.
Sebastian Schaffert, François Bry, Joachim Baumeister, and Malte Kiesel. Semantic wikis. IEEESoftware, 25(4):8–11, 2008.
15
Guus Schreiber, Hans Akkermans, Anjo Anjewierden, Robert de Hoog, Nigel Shadbolt, Wal-ter Van de Velde, and Bob Wielinga. Knowledge Engineering and Management - The Com-monKADS Methodology. MIT Press, 2 edition, 2001.
Wikipedia. Continuum (theory) — wikipedia, the free encyclopedia, 2010. [Online; accessed10-February-2010].
Wordnet. Continuum — lexicon entry, 2010. [Online; accessed 10-February-2010].
16
Publications
Monographs & Special Issues
[1] J. Baumeister. Agile Development of Diagnostic Knowledge Systems. IOS Press, AKA,DISKI 284, 2004.
[2] J. Baumeister and G. J. Nalepa, editors. Special Issue on Knowledge and Software Engi-neering for Intelligent Systems, to appear. International Journal of Knowledge Engineeringand Data Mining (IJKEDM), 2010.
Proceedings
[1] J. Baumeister and D. Seipel, editors. KESE: 1st Workshop on Knowledge Engineering andSoftware Engineering. Workshop notes of 28th Annual German Conference on ArtificialIntelligence (KI-2005), Koblenz, Germany, 2005.
[2] J. Baumeister and D. Seipel, editors. KESE: 2nd Workshop on Knowledge Engineering andSoftware Engineering. Workshop notes of 29th Annual German Conference on ArtificialIntelligence (KI-2006), Bremen, Germany, 2006.
[3] J. Baumeister and M. Schaaf, editors. Proceedings of the Workshop: ’Knowledge and Ex-perience Management’ (German SIG meeting, FGWM). 2007.
[4] J. Baumeister and D. Seipel, editors. KESE: 3rd Workshop on Knowledge Engineering andSoftware Engineering. Workshop notes of 30th Annual German Conference on ArtificialIntelligence (KI-2007), CEUR Proceedings 282, Osnabrück, Germany, 2007.
[5] J. Baumeister and M. Atzmueller, editors. Lernen, Wissen und Adaptivität. UniversityWürzburg, Computer Science, TR 448, 2008.
[6] J. Baumeister and N. Müller, editors. Proceedings of the Workshop: ’Knowledge and Expe-rience Management’ (German SIG meeting, FGWM). 2008.
[7] G. J. Nalepa and J. Baumeister, editors. KESE: 4th Workshop on Knowledge Engineeringand Software Engineering. Workshop notes of 31th Annual German Conference on ArtificialIntelligence (KI-2008), CEUR Proceedings 425, Kaiserslautern, Germany, 2008.
[8] J. Baumeister and G. J. Nalepa, editors. KESE: 5th Workshop on Knowledge Engineeringand Software Engineering. Workshop notes of 32nd Annual German Conference on Artifi-cial Intelligence (KI-2009), CEUR Proceedings 486, Paderborn, Germany, 2009.
17
Book Chapters
[1] M. Neumann and J. Baumeister. A Rule–Based vs. a Set-Covering Implementation of theKnowledge System LIMPACT and its Significance for Maintenance and Discovery of Eco-logical Knowledge. In Modelling Community Structure in Freshwater Ecosystems, pages401–410. Springer, Berlin, 2005.
[2] J. Baumeister, D. Seipel, and F. Puppe. Agile Development of Rule Systems. In Giurca,Gasevic, and Taveter, editors, Handbook of Research on Emerging Rule-Based Languagesand Technologies: Open Solutions and Approaches. IGI Publishing, 2009.
[3] S. Schaffert, F. Bry, J. Baumeister, and M. Kiesel. Semantische Wikis. In A. Blumauer andT. Pellegrini, editors, Social Semantic Web. Springer, 2009.
Journal Articles
[1] M. Neumann, J. Baumeister, M. Liess, and R. Schulz. An Expert System to Estimate thePesticide Contamination of Small Streams using Benthic Macroinvertebrates as Bioindi-cators, Part 2: The Knowledge Base of LIMPACT. Ecological Indicators, 2(4):391–401,2002.
[2] M. Neumann, J. Baumeister, M. Liess, and R. Schulz. LIMPACT: Ein Expertensys-tem zur Abschätzung der Pflanzenschutzmittel-Belastung kleiner Fließgewässer mittelsder Makroinvertebraten-Fauna. Umweltwissenschaften und Schadstoff-Forschung (USWF),3:152–156, 2002.
[3] J. Baumeister, D. Seipel, and F. Puppe. Incremental Development of Diagnostic Set-Covering Models with Therapy Effects. International Journal of Uncertainty, Fuzzinessand Knowledge-Based Systems, 11(Suppl. Issue 2):25–49, 2003.
[4] J. Baumeister, F. Puppe, and D. Seipel. An Agile Process Model for Developing DiagnosticKnowledge Systems. Künstliche Intelligenz, 3/04:12–16, 2004.
[5] M. Neumann, J. Baumeister, and F. Puppe. ILMAX: A System for Managing ExperienceKnowledge in a long–term Study of Stream Ecosystem Regeneration – An Application ofEcological Informatics. Management of Environmental Quality: An International Journal,15(3):306–317, 2004.
[6] D. Seipel and J. Baumeister. Declarative Methods for the Evaluation of Ontologies. Kün-stliche Intelligenz, 4/04:51–57, 2004.
[7] J. Baumeister and D. Seipel. Anfragesprachen für das Semantic Web. Informatik Spektrum,28(1):40–44, 2005.
[8] M. Atzmueller, J. Baumeister, M. Goller, and F. Puppe. A Datagenerator for EvaluatingMachine Learning Methods. Künstliche Intelligenz, 3/06:57–63, 2006.
18
[9] S. Schaffert, F. Bry, J. Baumeister, and M. Kiesel. Semantic Wiki. Informatik Spektrum,30(6):434–439, 2007.
[10] K. Nadrowski, J. Baumeister, and V. Wolters. LaDy: Knowledge Wiki zur kollabora-tiven und wissensbasierten Entscheidungshilfe zu Umweltveränderung und Biodiversität.Naturschutz und Biologische Vielfalt, 60:171–176, 2008.
[11] S. Schaffert, F. Bry, J. Baumeister, and M. Kiesel. Semantic Wikis. IEEE Software,25(4):8–11, 2008.
[12] J. Baumeister, J. Reutelshoefer, and F. Puppe. KnowWE: A Semantic Wiki for KnowledgeEngineering. Applied Intelligence, to appear, 2010.
[13] J. Baumeister and D. Seipel. Anomalies in Ontologies with Rules. Web Semantics: Science,Services and Agents on the World Wide Web, 8(1):55–68, 2010.
Submitted Journal Articles
[1] J. Baumeister. Advanced Empirical Testing. Knowledge-Based Systems, submitted Mar 26,2009.
[2] J. Baumeister and M. Freiberg. Knowledge Visualization for Evaluation Tasks. Knowledgeand Information Systems, submitted Jan 15, 2010.
International Conferences
[1] J. Baumeister, M. Atzmueller, and F. Puppe. Inductive Learning for Case-Based Diagnosiswith Multiple Faults. In ECCBR’02: Proceedings of the 6th European Conference onCase-Based Reasoning, LNAI 2416, pages 28–42. Springer, Berlin, 2002.
[2] M. Neumann and J. Baumeister. A Rule-Based vs. a Model-Based Implementation ofthe Knowledge System LIMPACT and its Significance for Maintenance and Discovery ofEcological Knowledge. In ISEI’02: Proceedings of the 3rd Conference of the InternationalSociety for Ecological Informatics. 2002.
[3] M. Atzmueller, J. Baumeister, and F. Puppe. Inductive Learning of Simple DiagnosticScores. In ISMDA’03: Proceedings of the International Symposium of Medical Data Anal-ysis, LNCS 2868, pages 23–30. Springer, Berlin, 2003.
[4] M. Neumann, J. Baumeister, and F. Puppe. ILMAX: A System for Managing ExperienceKnowledge in a long–term Study of Stream Ecosystem Regeneration – an Applicationof Ecological Informatics. In ITEE’2003: Proceedings of the 1st International NAISOSymposium on Information Technologies in Environmental Engineering. 2003.
19
[5] M. Atzmueller, W. Shi, J. Baumeister, F. Puppe, and J. A. Barnden. Case-Based Ap-proaches for Diagnosing Multiple Disorders. In FLAIRS’04: Proceedings of the 17th In-ternational Florida Artificial Intelligence Research Society Conference, pages 154–159.2004.
[6] J. Baumeister, D. Seipel, and F. Puppe. Refactoring Methods for Knowledge Bases. InEKAW’04: Engineering Knowledge in the Age of the Semantic Web: 14th InternationalConference, LNAI 3257, pages 157–171. Springer, Berlin, 2004.
[7] J. Baumeister, D. Seipel, and F. Puppe. Using Automated Tests and Restructuring Methodsfor an Agile Development of Diagnostic Knowledge Systems. In FLAIRS’04: Proceedingsof the 17th International Florida Artificial Intelligence Research Society Conference, pages319–324. 2004.
[8] W. Shi, J. A. Barnden, J. Baumeister, and M. Atzmueller. An Intelligent Diagnosis SystemHandling Multiple Disorders. In ICIIP’04: Proceedings of the International Conferenceon Intelligent Information Processing. 2004.
[9] M. Atzmueller, J. Baumeister, A. Hemsing, E.-J. Richter, and F. Puppe. Subgroup Miningfor Interactive Knowledge Refinement. In AIME’05: Proceedings of the 10th Conferenceon Artificial Intelligence in Medicine, LNAI 3581, pages 453–462. Springer, Berlin, 2005.
[10] M. Atzmueller, J. Baumeister, and F. Puppe. Quality Measures and Semi-Automatic Min-ing of Diagnostic Rule Bases. In INAP/WLP’04: Applications of Declarative Programmingand Knowledge Management (selected papers), LNAI 3392, pages 65–78. Springer, Berlin,2005.
[11] J. Baumeister, R. Knauf, and F. Puppe. Semi-Automatic Generation of Test Cases byCase Morphing. In FLAIRS’05: Proceedings of the 18th International Florida ArtificialIntelligence Research Society Conference, pages 814–815. AAAI Press, 2005.
[12] J. Baumeister and D. Seipel. Smelly Owls – Design Anomalies in Ontologies. InFLAIRS’05: Proceedings of the 18th International Florida Artificial Intelligence ResearchSociety Conference, pages 215–220. AAAI Press, 2005.
[13] G. Buscher, J. Baumeister, and F. P. D. Seipel. User–Centered Consultation by a Societyof Agents. In K-CAP ’05: Proceedings of the 3rd International Conference on KnowledgeCapture, pages 27–34. ACM, New York, NY, USA, 2005.
[14] M. Hopfner, D. Seipel, and J. Baumeister. A PROLOG Tool for Slicing Source Code. InWLP’05: Proceedings of the 19th Workshop on (Constraint) Logic Programming. 2005.
[15] D. Seipel, M. Hopfner, and J. Baumeister. Declarative Querying and Visualizing Knowl-edge Bases in XML. In INAP/WLP’04: Applications of Declarative Programming andKnowledge Management (selected papers), LNAI 3392, pages 16–31. Springer, Berlin,2005.
20
[16] W. Shi, J. A. Barnden, M. Atzmueller, and J. Baumeister. An Intelligent Diagnosis SystemHandling Multiple Disorders. IFIP International Federation for Information Processing,163:421–430, 2005.
[17] M. Atzmueller, J. Baumeister, and F. Puppe. Semi-Automatic Learning of Simple Diagnos-tic Scores utilizing Complexity Measures. Artificial Intelligence in Medicine, 37(1):19–30,2006.
[18] J. Baumeister, M. Atzmueller, P. Kluegl, and F. Puppe. Conservative and Creative Strate-gies for the Refinement of Scoring Rules. In FLAIRS’06: Proceedings of the 19th Interna-tional Florida Artificial Intelligence Research Society Conference, pages 408–413. 2006.
[19] J. Baumeister, M. Atzmueller, and F. Puppe. Introspective Subgroup Analysis for Interac-tive Knowledge Refinement. In FLAIRS’06: Proceedings of the 19th International FloridaArtificial Intelligence Research Society Conference, pages 402–407. 2006.
[20] J. Baumeister, J. Bregenzer, and F. Puppe. Gray Box Robustness Testing of Rule Systems.In KI’06: Proceedings of the 29th Annual German Conference on Artificial Intelligence,LNAI 4314, pages 346–360. Springer, 2006.
[21] J. Baumeister and D. Seipel. Verification and Refactoring of Ontologies With Rules. InEKAW’06: Proceedings of the 15th International Conference on Knowledge Engineeringand Knowledge Management, pages 82–95. Springer, Berlin, 2006.
[22] M. Atzmueller, J. Baumeister, P. Klügl, and F. Puppe. Rapid Knowledge Capture UsingSubgroup Discovery with Incremental Refinement. In K-CAP ’07: Proceedings of the4th International Conference on Knowledge Capture, pages 31–38. ACM, New York, NY,USA, 2007.
[23] M. Atzmueller, J. Baumeister, and F. Puppe. Pattern–Constrained Test Case Generation. InFLAIRS’07: Proceedings of the 20th International Florida Artificial Intelligence ResearchSociety Conference, pages 518–523. 2007.
[24] J. Baumeister, T. Kleemann, and D. Seipel. Towards the Verification of Ontologies withRules. In FLAIRS’07: Proceedings of the 20th International Florida Artificial IntelligenceResearch Society Conference, pages 524–529. 2007.
[25] J. Baumeister, J. Reutelshoefer, and F. Puppe. KnowWE – Community–based KnowledgeCapture with Knowledge Wikis. In K-CAP ’07: Proceedings of the 4th InternationalConference on Knowledge Capture, pages 189–190. ACM, New York, NY, USA, 2007.
[26] J. Baumeister, M. Menge, and F. Puppe. Visualization Techniques for the Evaluation ofKnowledge Systems. In FLAIRS’08: Proceedings of the 21th International Florida Artifi-cial Intelligence Research Society Conference, pages 329–334. AAAI Press, 2008.
[27] D. Seipel and J. Baumeister. Declarative Specification and Interpretation of Rule-BasedSystems. In FLAIRS’08: Proceedings of the 21th International Florida Artificial Intelli-gence Research Society Conference, pages 359–364. AAAI Press, 2008.
21
[28] J. Baumeister. Advanced Measures for Empirical Testing. In FLAIRS’09: Proceedings ofthe 22th International Florida Artificial Intelligence Research Society Conference, pages378–383. AAAI Press, 2009.
[29] J. Baumeister and G. J. Nalepa. Verification of Distributed Knowledge in Semantic Knowl-edge Wikis. In FLAIRS’09: Proceedings of the 22th International Florida Artificial Intel-ligence Research Society Conference, pages 384–389. AAAI Press, 2009.
[30] J. Baumeister, J. Reutelshoefer, and F. Puppe. Continuous Knowledge Engineering withSemantic Wikis. In CMS’09: Proceedings of 7th Conference on Computer Methodsand Systems (Knowledge Engineering and Intelligent Systems), pages 163–168. Opro-gramowanie Naukowo-Techniczne, 2009.
National Conferences
[1] M. Neumann, J. Baumeister, M. Liess, and R. Schulz. LIMPACT: Ein Expertensys-tem zur Abschätzung der Pflanzenschutzmittel-Belastung kleiner Fließgewässer mittels derMakroinvertebraten-Fauna. In SETAC-GLB’01: Proceedings der SETAC-GLB Tagung.2001.
[2] M. Atzmueller, J. Baumeister, and F. Puppe. Evaluation of two Strategies for Case-BasedDiagnosis handling Multiple Faults. In WM’03: Proceedings of the 2nd Conference "Pro-fessionelles Wissensmanagement". 2003.
[3] M. Atzmueller, J. Baumeister, A. Hemsing, E.-J. Richter, and F. Puppe. Using SubgroupMining for the Refinement of Knowledge Systems. In GfKl’05: Proceedings of the 29thAnnual Conference of the German Classification Society. 2005.
[4] M. Atzmueller, J. Baumeister, and F. Puppe. Exemplifying Subgroup Mining Results forInteractive Knowledge Refinement. In LIT’05: Proceedings of the 13th Leipziger InformatikTage, LNI P-72, pages 101–106. 2005.
[5] J. Baumeister, J. Bregenzer, and F. Puppe. A Methodological View on Robustness Testing ofRule-Based Knowledge Systems. In LIT’05: Proceedings of the 13th Leipziger InformatikTage, LNI P-72, pages 131–138. 2005.
[6] S. Schulz, J. Baumeister, A. Crössmann, F. Puppe, and P. Pauli. www.icd-forum.de -Ein internetbasiertes Programm für Patienten mit implantiertem Cardioverter Defibrillator.Beiträge zur Gesundheitspsychologie, Gmünder Hochschulreihe Nr. 29, 2007.
[7] S. M. Schulz, J. Baumeister, G. W. Alpers, A. Crössmann, H. Neuser, F. Puppe, and P. Pauli.An Internet-based Intervention to Reduce Cardiac Fear in Patients with Implantable Car-dioverter Defibrillator. Abstract in Conference Proceedings of the 9. Jahrestagung derGesellschaft für Angstforschung, 2007.
22
Workshops
[1] J. Baumeister, D. Seipel, and F. Puppe. Incremental Development of Diagnostic Set–Covering Models with Therapy Effects. In Proceedings of the KI-2001 Workshop on Un-certainty in Artificial Intelligence. 2001.
[2] J. Baumeister and D. Seipel. Diagnostic Reasoning with Multilevel Set-Covering Models.In DX’02: Proceedings of the 13th International Workshop on Principles of Diagnosis.2002.
[3] A. Hörnlein, J. Baumeister, and F. Puppe. Modelle für die Generierung von Folgesitzungenzur Therapieüberwachung in fallbasierten Trainingssystemen. In CBT’03: Proceedingszum 7. Workshop der GMDS AG Computergestützte Lehr- und Lernsysteme in der Medizin.Shaker, 2003.
[4] K.-W. Lorenz, J. Baumeister, C. Greim, N. Roewer, and F. Puppe. QualiTEE - An Intelli-gent Guidance and Diagnosis System for the Documentation of Transesophageal Echocar-diography Examinations. ESCTAIC’03: Proceedings of the 14th Annual Meeting of theEuropean Society for Computing and Technology in Anaesthesia and Intensive Care, 2003.
[5] N. Bruemmer, J. Baumeister, D. Riewenherm, F. Puppe, and J. Broscheit. Visual Devel-opment of Temporal Patterns for Medical Data Abstraction. In IDAMAP’06: Proceedingsof the Workshop on Intelligent Data Analysis in Biomedicine and Pharmacology, pages37–38. 2006.
[6] J. Baumeister, J. Reutelshoefer, K. Nadrowski, and A. Misok. Using Knowledge Wikis toSupport Scientific Communities. In SCOOP’07: Proceedings of 1st Workshop on ScientificCommunities of Practice. Bremen, Germany, 2007.
[7] J. Baumeister, J. Reutelshoefer, and F. Puppe. Markups for Knowledge Wikis. InSAAKM’07: Proceedings of the Semantic Authoring, Annotation and Knowledge MarkupWorkshop, pages 7–14. Whistler, Canada, 2007.
[8] J. Baumeister and F. Puppe. Web-based Knowledge Engineering with Knowledge Wikis.In Proceedings of Symbiotic Relationships between Semantic Web and Knowledge Engi-neering (AAAI 2008 Spring Symposium). 2008.
[9] J. Baumeister, J. Reutelshoefer, F. Haupt, and K. Nadrowski. Capture and Refactoring inKnowledge Wikis – Coping with the Knowledge Soup. In SCOOP’08: Proceedings of 2ndWorkshop on Scientific Communities of Practice. Bremen, Germany, 2008.
[10] J. Reutelshoefer, J. Baumeister, and F. Puppe. Ad-Hoc Knowledge Engineering with Se-mantic Knowledge Wikis. In SemWiki’08: Proceedings of 3rd Semantic Wiki workshop -The Wiki Way of Semantics (CEUR Proceedings 360). 2008.
[11] J. Baumeister, J. Reutelshoefer, and F. Puppe. Engineering on the Knowledge Formaliza-tion Continuum. In SemWiki’09: Proceedings of 4th Semantic Wiki workshop. 2009.
23
[12] M. Freiberg, J. Baumeister, and F. Puppe. The Usability Stack: Reconsidering UsabilityCriteria regarding Knowledge-Based Systems. In LWA-2009 (Special Track on KnowledgeManagement). 2009.
[13] R. Hatko, J. Baumeister, and F. Puppe. DiaFlux: Diagnostic Flows in Wikis. InFGWM’09: Proceedings of German Workshop of Knowledge and Experience Manage-ment (at LWA’09). 2009.
[14] J. Reutelshoefer, F. Lemmerich, F. Haupt, and J. Baumeister. An Extensible Semantic WikiArchitecture. In SemWiki’09: Fourth Workshop on Semantic Wikis – The Semantic WikiWeb (CEUR proceedings 464). 2009.
Miscellaneous Publications
[1] J. Broscheit, K.-W. Lorenz, J. Baumeister, P. Kranke, and C. Greim. Determinants of Alarm-Rate and the Potential of Intelligent Monitoring Systems in Routine Anaesthesiological Use.ESCTAIC’02: Proceedings of the 13th Annual Meeting of the European Society for Com-puting and Technology in Anaesthesia and Intensive Care, 2002.
[2] K.-W. Lorenz, C. Greim, J. Baumeister, F. Puppe, and N. Roewer. EchoDOC - intelligentesDiagnose- und Dokumentationssystem für die TEE-Untersuchung. DGAI’04: Proceedingsof the 51st Annual Meeting of the German Association of Anaesthesiologists, 2004.
[3] N. Bruemmer, J. Baumeister, and F. Puppe. Relations between Visual and Textual Represen-tations of Temporal Patterns for Medical Data Abstraction. Technical Report 394, ComputerScience, University of Würzburg, Germany, 2006.
[4] G. Buscher, J. Baumeister, F. Puppe, and D. Seipel. Semi-Distributed Development ofAgent-Based Consultation Systems. In EKAW’06: Poster–Proceedings of the 15th Inter-national Conference on Knowledge Engineering and Knowledge Management. 2006.
[5] M. Freiberg and J. Baumeister. A Survey on Usability Evaluation Techniques and an Anal-ysis of their actual Application. Technical Report 450, Computer Science, University ofWürzburg, Germany, 2008.
24
