Computer-supported analysis of scientific measurements

In the past decade, large-scale databases and knowledge bases have become available to researchers working in a range of scientific disciplines. In many cases these databases and knowledge bases contain measurements of properties of physical objects which have been obtained in experiments or at observation sites. As examples, one can think of crystallographic databases with molecular structures and property databases in materials science. These large collections of measurements, which will be called measurement bases, form interesting resources for scientific research. By analyzing the contents of a measurement base, one may be able to find patterns that are of practical and theoretical importance. With the use of measurement bases as a resource for scientific inquiry questions arise about the quality of the data being analyzed. In particular, the occurrence of conflicts and systematic errors raises doubts about the reliability of a measurement base and compromises any patterns found in it. On the other hand, conflicts and systematic errors may be interesting patterns in themselves and warrant further investigation. These considerations motivate the topic that will be addressed in this thesis: the development of systematic methods for detecting and resolving conflicts and identifying systematic errors in measurement bases. These measurement analysis (MA) methods are implemented in a computer system supporting the user of the measurement base. Despite their obvious importance, MA methods for con ict resolution and error identification have been largely unexplored thus far. Statistical methods assist in detecting conflicts between measurements, but do not offer much help in resolving them. In addition, they focus on random errors and largely neglect the problem of systematic errors. In contrast with statistical methods, the methods developed in this thesis draw upon knowledge about the domain under study. More specically, they are model-based MA methods since this knowledge takes the form of models of the physical systems investigated in the experiments. Chapter 2 provides a framework for conflict detection, conflict resolution, and error identification by relating conflicts and errors to the experiments in which measurements are performed. An experiment is conceptualized as the activity of creating and sustaining a controlled physical system, referred to as an experimental system. Measurement of a certain property amounts to an empirical determination of the value of a quantity of the experimental system. A conflict between two property measurements can be explained by reference to structural dierences between the experimental systems on which the measurements are performed and dierences between the experimental conditions. A systematic error in a property measurement can be predicted from differences between the structure of the experimental system actually investigated and the structure of a hypothetical ideal experimental system, and from differences between the actual experimental conditions and the ideal experimental conditions. Experimental systems are modeled by means of dierential equations. More particularly, qualitative differential equations (QDEs) are used, since much of the knowledge about the systems will be qualitative in nature, especially when certain idealized experimental circumstances cannot be realized. Given this representation, the methods for model-based conflict resolution and error identification can be elaborated by means of two techniques from qualitative reasoning (QR): qualitative simulation and comparative analysis. Chapter 3 reviews the well-known qualitative simulation algorithm QSIM which is used to infer the possible qualitative behaviors of an experimental system from an initial state representing the experimental conditions. QSIM has a solid foundation in mathematics which allows one to prove certain properties of the algorithm. In particular, all genuine possible behaviors of an experimental system are inferred, but occasionally spurious behaviors as well. Chapter 4 introduces the CEC* algorithm for comparative envisionment construction which allows one to compare a model and behavior of one experimental system with a model and behavior of another system. CEC* improves upon existing comparative analysis algorithms in that it is able to compare structurally dierent experimental systems. Starting from initial relative values for some of the quantities, CEC* derives possible comparative behaviors of the two systems. A comparative behavior describes the differential dynamics of the experimental systems being compared in a qualitative manner. An explanatory comparative analysis finds possible causes of differences in the response of two systems, whereas a predictive comparative analysis finds possible consequences of differences in the initial conditions. As in QSIM, certain guarantees can be given on the outcome of a comparative analysis. All genuine comparative behaviors of the systems will be inferred, but sometimes spurious comparative behaviors as well. In chapter 5 the techniques for qualitative simulation and comparative analysis are combined to formalize the tasks of con ict resolution and comparative analysis. In addition, a standard statistical method for con ict detection is given. Conflict resolution is defined as an explanatory comparative analysis, where the resulting comparative behaviors represent possible explanations of the conflict between two measurements. Error identification is defined as a predictive comparative analysis with the comparative behaviors pointing at possible systematic errors in the measurement. From the formal properties of QSIM and CEC* it can be proven that all genuine explanations of a conflict and all genuine predictions of a systematic error are found, but that the occurrence of spurious explanations and predictions cannot be excluded. The conflict detection, conflict resolution, and error identification algorithms have been implemented in Common Lisp and together form the KIMA system for Knowledge- Intensive Measurement Analysis. KIMA is built on top of the implementations of QSIM and CEC* and repeatedly calls the main functions of these programs. KIMA has been successfully applied in a case-study on a realistic though simplified problem: the analysis of measurements of the fracture strength of brittle materials obtained in tension tests and four-point bend tests. Chapter 6 reviews basic theories on brittle fracture and fracture testing which underlie the models required by the conflict resolution and error identification algorithms. The results of the case-study are presented in chapter 7. KIMA is shown to be able to reproduce a number of interesting phenomena reported in the literature on tension tests and four-point bend tests. In chapter 8 the MA methods and their application are discussed in the context of related work. In particular, attention is given to different forms of knowledge-based measurement analysis, the use of qualitative knowledge, the relationship between modelbased measurement analysis and model-based diagnosis, the (computer-supported) construction and revision of models of experimental systems, the generality of the methods for conflict resolution and error identication, and the practical use of the methods. The concluding chapter 9 summarizes the two main contributions of the thesis: first, the development of methods for model-based con ict resolution and error identication which supplement conventional statistical analyses; and second, the development of a general technique for comparative analysis which improves upon existing approaches and may prove useful for design, diagnosis, and discovery problems as well. A few directions for further research are indicated and the chapter concludes with a speculative outlook by viewing model-based measurement analysis as a part of future computer-supported discovery environments.