Observation Theory: Estimating the Unknown (edX)

TU Delft’s approach to observation theory is world leading and based on decades of experience in research and teaching in geodesy and the wider geosciences. The theory, however, can be applied to all the engineering sciences where measurements are used to estimate unknown parameters.

The course introduces a standardized approach for parameter estimation, using a functional model (relating the observations to the unknown parameters) and a stochastic model (describing the quality of the observations). Using the concepts of least squares and best linear unbiased estimation (BLUE), parameters are estimated and analyzed in terms of precision and significance.

The course ends with the concept of overall model test, to check the validity of the parameter estimation results using hypothesis testing. Emphasis is given to develop a standardized way to deal with estimation problems. Most of the course effort will be on examples and exercises from different engineering disciplines, especially in the domain of Earth Sciences.

This course is aimed towards Engineering and Earth Sciences students at Bachelor’s, Master’s and postgraduate level.

What you'll learn:

- How to translate real-life estimation problems to easy mathematical models

- Practical understanding of least squares estimation and best linear unbiased estimation, and how to apply these methods

- How to assess and describe the quality of your estimators in the form of precision and confidence interval

- How to check the validity of your estimation results

Prerequisites:

- Calculus (high school level)

- Probability concepts like expectation, variance, the normal distribution and other probability density functions

- Basic matrix manipulation (by hand and with e.g. Python or Matlab)

Syllabus

Week 1: Introduction

Introduction on what is “estimation” and when do we need it? What are the generic sources of uncertainty in observations, and what concepts are needed, e.g. deterministic vs. stochastic parameters, random vs. systematic errors, precision vs. accuracy, bias, and the probability distribution function as a metric of randomness. All the concepts are explained by various practical examples.

Week 2: Mathematical models

Learn how to develop a systematic approach to translate real-life problems into mathematical models in the form of observation-equation system including four fundamental blocks: vector of observations, vector of unknown parameters, linear (or linearized) functional relation between observations and unknowns, and stochastic characteristics of observations in the form of dispersion (or covariance matrix) of the observation vector. As well as discussion on different concepts, such as linear vs. nonlinear models, functional vs. stochastic models, consistent vs. inconsistent models, over/under –determined models, redundancy, and solvability of observation-equation systems. All the aforementioned concepts are explained by various practical examples.

Week 3: Least Squares Estimation (LSE)

Given a mathematical model, how to find an estimate that predicts the observations as close as possible? Introduction to (weighted) least squares estimation (WLSE), its mathematical logic and its main properties. Different applications of WLSE are demonstrated via practical examples, as well as discussion on numerical/computational aspects of applying WLSE.

Week 4: Best Linear Unbiased Estimation (BLUE)

How to find the most precise and accurate estimate in linear models? Introduction to the concept of Best Linear Unbiased Estimation (BLUE), its theory and implication, and its relation to other estimators such as WLSE, maximum likelihood, and minimum variance estimators. The concept of BLUE and its application in various real problems are demonstrated by examples and exercises.

Week 5: How precise is the estimate?

Discussion on how the uncertainty/randomness in observations (depicted by a stochastic model) propagates to the uncertainty/randomness of estimates (depicted by probability density function or covariance matrix of estimators). Introduction to the concept of error propagation and its application in specification of the uncertainty/precision of estimates, inferring confidence intervals or statistical tolerance levels of the results, and describing the expected variability of the results of an estimation. The interpretation of covariance matrices and confidence intervals is discussed and clarified via different examples and exercises.

Week 6: Does the estimate make sense?

Introduction to a probabilistic decision making process (or statistical hypothesis testing) in validating the results of estimation in order to avoid wrong decisions/interpretation of the results. Students learn how to verify the validity of a chosen mathematical model, and how to detect and identify model misspecifications. The concepts are explained by various practical examples.