Measurement errors and signal preprocessing in spectroscopy

One of the most important issues in multivariate calibration is measurement uncertainty. Although measurement errors are an integral and complex part of chemical measurements, most multivariate calibration methods implicitly assume normally distributed, uncorrelated noise, or attempt to accommodate the error structure through routine preprocessing. Unfortunately, preprocessing and multivariate calibration methods are often used without a thorough knowledge of their effects on the data and its measurement errors. This can lead to inappropriate transformations and sub-optimal regression models. This work looks at strategies for dealing with measurement errors by exploring the characteristics of measurement errors, in particular instrumental methods, and investigating selected techniques that deal with nonidealities in measurement spectra.; This study begins with the development of a systematic approach for characterizing measurement errors through error covariance matrices. Through the visualization of covariance and correlation matrices and bilinear modeling approaches, a better understanding of the contributing factors of the error structure is obtained. This strategy is demonstrated with four different spectroscopic data sets (UV/Vis absorbance, near-infrared (NIR) reflectance, fluorescence emission and short-wave NIR absorbance). The corresponding error covariance matrices can be used in multivariate analysis with maximum likelihood principal components regression (MLPCR), although a frequent difficulty arises with the large size of the error covariance matrix. Wavelet transforms (WT) are shown to be a useful preprocessing step for the compression of these error structures. A careful reduction of wavelet coefficients can also lead to error reduction and improved predictive ability in MLPCR.; One type of measurement noise that MLPCR cannot correct is multiplicative effects, which are directly proportional to the signal. A well-known preprocessing method in NIR spectroscopy, multiplicative signal correction (MSC), is often used to mitigate multiplicative noise in the measurement error structure. This can be a beneficial preprocessing step, although it is demonstrated that MSC can be problematic in some circumstances. A more direct approach to the preprocessing of NIR spectra is to use information on pathlength distributions to account for the effects of light scattering. A novel preprocessing method is proposed to account for these effects by using information from photon time-of-flight measurements.