Fitting to an orthonormal set of functions

3.8 Fitting to an orthonormal set of functions

We now define the columns of the matrix \(\tilde{M}\) as vectors \(\vec{M}_j\) by

\[ \vec{M}_j:= \vect{f_j(x_1) \\ f_j(x_2) \\ \vdots \\ f_j(x_N)} \;. \]

and restrict the calculation to special fit functions \(f_j(x_i)\) with the properties

\begin{equation*} \left\langle \vec{M}_i|\vec{M}_j\right\rangle = \delta_{i,j}\;, \label{Orthonormal1} \end{equation*}

(3.9)

i.e. the fit functions represent an orthonormal set of base vectors for the measured points \(x_{m,i}\).
For such a set of fit functions the matrix \(M^TM\) is just the unity matrix

\[ \tilde{M}^T\tilde{M} = \left(\begin{array}{cccc} 1&0&\cdots&0\\ 0&1& &0\\ \vdots& &\ddots&\vdots\\ 0&0&\cdots&1 \end{array}\right) \;, \]

and Eq. (3.8) reduces to

\begin{equation*} \vec{a}_{opt} = \tilde{M}^T \vec{y}_m \;, \label{OptCon5} \end{equation*}

(3.10)

or written in components

\begin{equation*} a_{j,\;opt} = \left\langle \vec{M}_j| \vec{y}_m\right\rangle \;. \label{OptCon6} \end{equation*}

(3.11)

Combining Eq. (3.11) with Eq. (3.3) we get

\begin{equation*} \vec{y}_{opt} = \sum_{j=1}^J \left\langle \vec{M}_j| \vec{y}_m \right\rangle \vec{M}_j \;. \label{Projection1} \end{equation*}

(3.12)

Eq. (3.12) is very simple but most important. The scalar product \(\left\langle \vec{M}_j| \vec{y}_m \right\rangle \) calculates the projection of the (measured) vector \(\vec{y}_m\) onto a base (function) vector \(\vec{M}_j\). If \(\vec{y}_m\) would be spanned up by the base vectors \(\vec{M}_j\), the calculated optimal vector \(\vec{y}_{opt} = \vec{y}_m\). In general, the base vectors only span up a subspace of all possible measurement results \(\vec{y}_m\), i.e. some components are missing in Eq. (3.12), but still the projections are the best result with respect to the least square error between measured and fitted data.
The main effort for finding a least square fit just by calculating projections, i.e. scalar products, is to find a set of orthonormal functions for a set of measured points \(x_{m,i}\).To get a good, efficient, and useful fit, this functions should span up a large subspace of possible measurement results \(y_{m,i}\) with a least number of fitting parameters;in most cases this is not only a mathematical but more a physical probleme.