Chapter 6
Function spaces
Context: Before we dive deeper into the numerical solution of partial differential equations, we need to introduce a mathematical concept: Function spaces, or more concretely Hilbert spaces. Function spaces are useful because they formalize the series expansion and provide access to the coefficients of the expansion through the concept of basis functions.
6.1 Vectors
As an introduction, let us recall the usual Cartesian vectors. We can represent a vector \(\v {a}=(a_1, a_2, a_3)\) as a linear combination of basis vectors \(\hat {e}_1\), \(\hat {e}_2\) and \(\hat {e}_3\), \begin {equation} \v {a} = a_1 \hat {e}_1 + a_2 \hat {e}_2 + a_3 \hat {e}_3. \label {eq:Cartesian-decomposition} \end {equation} The unit vectors \(\hat {e}_1\), \(\hat {e}_2\) and \(\hat {e}_3\) are of course the vectors that span the Cartesian coordinate system. (In previous chapters, we also used the notation \(\hat {x}\equiv \hat {e}_1\), \(\hat {y}\equiv \hat {e}_2\) and \(\hat {z}\equiv \hat {e}_3\).) The numbers \(a_1\), \(a_2\) and \(a_3\) are the components or coordinates of the vector, but also the coefficients multiplying the unit vectors in Eq. \eqref{eq:Cartesian-decomposition}. In this sense, they are identical to the coefficients of the series expansion, with the difference that the \(\hat {e}_i\)s are orthogonal, i.e. \begin {equation} \hat {e}_i\cdot \hat {e}_j = \delta _{ij} \end {equation} where \(\delta _{ij}\) is the Kronecker-\(\delta \). Two Cartesian vectors \(\v {a}\) and \(\v {b}\) are orthogonal if the scalar product between them vanishes: \begin {equation} \v {a}\cdot \v {b} = \sum _i a_i^* b_i = 0 \label {eq:vecscalar} \end {equation} Using the scalar product, we can obtain the components as \(a_i = \v {a}\cdot \hat {e}_i\), which can be interpreted as the project of \(\v {a}\) on the respective basis vector. This is a direct consequence of the orthogonality of the basis vectors \(\hat {e}_i\).
6.2 Functions
In the previous section, we claimed that the basis functions from Chapter 5 are not orthogonal. For this we need an idea for orthogonality of functions. With a definition of a scalar product between two functions, we can then define orthogonality as the vanishing of this scalar product.
We now introduce a scalar product on functions (or function spaces). Given two functions \(g(x)\) and \(f(x)\) on the interval \(x\in [a,b]\), define the scalar product as \begin {equation} (f,g) = \int _a^b \dif x\, f^*(x) g(x), \label {eq:funcscalar} \end {equation} where \(f^*(x)\) is the complex conjugate of \(f(x)\). This scalar product or inner product is a map to a real number with the properties
- Positive definite: \((f,f)\ge 0\) and \((f,f)=0\Leftrightarrow f=0\)
- Sesquilinear: \((\alpha f+\beta g,h)=\alpha ^*(f,h)+\beta ^*(g,h)\) and \((f,\alpha g+\beta h)=\alpha (f,g)+\beta (f,h)\)
- Hermitian: \((f,g)=(g,f)^*\)
The scalar products Eq. \eqref{eq:vecscalar} and \eqref{eq:funcscalar} both fulfill these properties.
Note: The scalar product between two functions can be defined more generally with a weight function \(w(x)\), \begin {equation} (f,g) = \int _a^b \dif x\, f^*(x) g(x) w(x). \end {equation} The question of orthogonality between functions can thus only be answered with respect to a certain definition of the scalar product. For example, Chebyshev polynomials, see e.g. Boyd (2000), are othogonal with respect to a scalar product with weight function \(w(x)=(1-x^2)^{-1/2}\).
Other notations for the scalar product that are often found in the literature are \(\langle f,g \rangle \) or \(\langle f|g \rangle \). The latter is particularly common in the physics literature, in particular in quantum mechanics.
6.3 Basis functions
Let us now return to the series expansion, \begin {equation} f_N(x) = \sum _{n=1}^N a_n \varphi _n(x). \label {eq:series2} \end {equation} The functions \(\varphi _n(x)\) are called basis functions. A necessary property of the basis functions is their linear independence. The functions are linearly independent if none of the basis functions themselves can be written as a linear combination of the others, i.e. in the form of the series expansion Eq. \eqref{eq:series2}, of the other basis functions. A consequence is that \begin {equation} \sum _{n=1}^{N}a_n \varphi _n(x) = 0 \end {equation} if and only if all \(a_n=0\).
Linearly independent elements form a basis. This basis is called complete if all relevant functions (= elements of the underlying vector space) can be represented by the series expansion \eqref{eq:series2}. (Proofs of the completeness of basis functions are complex and outside the scope of these notes.) The coefficients \(a_n\) are called coordinates or coefficients. The number of basis functions or coordinates \(N\) is called the dimension of the vector space.
Note: A vector space is a set on which the operations of addition and scalar multiplication are defined with the usual properties, such as the existence of neutral and inverse elements and associative, commutative and distributive laws. If this space is defined on functions, it is also referred to as a function space. If there is also an inner product such as Eq. \eqref{eq:funcscalar}, then we speak of a Hilbert space.
6.3.1 Orthogonality
Particularly useful basis functions are orthogonal. Using the scalar product, we can now define orthogonality for these functions. Two functions \(f\) and \(g\) are orthogonal if the scalar product vanishes, \((f, g) = 0\). A set of mutually orthogonal basis functions satisfies \begin {equation} (\varphi _n, \varphi _m) = \nu _{n} \delta _{nm}, \end {equation} where \(\delta _{nm}\) is the Kronecker-\(\delta \). For \(\nu _n\equiv (\varphi _n, \varphi _n)=1\) the basis is called orthonormal.
Orthogonality is useful because it directly allows us to obtain the coefficients of the series expansion \eqref{eq:series2}: \begin {equation} (\varphi _n, f_N) = \sum _{i=1}^N a_i (\varphi _n, \varphi _i) = \sum _{i=1}^N a_i \nu _i \delta _{ni} = a_n \nu _n \end {equation} which yields the coefficients as \begin {equation} a_n = \frac {(\varphi _n, f_N)}{(\varphi _n, \varphi _n)}. \label {eq:coordinates} \end {equation} The coefficients are given by the projection (the scalar product) of the function onto the basis vectors. Remember that the following also applies to Cartesian vectors: \(a_n = \v {a}\cdot \hat {e}_n\). (The normalization factor can be omitted here because \(\hat {e}_n\cdot \hat {e}_n=1\), i.e. the Cartesian basis vectors are orthonormal!) The coefficient given by Eq. \eqref{eq:coordinates} can be thought of as coordinates of the function, similar to the coordinates in Cartesian space.
Note: A useful identity for an expansion into orthogonal bases is Parseval’s theorem. Because scalar products between different basis functions vanish, the square (or power) of a series expansion is given by \begin {equation} (f_N, f_N) = \sum _{n=1}^N |a_n|^2 \nu _n, \end {equation} or for orthonormal basis functions \begin {equation} (f_N, f_N) = \sum _{n=1}^N |a_n|^2. \end {equation}
6.3.2 Fourier basis
An important set of basis functions is the Fourier basis, \begin {equation} \varphi _n(x) = \exp \left ( i q_n x \right ), \label {eq:fourier-basis} \end {equation} on the interval \(x\in [0,L]\) with \(q_n = 2\pi n/L\) and \(n\in \mathbb {Z}\). The Fourier basis is periodic on this interval and is shown in Fig. 6.1. It can easily be shown that \begin {equation} (\varphi _n, \varphi _m) = L \delta _{nm}, \end {equation} so that the Fourier basis is orthogonal. The coefficients \(a_n\) of the Fourier series, \begin {equation} f_\infty (x) = \sum _{n=-\infty }^\infty a_n \varphi _n(x), \label {eq:fourierseries} \end {equation} can thus be obtained as \begin {equation} a_n=\frac {1}{L}(\varphi _n, f_\infty )=\frac {1}{L}\int _0^L \dif x\, f_\infty (x) \exp \left (-i q_n x\right ). \end {equation} This is the well-known formula for the coefficients of the Fourier series.
Note: Conceptually, the Fourier basis describes different frequency components, while the basis of the finite elements described in the next section describes spatial localization.
For a real-valued function with \(f_\infty (x)\equiv f_\infty ^*(x)\), we get \begin {equation} \sum _{n=-\infty }^\infty a_n \varphi _n(x) \equiv \sum _{n=-\infty }^\infty a_n^* \varphi _{-n}(x) \end {equation} because \(\varphi ^*_n(x)=\varphi _{-n}(x)\). This means \(a_n=a^*_{-n}\) is a necessary condition to obtain a real-valued \(f(x)\). This has implications for truncating the Fourier series to a finite number of terms \(N\). In particular, we need to truncate symmetrically, i.e. \begin {equation} f_N(x) = \sum _{n=-(N-1)/2}^{(N-1)/2} a_n \exp \left (iq_n x\right ) \end {equation} with odd \(N\).
6.3.3 Finite elements
Another basis set that is important for numerical analysis is the finite-element basis. In contrast to the Fourier basis, which only becomes zero at isolated points in the entire domain, the finite element basis is localized in space and is zero for large areas of the domain. It thus divides the domain into spatial sections.
In its simplest form, the basis consists of localized piece-wise linear functions, the “tent” functions, \begin {equation} \varphi _n(x) = \left \{ \begin {array}{ll} \frac {x-x_{n-1}}{x_n - x_{n-1}} & \text {for}\; x\in [x_{n-1},x_n]\\ \frac {x_{n+1-x}}{x_{n+1} - x_n} & \text {for}\; x\in [x_n,x_{n+1}] \\ 0 & \text {else} \end {array} \right . \label {eq:finite-element-basis} \end {equation} Here, the \(x_n\) are the nodes (also known as grid points) between which the tents are spanned. The functions are constructed in such a way that the maximum value is \(1\) and \(\int _0^L \dif x\,\varphi _n(x)=(x_{n+1}-x_{n-1})/2\). This basis is the simplest form of the finite element basis and is shown in Fig. 6.2. Higher order polynomials can be used for greater accuracy.
An important remark at this point is that the basis of the finite elements not is orthogonal. The scalar product does not vanish for the nearest neighbors. This is the case because two neighbors have an overlapping rising and falling edge. One obtains \begin {align} M_{nn} \equiv (\varphi _n, \varphi _n) &= \frac {1}{3}(x_{n+1}-x_{n-1}) \\ M_{n,n+1} \equiv (\varphi _n, \varphi _{n+1}) &= \frac {1}{6}(x_{n+1}-x_n) \\ M_{nm} \equiv (\varphi _n, \varphi _m) &= 0 \quad \text {for}\;|n-m|>1 \end {align}
for the scalar products.
Nevertheless, we can use these relations to determine the coefficients of a series expansion, \begin {equation} f_N(x) = \sum _{n=0}^{N-1} a_n \varphi _n(x), \end {equation} which yields \begin {equation} \begin {split} (\varphi _n, f_N(x)) &= a_{n-1} (\varphi _n, \varphi _{n-1}) + a_n (\varphi _n, \varphi _n) + a_{n+1} (\varphi _n, \varphi _{n+1}) \\ &= M_{n,n-1} a_{n-1} + M_{nn} a_n + M_{n,n+1} a_{n+1}. \end {split} \end {equation} We can express this as \begin {equation} (\varphi _n, f_N(x)) = \left [ \t {M}\cdot \v {a} \right ]_n \end {equation} where \([\v {v}]_n=v_n\) denotes the \(n\)th component of the vector enclosed by the two square brackets \([\cdot ]_n\). The matrix \(\t {M}\) is sparse. For an orthogonal basis, such as the Fourier basis of section 6.3.2, this matrix is diagonal. For a basis with identical distances \(x_{n+1}-x_n=1\) of the grid points \(x_n\), the matrix has the following form \begin {equation} \t {M} = \begin {pmatrix} 2/3 & 1/6 & 0 & 0 & 0 & 0 & 0 & \cdots \\ 1/6 & 2/3 & 1/6 & 0 & 0 & 0 & 0 & \cdots \\ 0 & 1/6 & 2/3 & 1/6 & 0 & 0 & 0 & \cdots \\ 0 & 0 & 1/6 & 2/3 & 1/6 & 0 & \cdots \\ 0 & 0 & 0 & 1/6 & 2/3 & 1/6 & \cdots \\ 0 & 0 & 0 & 0 & 1/6 & 2/3 & \cdots \\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \ddots \end {pmatrix}. \end {equation} To find the coefficients \(a_n\), we must solve a (sparse) linear system of equations. The matrix \(\t {M}\) is also called the mass matrix.
Note: Basis sets that are different from zero only at individual points are called spectral basis sets. In particular, the Fourier basis is a spectral basis set for periodic functions. The orthogonal polynomials are important spectral basis sets that are also used in numerical analysis. For example, Chebyshev polynomials are good basis sets for non-periodic functions defined on closed intervals. The finite-element basis is not a spectral basis.