Orthonormal Bases of Vectors

Basic understanding of vector operations is provided by Vector Operations.

Linear Combination of Vectors

$u\_1,u\_2,...,u\_n$ are combined with coefficients $a\_k$ and give a linear combination which is a vector: $a\_1u\_1+a\_2u\_2+...+a\_nu\_n$

Up to Three Dimensions

Orthonormal Bases

Let $r,s,t$ be orthonormal vectors in three-dimensional space. Then every vector $v=v\_1,v\_2,v\_3$ can be written as

$$v=(v\cdot r)r+(v\cdot s)s+(v\cdot t)t$$

This arises from solving $v=c\_1r+c\_2s+c\_3t$ for the constants. As every vector in a space can be expressed in terms of this linear combination of $r,s,t$, this set of vectors is called a basis. If they are orthonormal then it is an orthonormal basis.

Linear Combination Approximation

if $r,s,t$ is an orthonormal basis for three-dimensional space and v is a vector, then the linear combination of $r,s$ that is closest to $v$ is.

$$p(r,s)=(v\cdot r)r+(v\cdot s)s$$

Equivalent of dropping the vector $v$ perpendicularly onto the plane spanned by $r,s$. The image demonstrates the idea.

Higher Dimensions

In a n-dimensional space with $u\_1,u\_2,...,u\_n$ basis vectors, any n-dimensional vector can be written as

$$v=\sum \_{k=1}^n(v\cdot u\_k)u\_k$$

The best approximation to this vector is the same formula, but with $1\le d<n$, a lower-dimensional vector, that tries to approximate our n-dimensional $v$. The best approximation of $v$ in $d$ dimensions is the vector $p$, the “orthogonal projection of $v$ onto the span of $u\_1,u\_2,...,u\_c$“.

$$p=\sum \_{k=1}^d(v\cdot u\_k)u\_k$$