Intro
흔히 Norm은 벡터의 크기를 구할 때 사용했을 것이다. 그러나 이는 Norm의 일부에 불과하다.
앞으로 배워나갈 Norm은 벡터, 행렬, 함수 등에 대해 적용할 수 있고 각각 그 의미를 갖는다. 최종적으로는 이 의미 속에서 선형 시스템의 안정성과 최적 제어 이론을 학습할 것이다.
Vector Norm
- vector norm (2-norm)
: The norm for a vector is a measure to define a size on a length of the vector
A real-valued function $||\cdot||$ defined on X is called a norm on X if it satisfy the following properties.
$$\begin{aligned}
& ||\vec{x}|| \geq 0 \\
& ||\vec{x}||=0 \textit{ iff } x=0 \\
& ||a\vec{x}||=|a|||\vec{x}|| \\
& ||\vec{x}+\vec{y}|| \leq ||\vec{x}|| + ||\vec{y}|| \end{aligned}$$
- vector p-norm
$$\vec{x} \in \mathbb{C}^{n\times 1}$$
$$||\vec{x}||_p = (\sum_{i=1}^n |x_i|^p)^{\frac{1}{p}}$$
vector 1-norm : $||\vec{x}||_1=\sum_{i=1}^n|x_i|$
vector 2-norm : $||\vec{x}||_2=(\sum_{i=1}^n|x_i|^2)^{\frac{1}{2}}$
vector $\infty$-norm : $||\vec{x}||_\infty=\underset{1\le i\le n}{max}|x_i|$
* $L_p$ norm : p-norm of a vector
- 2-norm for $\vec{x}\in\mathbb{C},\vec{y}\in\mathbb{C}$
$$(i) \quad ||\vec{x}||_2=\sqrt{\vec{x}^*\vec{x}}$$
$$(ii) \quad \; if \; n\ge m, ||\vec{x}||_2=||\vec{y}||_2 \; iff \; \exists U \in \mathbb{C}^{n\times m} \\
such \; that \; \vec{x}=U\vec{y} \; and \; U^*U=I$$
$$(iii) \quad n=m \; then \; |\vec{x}^*\vec{y}|\leq||\vec{x}||||\vec{y}||. \\
\textit{The equality hold iff}\; \vec{x}=\alpha\vec{y}$$
- Normed vector space
Consider $\vec{x}=\begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$. And $||x||_p=1$.
$$||x||_1=|x_1|+|x_2|=1$$
$$||x||_2=\sqrt{|x_1|^2+|x_2|^2}=1$$
$$||x||_{\infty}= max|x_i|$$
$$\downarrow$$
$$\text{There is a bounded space of }\; ||\vec{x}||_p$$
$$l_p=\{\vec{x} | \vec{x}\in\mathbb{R}^{\infty} \text{ with bounded} ||\vec{x}||_p\}$$
Ex.
$x=[1,1]^T \in l_\infty$
$x=[0.5, 0.,5]^T \in l_1$
Function norm
- p-norm of a function $f(t)$
$$||f(t)||_p=(\int_{\infty}^{\infty}|f(t)|^pdt)^\frac{1}{p}$$
function 1-norm : $ \int_{\infty}^{\infty}|f(t)|dt $
function 2-norm : $ (\int_{\infty}^{\infty}|f(t)|^2dt)^\frac{1}{2} $
function $\infty$-norm : $\underset{t}{max}|f(t)|$
- Holder's inequality
$$\int_{\infty}^{\infty}|u(t)y(t)|dt \leq (\int_{\infty}^{\infty}|u(t)|^pdt)^\frac{1}{p} (\int_{\infty}^{\infty}|y(t)|^qdt)^\frac{1}{q}$$
$$||u(t)y(t)||_1 \leq ||u(t)||_p||y(t)||_q$$
$$for \; \frac{1}{p}+\frac{1}{p}=1$$
- Minkowski's inequality
$$||u(t)+y(t)||_p \leq ||u||_p + ||y||_p$$
Matrix norm
- Induced Matrix norm
$$\vec{y}=A\vec{x} \\ A\in\mathbb{C}^{m\times n}, x\in\mathbb{C}^{n\times 1}, y\in\mathbb{C}^{m\times 1}$$
$$||A||_p=\underset{||\vec{x}||_p\neq0}{max}\frac{||\vec{y}||_p}{||\vec{x}||_p}=\underset{||\vec{x}||_p=1}{max}||\vec{y}||_p=\underset{||\vec{x}||_p=1}{max}||A\vec{x}||_p$$
- Induced Matrix 1-norm
$$||A||_1=\underset{||x||_1=1}{max}||A\vec{x}||_1=\underset{j=1,2,,,m}{max}(\sum_{i=1}^m|a_{ij}|)$$
$$\text{: maximun sum of column}$$
- Induced Matrix 2-norm
$$||A||_2=\underset{||x||_2=1}{max}||A\vec{x}||_2=\bar{\sigma}(A)$$
$$\text{: biggest singular value of A}$$
- Frobenius Matrix norm
$$||A||_F=\sqrt{tr(A^*A)}=\sqrt{\sum_{i=1}^n\sum_{j=1}^m|a_{ij}|^2}=\sqrt{\sum_i\sigma_i^2(A)}$$
- Norm properties
* Unitrary matrix $U*U=I$
$$||A||_2=||UA||_2=||AU||_2$$
$$||A||_F=||UA||_F=||AU||_F$$
$$||AB||_p \leq ||A||_p||B||_p$$
$$||A^{-1}||_p \geq ||A||_p^{-1}$$
- Parseval's theorem
The 2-norm of the $f(t)$ is same as $F(w)$ which is fourier transform of $f(t)$
$$||F(w)|| \triangleq \sqrt{\frac{1}{2\pi}\int_{\infty}^{\infty}|F(w)|^2dw}$$
Then,
$$\sqrt{\int_{\infty}^{\infty}|f(t)|^2dt}= \sqrt{\frac{1}{2\pi}\int_{\infty}^{\infty}|F(w)|^2dw} $$
$$||f(t)||_2=||F(w)||_2$$
- Normed function space
$$L_p=\{f(t) | \text{with bounded} ||f(t)||_p\}$$
$$L_2 \text{ space : Hilbert space}$$
$$L_p \text{ space : Banach space}$$
Ex.
$sin(t) \notin L_1$
$sin(t) \notin L_2$
$sin(t) \in L_\infty$
