Vector Norm on $ℝ^{n}$

Yun-Hao Lee, Cheng-Han Huang

Credit: David Jakab from Pexels

1. Vector Norm on $ℝ^{n}$

Definition 1.1. A norm $∥ \cdot ∥$ on $ℝ^{n}$ is a function $∥ \cdot ∥ : ℝ^{n} \to ℝ$ satisfying the following

(nonnegativity) $| | x | | \geq 0$ for any $x \in ℝ^{n}$ and $| | x | | = 0$ if and only if $x = 0$ .
(positivity homogeneity) $| | λx | | = | λ | | | x | |$ for any $x \in ℝ^{n}$ and $λ \in ℝ$ .
(triangle inequality) $| | x + y | | \leq | | x | | + | | y | |$ for any $x, y \in ℝ^{n}$ .

Definition 1.2 ( $l_{p}$ norm). The $l_{p}$ norm ( $p \geq 1$ ) is defined by

| | x | |_{p} = \sqrt[p]{\sum_{i = 1}^{n} | x_{i} |^{p}}

Example 1.3. The $l_{1}$ norm for a vector $x \in ℝ^{n}$ is computed by

| | x | |_{1} = \sum_{i = 1}^{n} | x_{i} | = | x_{1} | + | x_{2} | + . . . + | x_{n} |

and is usually called Manhattan distance.

Example 1.4. The Euclidean norm on $ℝ^{n}$ , defined by

| | x | | = \sqrt{\sum_{i = 1}^{n} | x_{i} |^{2}}

is $l_{2}$ norm.

Example 1.5. We can compute $l_{\infty}$ norm by $| | x | |_{\infty} = {max}_{1 \leq i \leq n} | x_{i} |$ .

Proposition 1.6. If $0 < p < 1$ and $2 \leq n$ , $\sqrt[p]{\sum_{i = 1}^{n} | x_{i} |^{p}}$ is not a norm. One may check the property by giving a counterexample of two elements that does not satisfy the triangle inequality.

Proposition 1.7. $\lim_{p \to \infty} | | x | |_{p} = | | x | |_{\infty}$

Theorem 1.8. (Cauchy-Schwarz inequality) For any $x, y \in ℝ^{n}$ , we have
$| x^{T} y | \leq | | x | |_{2} \times | | y | |_{2}$ . Equality holds if and only if $y = 0$ or $x = ry$ for some $r \in ℝ$ .

Proof. If $y = 0$ , clearly equality hols. Suppose $y \neq 0$ . Then

\begin{array}{l} 0 \leq ⟨ x - \frac{⟨ x, y ⟩}{| | y | |_{2}^{2}} y, x - \frac{⟨ x, y ⟩}{| | y | |_{2}^{2}} y ⟩ = ⟨ x, x ⟩ - 2 (\frac{⟨ x, y ⟩}{| | y | |_{2}^{2}} ⟨ x, y ⟩) + \frac{{⟨ x, y ⟩}^{2}}{| | y | |_{2}^{4}} ⟨ y, y ⟩ = ⟨ x, x ⟩ - \frac{{⟨ x, y ⟩}^{2}}{| | y | |_{2}^{2}} \\ \Rightarrow & \frac{{⟨ x, y ⟩}^{2}}{| | y | |_{2}^{2}} \leq ⟨ x, x ⟩ \Rightarrow {⟨ x, y ⟩}^{2} \leq | | x | |_{2}^{2} | | y | |_{2}^{2} \Rightarrow | ⟨ x, y ⟩ | \leq | | x | | \times | | y | | \\ \Rightarrow & | x^{T} y | \leq | | x | | \times | | y | | \end{array}

Check that if $x = ry$ , then equality holds.
RHS of the above inequality is $| x^{T} y | = | r | | | y | |_{2}^{2}$ ; LHS of the above inequality is $| | x | |_{2} \times | | y | |_{2} = | | ry | |_{2} \times | | y | |_{2} = | r | | | y | |_{2}^{2}$ . Therefore equality holds.
Conversely, condition of equality can be obtained by straightforward computation from $0 = ⟨ x - \frac{⟨ x, y ⟩}{| | y | |_{2}^{2}} y, x - \frac{⟨ x, y ⟩}{| | y | |_{2}^{2}} y ⟩$ .

Reference:

Beck, A. (2014). Introduction to nonlinear optimization: Theory, algorithms, and applications with MATLAB. Society for Industrial and Applied Mathematics.

Vector Norm on ℝn

1. Vector Norm on ℝn

Reference:

Vector Norm on $ℝ^{n}$

1. Vector Norm on $ℝ^{n}$