For easy reference, I’ve decided to put some of the more “building block” definitions in a separate post. Most of the definitions are from Wikipedia and Axler. In addition, most concepts are introduced in the context of metric spaces with further discussion in Euclidean space.
Sets
Types
We’ll begin with the field, which is just a set with some binary operations. Binary operations are basically just functions taking in two elements from a set, $S$, and mapping them to some (unique) element also in $S$ (i.e. $f: S \times S \rightarrow S$).
Definition (Field). A field is a set, $F$, and the two binary operations of addition (which we denote with $+$) and multiplication (which we denote with $\cdot$) that satisfy the following axioms. For any $a, b, c$ in $F$:
Associativity: $(a + (b + c)) = (a + b) + c$ and $a \cdot (b \cdot c) = (a \cdot b) \cdot c$
Commutativity: $a+b = b+a$ and $a \cdot b = b \cdot a$
Identity: $0$ and $1$ are elements in $F$ such that $a + 0 = a$ and $a \cdot 1 = a$
Distributivity: $a \cdot (b + c) = (a \cdot b) + (a \cdot c)$
Additive Inverse: For any $a$ in $F$, there exists its additive inverse in $F$, denoted by $-a$, which satisfies $a + (-a) = 0$
Multiplicative Inverse: For any $a \neq 0$ in $F$, there exists its multiplicative inverse in $F$, denote by $a^{-1}$, which satisfies $a \cdot a^{-1} = 1$
A set, $M$, with a metric, $d$, together as an ordered pair, $(M, d)$, forms a metric space.
Characteristics
Sets (i.e. spaces for our purposes) can be described in a lot of different ways. First, let’s look at what it means to be complete.
Definition (Complete). A metric space, $(M, d)$, is called complete if every Cauchy sequence of points in $M$ has its limit in $M$. The metric, $d$, is also called complete.
This implies a complete field is a field equipped with complete metric.
We can also describe a set as open or closed.
Definition (Open/Closed). Let $U$ be a subset of metric space $(M, d)$. We call $U$ open if, for any $x \in U$, there exists a real number $\epsilon > 0$ such that for any $y \in M$ where $d(x, y) < \epsilon$, we have that $y \in M$. The complement of an open set is called closed. Equivalently, $U$ is called open if for every $u \in U$, there exists some $\delta > 0$ such that the open ball centered at $u$ with radius $\delta$ is contained in $U$. That is: $$ B(x, \delta) = \{ y \in M: d(y - x) < \delta \} \subseteq U $$
Though a bit confusing, sets can also be both closed and open (also called clopen). The empty set is an example.
Open sets have some nice properties including the fact that the (finite or infinite) union of open sets is open, and the finite intersection of open sets is open. In contrast, the (finite or infinite) intersection of closed sets is closed, and the finite union of closed sets is closed.
Ordering
Elements can be compared using orders, which are binary relations denoted by $\leq$.
Definition (Partial Order). A partial order is a binary relation, $\leq$, between a set, $X$, and itself satisfying the following for any $a, b, c \in X$:
Reflexivity: $a \leq a$
Antisymmetry: $a \leq b$ and $b \leq a \implies a = b$
Transitivity: $a \leq b$ and $b \leq c \implies a \leq c$
Partial orders as defined above are sometimes called reflexive, weak, or non-strict. A strict partial order is a binary relation, $<$, between a set, $X$, and itself satisfying the following for all $a, b, c \in X$:
Irreflexivity: $\neg(a < a)$
Asymmetry: $a < b \implies \neg (b < a)$
Transitivity: $a < b$ and $b < c \implies a < c$
A total order is a partial order satisfying one additional property.
Definition (Total Order). A total order, also called a linear order, is a partial order satisfying the additional property for all $a, b, c \in X$:
Totality: $a \leq b$ or $b \leq a$
Total orders as defined above are sometimes called non-strict. A strict total order is a strict partial order that satisfies the following additional proerpty for all $a, b \in X$:
Connectivity: $a \neq b \implies a < b$ or $b < a$
Joining a field, $F$, with a total order, $\leq$, defines an ordered field if the following are satisfied for all $a,b,c \in F$:
If $a \leq b$, then $a + c \leq b + c$
If $0 \leq a$ and $0 \leq b$ then $0 \leq a \cdot b$
Sequences
Sequences will also come up a lot, and it’s important to have a good handle on how these can be characterized.
Definition (Cauchy). Let $x_1, x_2, x_3, \dots$ denote a sequence defined on some metric space, $(X, d)$. The sequence is called Cauchy if, for any positive $r \in \mathbb{R}$, there exists a positive $N \in \mathbb{N}$ such that for all $m, n > N$: $$ d(x_m, x_n) < r $$
Some sequences converge, which means they have a limit.
Definition (Limit). Let $\{ x_n \}_{n \geq 0}$ be a sequence of elements in a metric space $(M, d)$, and let $L \in M$. If, for any $\epsilon > 0$, there exists $N$ such that for all $n > N$: $$ d(L, x_n) < \epsilon $$ then we call $L$ the limit of the sequence $\{ x_n \}_{n \geq 0}$ and denote it by: $$ \underset{n \rightarrow \infty}{\lim} x_n = L $$
The above definition is for general metric spaces. Often we are dealing with $\mathbb{R}^n$, in which case we will choose either the $\ell^2$ norm or the infinity norm. Limits can also be defined coordinate-wise for vector-valued elements.
Norms
First, the $\ell^2$ norm, which is one of the most commonly encountered norms (at least for me). It is also called the Euclidean norm.
Definition ($\rvert \rvert \cdot \rvert \rvert_2$). Let $x = (x_1, \dots, x_n) \in \mathbb{R}^n$. The $\ell^2$ norm is defined as: $$ \rvert \rvert x \rvert \rvert_2 = \sqrt{ \rvert x_1 \rvert ^2 + \dots + \rvert x_n \rvert^2} $$
Now, the infinity norm.
Definition ($\rvert \rvert \cdot \rvert \rvert_{\infty}$). Let $x = (x_1, \dots, x_n) \in \mathbb{R}^n$. The infinity norm is defined as: $$ \rvert \rvert x \rvert \rvert_{\infty} = \max \left\{ \rvert x_1 \rvert, \dots, \rvert x_n \rvert \right\} $$
Functions
First thing’s first: what’s a function?
Definition (Function). For sets $X$ and $Y$, a function, $f: X \rightarrow Y$, assigns a value in $Y$ to each value in $X$.
The set $X$ is called its domain, and the set $Y$ is called its codomain. The image of a function (also called its range) is the subset of elements in $Y$ that are mapped to by elements in $x$. In notation:
\[\text{Im}(f) = \{ f(x): x \in X \}\]
Types
A distance metric or function (or just metric for short) assigns a value to represent how “far apart” two elements are in a set.
Definition (Distance Metric). Let $X$ be a set. A distance metric is any function $d: X \times X \rightarrow \mathbb{R}$ satisfying the following for all $x, y, z, \in X$:
One of the most important characteristics of functions is continuity.
Definition (Continuity). Let $f: X \rightarrow Y$ be a function between metric spaces $(X, d_X)$ and $(Y, d_Y)$. The function $f$ is called continuous at a point $c \in X$ if, for any positive real $\epsilon > 0$, there exists a positive real $\delta > 0$ such that $d_Y(f(x), f(c)) < \epsilon$ for all $x \in X$ such that $d_X(x, c) < \delta$.
If $f$ is a real-valued function, then the definition is equivalent to saying that $\underset{n \rightarrow \infty}{\lim} f(x_n) = f(c)$ for every ${ x_n }_{n \geq 0}$ in $X$ with $\underset{n \rightarrow \infty}{\lim} x_n = c$.
A stronger kind of continuity is uniform continuity.
Definition (Uniform Continuity). Let $f: X \rightarrow Y$ be a function between metric spaces $(X, d_X)$ and $(Y, d_Y)$. The function $f$ is called uniformly continuous if, for every real $\epsilon > 0$, there exists a real $\delta > 0$ such that $d_x(f(x), f(y)) < \epsilon$ for all $x, y \in X$ such that $d_1(x, y) < \delta$.
Continuous, real-valued functions defined on a closed, bounded subset of the reals will be uniformly continuous. In addition, they will achieve their maxima and minima.