Skip to content

Latest commit

 

History

History
101 lines (75 loc) · 4.22 KB

index.md

File metadata and controls

101 lines (75 loc) · 4.22 KB

Notation

🏷️chap_notation

Throughout this book, we adhere to the following notational conventions. Note that some of these symbols are placeholders, while others refer to specific objects. As a general rule of thumb, the indefinite article "a" often indicates that the symbol is a placeholder and that similarly formatted symbols can denote other objects of the same type. For example, "$x$: a scalar" means that lowercased letters generally represent scalar values, but "$\mathbb{Z}$: the set of integers" refers specifically to the symbol $\mathbb{Z}$.

Numerical Objects

  • $x$: a scalar
  • $\mathbf{x}$: a vector
  • $\mathbf{X}$: a matrix
  • $\mathsf{X}$: a general tensor
  • $\mathbf{I}$: the identity matrix (of some given dimension), i.e., a square matrix with $1$ on all diagonal entries and $0$ on all off-diagonals
  • $x_i$, $[\mathbf{x}]_i$: the $i^\textrm{th}$ element of vector $\mathbf{x}$
  • $x_{ij}$, $x_{i,j}$,$[\mathbf{X}]{ij}$, $[\mathbf{X}]{i,j}$: the element of matrix $\mathbf{X}$ at row $i$ and column $j$.

Set Theory

  • $\mathcal{X}$: a set
  • $\mathbb{Z}$: the set of integers
  • $\mathbb{Z}^+$: the set of positive integers
  • $\mathbb{R}$: the set of real numbers
  • $\mathbb{R}^n$: the set of $n$-dimensional vectors of real numbers
  • $\mathbb{R}^{a\times b}$: The set of matrices of real numbers with $a$ rows and $b$ columns
  • $|\mathcal{X}|$: cardinality (number of elements) of set $\mathcal{X}$
  • $\mathcal{A}\cup\mathcal{B}$: union of sets $\mathcal{A}$ and $\mathcal{B}$
  • $\mathcal{A}\cap\mathcal{B}$: intersection of sets $\mathcal{A}$ and $\mathcal{B}$
  • $\mathcal{A}\setminus\mathcal{B}$: set subtraction of $\mathcal{B}$ from $\mathcal{A}$ (contains only those elements of $\mathcal{A}$ that do not belong to $\mathcal{B}$)

Functions and Operators

  • $f(\cdot)$: a function
  • $\log(\cdot)$: the natural logarithm (base $e$)
  • $\log_2(\cdot)$: logarithm to base $2$
  • $\exp(\cdot)$: the exponential function
  • $\mathbf{1}(\cdot)$: the indicator function; evaluates to $1$ if the boolean argument is true, and $0$ otherwise
  • $\mathbf{1}_{\mathcal{X}}(z)$: the set-membership indicator function; evaluates to $1$ if the element $z$ belongs to the set $\mathcal{X}$ and $0$ otherwise
  • $\mathbf{(\cdot)}^\top$: transpose of a vector or a matrix
  • $\mathbf{X}^{-1}$: inverse of matrix $\mathbf{X}$
  • $\odot$: Hadamard (elementwise) product
  • $[\cdot, \cdot]$: concatenation
  • $|\cdot|_p$: $\ell_p$ norm
  • $|\cdot|$: $\ell_2$ norm
  • $\langle \mathbf{x}, \mathbf{y} \rangle$: inner (dot) product of vectors $\mathbf{x}$ and $\mathbf{y}$
  • $\sum$: summation over a collection of elements
  • $\prod$: product over a collection of elements
  • $\stackrel{\textrm{def}}{=}$: an equality asserted as a definition of the symbol on the left-hand side

Calculus

  • $\frac{dy}{dx}$: derivative of $y$ with respect to $x$
  • $\frac{\partial y}{\partial x}$: partial derivative of $y$ with respect to $x$
  • $\nabla_{\mathbf{x}} y$: gradient of $y$ with respect to $\mathbf{x}$
  • $\int_a^b f(x) ;dx$: definite integral of $f$ from $a$ to $b$ with respect to $x$
  • $\int f(x) ;dx$: indefinite integral of $f$ with respect to $x$

Probability and Information Theory

  • $X$: a random variable
  • $P$: a probability distribution
  • $X \sim P$: the random variable $X$ follows distribution $P$
  • $P(X=x)$: the probability assigned to the event where random variable $X$ takes value $x$
  • $P(X \mid Y)$: the conditional probability distribution of $X$ given $Y$
  • $p(\cdot)$: a probability density function (PDF) associated with distribution $P$
  • ${E}[X]$: expectation of a random variable $X$
  • $X \perp Y$: random variables $X$ and $Y$ are independent
  • $X \perp Y \mid Z$: random variables $X$ and $Y$ are conditionally independent given $Z$
  • $\sigma_X$: standard deviation of random variable $X$
  • $\textrm{Var}(X)$: variance of random variable $X$, equal to $\sigma^2_X$
  • $\textrm{Cov}(X, Y)$: covariance of random variables $X$ and $Y$
  • $\rho(X, Y)$: the Pearson correlation coefficient between $X$ and $Y$, equals $\frac{\textrm{Cov}(X, Y)}{\sigma_X \sigma_Y}$
  • $H(X)$: entropy of random variable $X$
  • $D_{\textrm{KL}}(P|Q)$: the KL-divergence (or relative entropy) from distribution $Q$ to distribution $P$

Discussions