Optimization#
Mathematical Optimization#
An optimization problem consists of a set \(D\), called the domain, and a real-valued function \(f:D\to R\), called the objective fucntion.
minimization problem: find an \(x\in D\) such that \(f(x)\le f(y)\) for all \(y\in D\).
maximization problem: find an \(x\in D\) such that \(f(x)\ge f(y)\) for all \(y\in D\).
a mathematical program consist of:
an objective function \(f_0: R^n \to R\)
a set of \(m\) constriant functions \(f_i: \mathbb{R}^n \to \mathbb{R}, i=1,\dots, m\).
constant variables \(x\)
standard form
\(x\in R^n\) is the optimization variable
\(f_0:R^n\to R\) is the objective or cost function
\(f_i:R^n\to R, i=1,\dots, m\) is the inequality constraint functions
\(h_i:R^n\to R\) are the equality constraint functions
optimal values
\(p^*\)
\(x\) is feasible if \(x\in dom (f_0)\) and it satisfies the constraints
optimal point: a feasible \(x\) is optimal if \(f_0(x) = p^*\)
\(x\) is locally optimal if there is an \(R>0\) such that \(x\) is optimal for …
Example 1
\(f_0(x) = 1/x\), \(dom f_0 = R_{++}\)
\(f_0(x) = -\log x\), \(dom f_0 = R_{++}\)
\(f_0(x) = x\log x\), \(dom f_0 = R_{++}\)
feasibility is a special case i.e., \(f_0(x)=0\)
Definition 2 (Optimization problem)
A optimization problem, has the form
A vector \(x^*\) is called optimal, or solution to Definition 2,
if it has the smallest objective value among all vectors that satisfy the constraints.