algorithm_9linear_programming.pdf
0 likes49 views
The document outlines the components and concepts of linear programming, including decision variables, objective functions, and constraints. It discusses an example of profit maximization for a chocolatier and introduces the simplex method for finding optimal solutions. Additionally, it explains the concept of duality in linear programming, emphasizing the relationship between primal and dual problems.
algorithm_9linear_programming.pdf
- 1. Iris Hui-Ru Jiang Fall 2014 Linear Programming IRIS H.-R. JIANG Linear Programming ¨ Course contents: ¤ Linear programming ¤ Formulation ¤ Duality ¤ The simplex method ¨ Reading: ¤ Chapter 7 (Dasgupta) ¤ Chapter 29 (Cormen) Linear programming 2 IRIS H.-R. JIANG Linear Programming ¨ Linear programming describes a broad class of optimization tasks in which both the optimization criterion and the constraints are linear functions. ¨ Linear programming consists of three parts: ¤ A set of decision variables ¤ An objective function: n maximize or minimize a given linear objective function ¤ A set of constraints: n satisfy a set of linear inequalities involving these variables Linear programming 3 IRIS H.-R. JIANG Example: Profit Maximization (1/4) ¨ A boutique chocolatier has two products: ¤ A (Pyramide): profit $1 per box ¤ B (Nuit): profit $6 per box ¨ Constraints: ¤ The daily demand for these exclusive chocolates is limited to at most 200 boxes of A and 300 boxes of B ¤ The current workforce can produce a total of at most 400 boxes of chocolate per day ¨ Decision variables: ¤ x1 = Boxes of A ¤ x2 = Boxes of B ¨ Objective Function: ¤ Maximize profit Linear programming 4
- 2. Iris Hui-Ru Jiang Fall 2014 Linear Programming IRIS H.-R. JIANG Linear Programming ¨ Course contents: ¤ Linear programming ¤ Formulation ¤ Duality ¤ The simplex method ¨ Reading: ¤ Chapter 7 (Dasgupta) ¤ Chapter 29 (Cormen) Linear programming 2 IRIS H.-R. JIANG Linear Programming ¨ Linear programming describes a broad class of optimization tasks in which both the optimization criterion and the constraints are linear functions. ¨ Linear programming consists of three parts: ¤ A set of decision variables ¤ An objective function: n maximize or minimize a given linear objective function ¤ A set of constraints: n satisfy a set of linear inequalities involving these variables Linear programming 3 IRIS H.-R. JIANG Example: Profit Maximization (1/4) ¨ A boutique chocolatier has two products: ¤ A (Pyramide): profit $1 per box ¤ B (Nuit): profit $6 per box ¨ Constraints: ¤ The daily demand for these exclusive chocolates is limited to at most 200 boxes of A and 300 boxes of B ¤ The current workforce can produce a total of at most 400 boxes of chocolate per day ¨ Decision variables: ¤ x1 = Boxes of A ¤ x2 = Boxes of B ¨ Objective Function: ¤ Maximize profit Linear programming 4
- 3. Iris Hui-Ru Jiang Fall 2014 Linear Programming IRIS H.-R. JIANG Linear Programming ¨ Course contents: ¤ Linear programming ¤ Formulation ¤ Duality ¤ The simplex method ¨ Reading: ¤ Chapter 7 (Dasgupta) ¤ Chapter 29 (Cormen) Linear programming 2 IRIS H.-R. JIANG Linear Programming ¨ Linear programming describes a broad class of optimization tasks in which both the optimization criterion and the constraints are linear functions. ¨ Linear programming consists of three parts: ¤ A set of decision variables ¤ An objective function: n maximize or minimize a given linear objective function ¤ A set of constraints: n satisfy a set of linear inequalities involving these variables Linear programming 3 IRIS H.-R. JIANG Example: Profit Maximization (1/4) ¨ A boutique chocolatier has two products: ¤ A (Pyramide): profit $1 per box ¤ B (Nuit): profit $6 per box ¨ Constraints: ¤ The daily demand for these exclusive chocolates is limited to at most 200 boxes of A and 300 boxes of B ¤ The current workforce can produce a total of at most 400 boxes of chocolate per day ¨ Decision variables: ¤ x1 = Boxes of A ¤ x2 = Boxes of B ¨ Objective Function: ¤ Maximize profit Linear programming 4
- 4. Iris Hui-Ru Jiang Fall 2014 Linear Programming IRIS H.-R. JIANG Linear Programming ¨ Course contents: ¤ Linear programming ¤ Formulation ¤ Duality ¤ The simplex method ¨ Reading: ¤ Chapter 7 (Dasgupta) ¤ Chapter 29 (Cormen) Linear programming 2 IRIS H.-R. JIANG Linear Programming ¨ Linear programming describes a broad class of optimization tasks in which both the optimization criterion and the constraints are linear functions. ¨ Linear programming consists of three parts: ¤ A set of decision variables ¤ An objective function: n maximize or minimize a given linear objective function ¤ A set of constraints: n satisfy a set of linear inequalities involving these variables Linear programming 3 IRIS H.-R. JIANG Example: Profit Maximization (1/4) ¨ A boutique chocolatier has two products: ¤ A (Pyramide): profit $1 per box ¤ B (Nuit): profit $6 per box ¨ Constraints: ¤ The daily demand for these exclusive chocolates is limited to at most 200 boxes of A and 300 boxes of B ¤ The current workforce can produce a total of at most 400 boxes of chocolate per day ¨ Decision variables: ¤ x1 = Boxes of A ¤ x2 = Boxes of B ¨ Objective Function: ¤ Maximize profit Linear programming 4
- 5. IRIS H.-R. JIANG Example: Profit Maximization (2/4) ¨ A linear equation in x1 and x2 defines a line in the 2D plane ¨ A linear inequality designates a half-space ¨ The set of all feasible solutions of this linear program is the intersection of five half-spaces. It is a convex polygon Linear programming 5 IRIS H.-R. JIANG Example: Profit Maximization (3/4) ¨ Search for the optimal solution ¤ It is a general rule of linear programs that the optimum is achieved at a vertex of the feasible region. Linear programming 6 (100, 300) IRIS H.-R. JIANG Example: Profit Maximization (4/4) ¨ The Simplex method: hill climbing ¤ George Dantzig, 1947 ¤ Starts at a vertex, say (0, 0) ¤ Repeatedly looks for an adjacent vertex (connected by an edge of the feasible region) of better objective value ¤ Upon reaching a vertex that has no better neighbor, simplex declares it to be optimal and halts Linear programming 7 (100, 300) IRIS H.-R. JIANG Multipliers? ¨ Optimal: (x1, x2) = (100, 300); objective value = 1900 ¨ Can this answer be checked somehow? ¤ (1) + 6*(2): x1 + 6x2 <= 2000 ¤ 0*(1) + 5*(2) + (3): x1 + 6x2 <= 1900 ¤ The multipliers (0, 5, 1) constitute a certificate of optimality ¤ How would we systematically find the magic multipliers? Linear programming 8 max x1 + 6x2 x1 <= 200 (1) x2 <= 300 (2) x1 + x2 <= 400 (3) x1 , x2 >= 0.
- 6. IRIS H.-R. JIANG Example: Profit Maximization (2/4) ¨ A linear equation in x1 and x2 defines a line in the 2D plane ¨ A linear inequality designates a half-space ¨ The set of all feasible solutions of this linear program is the intersection of five half-spaces. It is a convex polygon Linear programming 5 IRIS H.-R. JIANG Example: Profit Maximization (3/4) ¨ Search for the optimal solution ¤ It is a general rule of linear programs that the optimum is achieved at a vertex of the feasible region. Linear programming 6 (100, 300) IRIS H.-R. JIANG Example: Profit Maximization (4/4) ¨ The Simplex method: hill climbing ¤ George Dantzig, 1947 ¤ Starts at a vertex, say (0, 0) ¤ Repeatedly looks for an adjacent vertex (connected by an edge of the feasible region) of better objective value ¤ Upon reaching a vertex that has no better neighbor, simplex declares it to be optimal and halts Linear programming 7 (100, 300) IRIS H.-R. JIANG Multipliers? ¨ Optimal: (x1, x2) = (100, 300); objective value = 1900 ¨ Can this answer be checked somehow? ¤ (1) + 6*(2): x1 + 6x2 <= 2000 ¤ 0*(1) + 5*(2) + (3): x1 + 6x2 <= 1900 ¤ The multipliers (0, 5, 1) constitute a certificate of optimality ¤ How would we systematically find the magic multipliers? Linear programming 8 max x1 + 6x2 x1 <= 200 (1) x2 <= 300 (2) x1 + x2 <= 400 (3) x1 , x2 >= 0.
- 7. IRIS H.-R. JIANG Example: Profit Maximization (2/4) ¨ A linear equation in x1 and x2 defines a line in the 2D plane ¨ A linear inequality designates a half-space ¨ The set of all feasible solutions of this linear program is the intersection of five half-spaces. It is a convex polygon Linear programming 5 IRIS H.-R. JIANG Example: Profit Maximization (3/4) ¨ Search for the optimal solution ¤ It is a general rule of linear programs that the optimum is achieved at a vertex of the feasible region. Linear programming 6 (100, 300) IRIS H.-R. JIANG Example: Profit Maximization (4/4) ¨ The Simplex method: hill climbing ¤ George Dantzig, 1947 ¤ Starts at a vertex, say (0, 0) ¤ Repeatedly looks for an adjacent vertex (connected by an edge of the feasible region) of better objective value ¤ Upon reaching a vertex that has no better neighbor, simplex declares it to be optimal and halts Linear programming 7 (100, 300) IRIS H.-R. JIANG Multipliers? ¨ Optimal: (x1, x2) = (100, 300); objective value = 1900 ¨ Can this answer be checked somehow? ¤ (1) + 6*(2): x1 + 6x2 <= 2000 ¤ 0*(1) + 5*(2) + (3): x1 + 6x2 <= 1900 ¤ The multipliers (0, 5, 1) constitute a certificate of optimality ¤ How would we systematically find the magic multipliers? Linear programming 8 max x1 + 6x2 x1 <= 200 (1) x2 <= 300 (2) x1 + x2 <= 400 (3) x1 , x2 >= 0.
- 8. IRIS H.-R. JIANG Example: Profit Maximization (2/4) ¨ A linear equation in x1 and x2 defines a line in the 2D plane ¨ A linear inequality designates a half-space ¨ The set of all feasible solutions of this linear program is the intersection of five half-spaces. It is a convex polygon Linear programming 5 IRIS H.-R. JIANG Example: Profit Maximization (3/4) ¨ Search for the optimal solution ¤ It is a general rule of linear programs that the optimum is achieved at a vertex of the feasible region. Linear programming 6 (100, 300) IRIS H.-R. JIANG Example: Profit Maximization (4/4) ¨ The Simplex method: hill climbing ¤ George Dantzig, 1947 ¤ Starts at a vertex, say (0, 0) ¤ Repeatedly looks for an adjacent vertex (connected by an edge of the feasible region) of better objective value ¤ Upon reaching a vertex that has no better neighbor, simplex declares it to be optimal and halts Linear programming 7 (100, 300) IRIS H.-R. JIANG Multipliers? ¨ Optimal: (x1, x2) = (100, 300); objective value = 1900 ¨ Can this answer be checked somehow? ¤ (1) + 6*(2): x1 + 6x2 <= 2000 ¤ 0*(1) + 5*(2) + (3): x1 + 6x2 <= 1900 ¤ The multipliers (0, 5, 1) constitute a certificate of optimality ¤ How would we systematically find the magic multipliers? Linear programming 8 max x1 + 6x2 x1 <= 200 (1) x2 <= 300 (2) x1 + x2 <= 400 (3) x1 , x2 >= 0.
- 9. IRIS H.-R. JIANG Duality (1/3) ¨ Multipliers yi’s must be nonnegative ¨ If the left-hand side to look like our objective function, the right- hand side is an upper bound on the optimum solution ¨ We want a tight bound! Linear programming 9 max x1 + 6x2 x1 <= 200 (1) x2 <= 300 (2) x1 + x2 <= 400 (3) x1 , x2 >= 0. IRIS H.-R. JIANG Duality (2/3) ¨ A new LP: finding multipliers that gives the best upper bound on our original LP ¤ Primal LP ¤ Any feasible value of dual LP is an upper bound on primal LP ¤ If we find a pair of primal and dual feasible values that are equal, they must be both optimal. Linear programming 10 ¤ Dual LP IRIS H.-R. JIANG Duality (3/3) ¨ Generic form: ¨ Dual theorem: If a linear program has a bounded optimum, then so does its dual, and the new optimum values coincide. ¤ Max-flow min-cut Linear programming 11 IRIS H.-R. JIANG The Simplex Algorithm Linear programming 12 Every constraint specifies an n-dimensional half-space Travel along “edges” until no improvement can be made
- 10. IRIS H.-R. JIANG Duality (1/3) ¨ Multipliers yi’s must be nonnegative ¨ If the left-hand side to look like our objective function, the right- hand side is an upper bound on the optimum solution ¨ We want a tight bound! Linear programming 9 max x1 + 6x2 x1 <= 200 (1) x2 <= 300 (2) x1 + x2 <= 400 (3) x1 , x2 >= 0. IRIS H.-R. JIANG Duality (2/3) ¨ A new LP: finding multipliers that gives the best upper bound on our original LP ¤ Primal LP ¤ Any feasible value of dual LP is an upper bound on primal LP ¤ If we find a pair of primal and dual feasible values that are equal, they must be both optimal. Linear programming 10 ¤ Dual LP IRIS H.-R. JIANG Duality (3/3) ¨ Generic form: ¨ Dual theorem: If a linear program has a bounded optimum, then so does its dual, and the new optimum values coincide. ¤ Max-flow min-cut Linear programming 11 IRIS H.-R. JIANG The Simplex Algorithm Linear programming 12 Every constraint specifies an n-dimensional half-space Travel along “edges” until no improvement can be made
- 11. IRIS H.-R. JIANG Duality (1/3) ¨ Multipliers yi’s must be nonnegative ¨ If the left-hand side to look like our objective function, the right- hand side is an upper bound on the optimum solution ¨ We want a tight bound! Linear programming 9 max x1 + 6x2 x1 <= 200 (1) x2 <= 300 (2) x1 + x2 <= 400 (3) x1 , x2 >= 0. IRIS H.-R. JIANG Duality (2/3) ¨ A new LP: finding multipliers that gives the best upper bound on our original LP ¤ Primal LP ¤ Any feasible value of dual LP is an upper bound on primal LP ¤ If we find a pair of primal and dual feasible values that are equal, they must be both optimal. Linear programming 10 ¤ Dual LP IRIS H.-R. JIANG Duality (3/3) ¨ Generic form: ¨ Dual theorem: If a linear program has a bounded optimum, then so does its dual, and the new optimum values coincide. ¤ Max-flow min-cut Linear programming 11 IRIS H.-R. JIANG The Simplex Algorithm Linear programming 12 Every constraint specifies an n-dimensional half-space Travel along “edges” until no improvement can be made
- 12. IRIS H.-R. JIANG Duality (1/3) ¨ Multipliers yi’s must be nonnegative ¨ If the left-hand side to look like our objective function, the right- hand side is an upper bound on the optimum solution ¨ We want a tight bound! Linear programming 9 max x1 + 6x2 x1 <= 200 (1) x2 <= 300 (2) x1 + x2 <= 400 (3) x1 , x2 >= 0. IRIS H.-R. JIANG Duality (2/3) ¨ A new LP: finding multipliers that gives the best upper bound on our original LP ¤ Primal LP ¤ Any feasible value of dual LP is an upper bound on primal LP ¤ If we find a pair of primal and dual feasible values that are equal, they must be both optimal. Linear programming 10 ¤ Dual LP IRIS H.-R. JIANG Duality (3/3) ¨ Generic form: ¨ Dual theorem: If a linear program has a bounded optimum, then so does its dual, and the new optimum values coincide. ¤ Max-flow min-cut Linear programming 11 IRIS H.-R. JIANG The Simplex Algorithm Linear programming 12 Every constraint specifies an n-dimensional half-space Travel along “edges” until no improvement can be made
- 13. IRIS H.-R. JIANG Vertex and Neighbors ¨ Pick a subset of the inequalities. If there is a unique point that satisfies them with equality, and this point happens to be feasible, then it is a vertex ¤ {2, 3, 7} ® A ¤ {4, 6} ® no vertex ¨ Two vertices are neighbors if they have n - 1 defining inequalities in common ¤ {2, 3, 7} ® A ¤ {1, 3, 7} ® C 13 Linear programming IRIS H.-R. JIANG The Simplex Algorithm ¨ On each iteration, simplex has two tasks: ¤ Task 1: Check whether the current vertex is optimal ¤ Task 2: Determine where to move next ¨ Both tasks are easy if the vertex happens to be at the origin ¤ Transform the coordinate system to move vertex u to the origin ¨ Task 1: ¤ The origin is optimal if and only if all ci <= 0 ¨ Task 2: ¤ We can move by increasing some xi for which ci > 0 ¤ Until we hit some other constraint 14 Linear programming IRIS H.-R. JIANG Example (1/3) Linear programming 15 IRIS H.-R. JIANG Example (2/3) Linear programming 16
- 14. IRIS H.-R. JIANG Vertex and Neighbors ¨ Pick a subset of the inequalities. If there is a unique point that satisfies them with equality, and this point happens to be feasible, then it is a vertex ¤ {2, 3, 7} ® A ¤ {4, 6} ® no vertex ¨ Two vertices are neighbors if they have n - 1 defining inequalities in common ¤ {2, 3, 7} ® A ¤ {1, 3, 7} ® C 13 Linear programming IRIS H.-R. JIANG The Simplex Algorithm ¨ On each iteration, simplex has two tasks: ¤ Task 1: Check whether the current vertex is optimal ¤ Task 2: Determine where to move next ¨ Both tasks are easy if the vertex happens to be at the origin ¤ Transform the coordinate system to move vertex u to the origin ¨ Task 1: ¤ The origin is optimal if and only if all ci <= 0 ¨ Task 2: ¤ We can move by increasing some xi for which ci > 0 ¤ Until we hit some other constraint 14 Linear programming IRIS H.-R. JIANG Example (1/3) Linear programming 15 IRIS H.-R. JIANG Example (2/3) Linear programming 16
- 15. IRIS H.-R. JIANG Vertex and Neighbors ¨ Pick a subset of the inequalities. If there is a unique point that satisfies them with equality, and this point happens to be feasible, then it is a vertex ¤ {2, 3, 7} ® A ¤ {4, 6} ® no vertex ¨ Two vertices are neighbors if they have n - 1 defining inequalities in common ¤ {2, 3, 7} ® A ¤ {1, 3, 7} ® C 13 Linear programming IRIS H.-R. JIANG The Simplex Algorithm ¨ On each iteration, simplex has two tasks: ¤ Task 1: Check whether the current vertex is optimal ¤ Task 2: Determine where to move next ¨ Both tasks are easy if the vertex happens to be at the origin ¤ Transform the coordinate system to move vertex u to the origin ¨ Task 1: ¤ The origin is optimal if and only if all ci <= 0 ¨ Task 2: ¤ We can move by increasing some xi for which ci > 0 ¤ Until we hit some other constraint 14 Linear programming IRIS H.-R. JIANG Example (1/3) Linear programming 15 IRIS H.-R. JIANG Example (2/3) Linear programming 16
- 16. IRIS H.-R. JIANG Vertex and Neighbors ¨ Pick a subset of the inequalities. If there is a unique point that satisfies them with equality, and this point happens to be feasible, then it is a vertex ¤ {2, 3, 7} ® A ¤ {4, 6} ® no vertex ¨ Two vertices are neighbors if they have n - 1 defining inequalities in common ¤ {2, 3, 7} ® A ¤ {1, 3, 7} ® C 13 Linear programming IRIS H.-R. JIANG The Simplex Algorithm ¨ On each iteration, simplex has two tasks: ¤ Task 1: Check whether the current vertex is optimal ¤ Task 2: Determine where to move next ¨ Both tasks are easy if the vertex happens to be at the origin ¤ Transform the coordinate system to move vertex u to the origin ¨ Task 1: ¤ The origin is optimal if and only if all ci <= 0 ¨ Task 2: ¤ We can move by increasing some xi for which ci > 0 ¤ Until we hit some other constraint 14 Linear programming IRIS H.-R. JIANG Example (1/3) Linear programming 15 IRIS H.-R. JIANG Example (2/3) Linear programming 16
- 17. IRIS H.-R. JIANG Example (3/3) 17 Linear programming IRIS H.-R. JIANG Standard Form ¨ Variants ¤ Either a maximization or a minimization problem ¤ Constraints can be equations and/or inequalities ¤ Variables are restricted to be nonnegative or unrestricted in sign ¨ Standard form ¤ Objective function: minimization ¤ Constraints: equations ¤ Variables: nonnegative Linear programming 18 Slack variables
- 18. IRIS H.-R. JIANG Example (3/3) 17 Linear programming IRIS H.-R. JIANG Standard Form ¨ Variants ¤ Either a maximization or a minimization problem ¤ Constraints can be equations and/or inequalities ¤ Variables are restricted to be nonnegative or unrestricted in sign ¨ Standard form ¤ Objective function: minimization ¤ Constraints: equations ¤ Variables: nonnegative Linear programming 18 Slack variables