Lecture2b algorithm
- 1. Lecture 2 Algorithm Part 2 23/10/2018 Lecture 2 Algorithm Part 2 1
- 2. Content Lecture 2 Algorithm Part 2 The Towers of Hanoi Permutation Backtracking and Branch-And-Bound The n-queens problem Maze Travelling Salesman NP-complete class 23/10/2018 Lecture 2 Algorithm Part 2 2
- 3. The Towers of Hanoi Overview The towers of Hanoi is a mathematical game or puzzle Developed in 1883 by a French mathematician Édouard Lucas The Game There are 3 rods with n disks in different sizes At first all disks are placed at the first rod in order so that a bigger size disk is below a smaller size disk 23/10/2018 Lecture 2 Algorithm Part 2 3
- 4. The Towers of Hanoi The objective of the puzzle is to move the entire stack to another rod, obeying the following rules: Only one disk may be moved at a time Each move consists of taking the upper disk from one of the rods and sliding it onto another rod You can put the disk on top of other disks which may already be present on that rod But no disk may be placed on top of a smaller disk Therefore it is only allowed to place a smaller disk on the top of a bigger disk 23/10/2018 Lecture 2 Algorithm Part 2 4
- 5. The Towers of Hanoi Example 3 rods 3 disks 23/10/2018 Lecture 2 Algorithm Part 2 5
- 6. The Towers of Hanoi To solve this game that means moving n disks from rod 1 to 3 the following process is considered: the first n-1 disks are moved from rod 1 to rod 2 than the last disk remaining on 1 are moved to 3 than n-1 disks are moved from 2 to 3 That means the roles of the three rods are always different In total you need 2n – 1 moves This is minimal Example: for 3 disk you need 23 – 1 moves = 7 23/10/2018 Lecture 2 Algorithm Part 2 6
- 7. The Towers of Hanoi 23/10/2018 Lecture 2 Algorithm Part 2 7 Move disks a and b to rod 2 Move disk c from rod 1 to 3 Move disks a and b from rod 2 to 3 Move disk a from 1 to 3 Move disk b from rod 1 to 2 Move disk a from 3 to 2 1 2 3 1 2 3 How to move disk c from 1 to 3: How to move disks a and b from 1 to 2: Note: the biggest disk is c, the middle disk is b and the smallest disk is a!
- 8. Permutation A permutation is a sequence of n different object in a row The order of the objects is altered by swapping the elements so that each possibility is reached There are n! different permutations for n objects Example A permutation of the three objects a, b, c would be: abc bac cab acb bca cba n = 3 3! = 3*2 *1 = 6 permutations 23/10/2018 Lecture 2 Algorithm Part 2 8
- 9. Permutation Problem Generate all possible permutation out of n object 23/10/2018 Lecture 2 Algorithm Part 2 9
- 10. Permutation Solution Two solution methods exist for this problem Method 1 For each permutation a1 a2 … an-1 you generate n new ones by putting the number n at all possible places: n a1 a2 … an-1 a1 n a2 …. an-1 …. a1 a2 … an-1 n 23/10/2018 Lecture 2 Algorithm Part 2 10
- 11. Permutation Example From the permutation 1 we got 2 new permutation using Method 1 by placing the new number 2 at all possible places: 21 12 From the permutation 21 we got three new permutation using Method 1 by placing the new number 3 at all possible places: 321 231 213 From the permutation 12 we got three new permutation using Method 1 by placing the new number 3 at all possible places: 312 132 123 These are all 6 possible permutation of the numbers 1, 2 and 3 23/10/2018 Algorithm Lecture 3 11
- 12. Permutation From the permutation 321 we got four new permutation using Method 1 by placing the new number 4 at all possible places: 4321 3421 3241 3214 From the permutation 231 we got four new permutation using Method 1 by placing the new number 4 at all possible places: 4231 2431 2341 2314 If you do this for all 6 given permutation of the numbers 1,2 and 3 you will receive 24 permutation of the numbers 1,2,3 and 4 23/10/2018 Algorithm Lecture 3 12
- 13. Permutation Method 2 For each permutation a1 a2 … an-1 an integer k with 1 ≤ k ≤ n is added and each ai is increased by 1 if ai ≥ k. Example From the permutation 231 we got 4 new permutation using Method 2 by adding 1, 2, 3, and 4: 3421 3412 2413 2314 Note The second method is more efficient if the permutation are placed in an array because disarrange of parts of the array is not necessary 23/10/2018 Lecture 2 Algorithm Part 2 13
- 14. Permutation Example Given permutation 321: 23/10/2018 Algorithm Lecture 3 14 3 2 1 3 2 1 3 2 1 3 2 1 4 3 2 1 4 3 1 2 4 2 1 3 3 2 1 4 Result First row: 1 compared with 3 3 > 1 3 + 1 = 4 1 compared with 2 2 > 1 2 + 1 = 3 1 compared with 1 1 = 1 1 + 1 = 2 Second row: 2 compared with 3 3 > 2 3 + 1 = 4 2 compared with 2 2 = 2 2 + 1 = 3 2 compared with 1 1 < 2 remains Third row: 3 compared with 3 3 = 3 3 + 1 = 4 3 compared with 2 2 < 3 remains 3 compared with 1 1 < 3 remains Fourth row: 4 compared with 3 3 < 4 remains 4 compared with 2 2 < 4 remains 4 compared with 1 1 < 4 remains
- 15. The n-queens problem This is a part of the chess game (pawn, king, queen, bishop, knight, rook) It considers only the queen token A queen on a chessboard threaten all fields in the same row in the same column on both diagonals Challenge: Find a position for n queens on a n x n chessboard such that they not threaten each other 23/10/2018 Lecture 2 Algorithm Part 2 15
- 16. The n-queens problem 23/10/2018 Algorithm Lecture 3 16 Example n = 4 4 queens are placed on a 4 x 4 chessboard not considered the threating You can place any queen in any row or column For the first queen you have 16 possibilities, for the second queen 15, for the third 14 and for the fourth 13 16*15*14*13 = 43680 Therefore if you try any possibility you would need a lot of time How can you solve this problem?
- 17. The n-queens problem Idea Build up a solution step by step by putting each queen one by one on the chessboard so that it is not threaten by any other queen If there is no possible place left for the queen k the k-1 queen is removed from the chessboard and placed on another not already tried position This is done as long as a solution is found or all possibilities are tried This is an example of a method called Backtracking! 23/10/2018 Lecture 2 Algorithm Part 2 17
- 18. The n-queens problem Definition A method in which every possibility is tested and in the case of a death end the step before is withdrawn and a new variant is tested is called a backtracking method. Used to solve constraint satisfaction problems like crosswords and puzzles Often the most efficient solution for parsing and optimization problems 23/10/2018 Lecture 2 Algorithm Part 2 18
- 19. The n-queens problem Backtracking is a systematical search in the whole set to find acceptable states In the most cases in backtracking it is very important to see as early as possible death ends to avoid a performance increase A death end is a state in the given problem where you can not do the next step without violating any of the criteria of the given problem Therefore you have to go back to the previous state (backtracking) 23/10/2018 Lecture 2 Algorithm Part 2 19
- 20. The n-queens problem A death end in the n-queen problem for n = 8 would be the following illustration: It is not possible to set another queen To find a solution you have to even remove all queens beside the first one Only if the second queen is placed from c7 to e7 (or f7/g7) a solution can be found 23/10/2018 Lecture 2 Algorithm Part 2 20
- 21. The n-queens problem If you try a step by step method for backtracking you have to consider that The numbers of possible moves are limited in a clever way To check immediately if the partial solution still fulfill the necessary criteria which are needed for the whole solution In the case of the n-queen problem you have (n2!)/(n2–n)! possibilities to place n queens on a chessboard (not considered the threating) For n = 8 this are 64!/56! ~ 1014 possibilities Extreme performance killing case if you try every possibility! 23/10/2018 Lecture 2 Algorithm Part 2 21
- 22. The n-queens problem If you look at the n-queen problem it is therefore necessary to: Consider for the queen k only the kth row on the chessboard for the next position To check for each new queen immediately if she is threaten by the other queens on the chessboard If you consider only the kth row for the queen k than the number of possibilities is reduced to nn 23/10/2018 Lecture 2 Algorithm Part 2 22
- 23. The n-queens problem Example n = 4 • In this case: 44 = 256. • If you threw away all position where even less then n queens threaten each other than only 17 possibilities are left • Under these possibilities there are only 2 solutions and 4 death ends 23/10/2018 Lecture 2 Algorithm Part 2 23
- 24. The n-queens problem Chose the data structure The efficiency of the backtracking algorithm depends strongly on the chosen data structure How efficient is The check if a solution is reached The identification of possible steps in a given situation The check of the usefulness of a partial solution The execution of a step If possible each step should have an effort of O(1) You can improve for example the implementation of the n-queen problem if you save the amount of threaten row, columns and diagonals 23/10/2018 Lecture 2 Algorithm Part 2 24
- 25. Backtracking Other examples for back tracking knapsack problem A knapsack/rucksack can carry a weight W. Given n objects with a certain value and weight. Problem: Objects should be chosen in such a way that a maximal value will be acquired but the total weight of the knapsack is not exceeded Dye Problem Given is a topographic map with N countries which are going to be dyed with c different colours. Problem: To find a colour scheme so that all countries with the same border having a different colour. Solitaire Sudoku The figures 1 to 9 are placed in a 9x9 matrix divided into 9 3x3 fields after particular rules 23/10/2018 Lecture 2 Algorithm Part 2 25
- 26. Backtracking 23/10/2018 Algorithm Lecture 3 26 Example
- 27. Backtracking 23/10/2018 Lecture 2 Algorithm Part 2 27 Example
- 28. Maze A maze is an intricate and complex network of interconnecting paths Sometimes placed in gardens Other name: labyrinth Starting point S somewhere in the maze Problem To find the way out of a maze 23/10/2018 Lecture 2 Algorithm Part 2 28 S
- 29. Maze Solution Mark each point used as starting for a new path (like crumb of bread) You move in one direction until you reach a wall Than you turn either to the right or to the left If you reach a death end you unmark the point and you go a step back and try the other direction (left/right) To find the shortest way out you can use the branch–and–bound method 23/10/2018 Lecture 2 Algorithm Part 2 29 . .. . . . .. .. .. . . . . .. . . . . .. . . . . S .. .. .. .. ..
- 30. Travelling Salesman Description Be n a number of place in a n x n distance matrix M where Mi,j is the distance between the places i and j Seek for the route with the minimal length such that all places are reached exactly one time and then return to the starting point If you seek for an optimization or improvement than a lot of branches can be cut off which are only produce inefficient solutions 23/10/2018 Lecture 2 Algorithm Part 2 30
- 31. Travelling Salesman 23/10/2018 Lecture 2 Algorithm Part 2 31 Definition A Branch-and-bound method is a general algorithm for finding an optimal solution of various optimization problems. It consists of a systematic enumeration of all possible solutions where inefficient solutions are eliminated by using upper and lower estimated bounds Be L the solution space and c: L R a function and lo a solution to find such that: c(lo) ≤ k for a given bound k or c(lo) ≤ c(l) for all l ϵ L (global minimum)
- 32. Travelling Salesman It is reasonable to find another function c’: L’ R such that c’ is an estimate efficient lower bound for all part solution l’ ϵ L’. l’ is derived from l The function c’ can be used to decide if it is efficient to pursue this part solution l’ or if it would be better to cut this brunch 23/10/2018 Lecture 2 Algorithm Part 2 32
- 33. Travelling Salesman In the travelling salesman problem L is the amount of all permutation over all the places 1…n and the function c the length of a route The function for a part solution can consider the already travelled distance and an approximation for the length of the route to the not reached places If you just try every permutation the effort is O(n!) (or O((n-1)!) The branch-and-bound method helps to reduce the effort but this is not a grantee 23/10/2018 Lecture 2 Algorithm Part 2 33
- 34. Travelling Salesman There is still no algorithm to really reduce the effort The problem is therefore computationally difficult but a large number of heuristics and exact methods are known, so that some instances with tens of thousands of cities can be solved This problem belongs to a set of so called NP-complete problems The theory is that if you solve one of the NP-complete problems all of them are solvable 23/10/2018 Lecture 2 Algorithm Part 2 34
- 35. Travelling Salesman Other branch-and-bound problems Canadian traveller problem Vehicle routing problem Route inspection problem Set TSP problem Seven Bridges of Königsberg Traveling repairman problem (minimum latency problem) Traveling tourist problem Tube Challenge 23/10/2018 Lecture 2 Algorithm Part 2 35
- 36. NP-complete class Algorithm can provide an answer in polynomial time are called class P or just P NP problems are Nondeterministic Polynomial Time (NP) problems NP-hard problems can be described by: “at least as hard as the hardest problem in NP” 23/10/2018 Lecture 2 Algorithm Part 2 36
- 37. NP-complete class In computational complexity theory, the complexity class NP-complete (NP-C or NPC) is a class of decision problems. A problem L is NP-complete if it has two properties: It is in the set of Nondeterministic Polynomial Time (NP) problems: Any given solution to L can be verified quickly (in polynomial time) It is also in the set of NP-hard problems: Any NP- problem can be converted into L by a transformation of the inputs in polynomial time 23/10/2018 Lecture 2 Algorithm Part 2 37
- 38. NP-complete class Euler diagram for P, NP, NP-complete, and NP-hard set of problems It is still unknown if P = NP or P ≠NP 23/10/2018 Lecture 2 Algorithm Part 2 38
- 39. NP-complete class A given solution to such a problem can be verified quickly There is no known way so far to locate a solution in the first place that means no fast solution is known to the NP- complete problems This implies that the time required solving the problem using any known algorithm increase very quickly Even for moderately large version of many of these problems the required time reaches into billions of years Therefore one of the unsolved problems in computer science today is the so called P versus NP problem 23/10/2018 Lecture 2 Algorithm Part 2 39
- 40. Any questions? 23/10/2018 Lecture 2 Algorithm Part 2 40
Editor's Notes
- #19: In computer science and linguistics, parsing, or, more formally, syntactic analysis, is the process Parsing: of analyzing a text, made of a sequence of tokens (for example, words), to determine its grammatical structure with respect to a given (more or less) formal grammar.