![Heuristic
● Experience based techniques for problem
solving, learning and discovery. [Adopted from
Wikipedia]
● Different types
Rule of thumb
–
– Common sense
– Educated Guess
● Meta-heuristics: Parameters that influence
employing a heuristic](https://image.slidesharecdn.com/optimizationheuristicsslideshare-130201014236-phpapp01/85/Optimization-Heuristics-3-320.jpg)
![Swarm Intelligence
● A collective behavior of self-organized
systems which are decentralized [Adopted from
Wikipedia]
● Can't predict how the systems behave
even without a centralized control
● Widely employed in Artificial Intelligence
● Example
– Ant Colony Optimization
● Natural ants ≈ Simulation agents
● Pheromones ≈ Recording position, quality](https://image.slidesharecdn.com/optimizationheuristicsslideshare-130201014236-phpapp01/85/Optimization-Heuristics-15-320.jpg)
Optimization Heuristics
- 1. A presentation on Optimization Heuristics by Kausal Malladi (Student, IIIT Bangalore)
- 2. Agenda ● Definition of a Heuristic ● Optimization heuristics – Genetic Algorithms – Hill Climbing – Tabu Search – Simulated Annealing – Swarm Intelligence ● With example applications
- 3. Heuristic ● Experience based techniques for problem solving, learning and discovery. [Adopted from Wikipedia] ● Different types Rule of thumb – – Common sense – Educated Guess ● Meta-heuristics: Parameters that influence employing a heuristic
- 4. Optimization Heuristics ● Always difficult to solve NP-Hard and NP- Complete computational problems ● Even with different optimization techniques, actual running time is never guaranteed ● We employ some rules / results based on experiments to state that a near-optimal solution can be obtained ● No proof as to why and how we get solution
- 5. Genetic Algorithms ● A heuristic that mimics natural evolution ● A population of Candidate Solutions evolved towards better solutions ● Generations ● Requires – Genetic Representation of solution domain – Fitness function to evaluate solution ● Applications: Game Theory
- 6. Local Search ● To solve hard Optimization problems ● Search Space : Domain of function to be optimized ● Finding a solution among number of candidate solutions, maximizing a criterion ● Sub-families: – Hill Climbing – Tabu Search – Simulated Annealing
- 7. Hill Climbing ● Iterative algorithm, starts with arbitrary solution ● Looks for better solutions incrementally ● Repeats until no further improvements ● Good for finding a local optimum ● Doesn't guarantee global optimum ● Simple, popular ● Works well, generally
- 8. Hill Climbing ● Popular example – TSP – Travelling Salesman Problem ● Known NP-Hard problem ● Initial solution may not be optimal ● Shorter route is more likely to be obtained ● Widely used in Artificial Intelligence ● Significant results in real-time systems ● Any-time algorithm ● Pitfall: Plateau
- 9. Tabu Search ● Iteratively proceeds from one potential solution S to an improved one S' in the neighbourhood of S ● Overcomes few pitfalls of other Local Search techniques (Example: Plateau) ● Visited solutions marked “tabu” ● Search progresses using Memory Structures ● Often a benchmark heuristic!
- 10. Tabu Search ● Memory structures – Describe ● Visited solutions ● User provided sets of rules – Categories ● Short term ● Intermediate term ● Long term ● Form tabu list
- 11. Tabu Search ● Issues – Only effective in discrete search spaces ● Workaround: A similarity measure – High dimensional search space ● Workaround: Create a tabu list consisting of attributes of a solution ● Can be more effective solution, has problems too ● Aspiration criteria introduced – Override solution's tabu state
- 12. Tabu Search ● Common example – TSP – Travelling Salesman Problem ● Tabu Search finds a satisficing solution ● Starts with an initial solution that can be found randomly or using some algorithm ● Order in which two cities are visited, is swapped ● Total travelling distance is the metric ● A acceptable solution added to tabu list if in neighbourhood of accepted solution
- 13. Simulated Annealing ● Inspiration: Annealing in Metallurgy ● Probabilistic meta-heuristic ● Approximates global optimum in a large search space ● Gives acceptably good solution if not the best ● Slow decrease in probability of accepting worse solutions
- 14. Simulated Annealing ● Example – TSP – Travelling Salesman Problem ● Metric under consideration is Mileage ● Metropolis Algorithm ● Pairwise changing order of visit to cities – Solutions that don't lower mileage also accepted ● e-∆D/T > R(0,1) – ∆D is the change of distance implied ● If T is large, many bad choices are made
- 15. Swarm Intelligence ● A collective behavior of self-organized systems which are decentralized [Adopted from Wikipedia] ● Can't predict how the systems behave even without a centralized control ● Widely employed in Artificial Intelligence ● Example – Ant Colony Optimization ● Natural ants ≈ Simulation agents ● Pheromones ≈ Recording position, quality
- 16. References ● http://www.iaeng.org/publication/WCE2007/WCE2007_pp61-64.pdf (Game Theory using Genetic Algorithms) ● http://mathworld.wolfram.com/SimulatedAnnealing.html (Simulated Annealing) ● http://artificialintelligence-notes.blogspot.in/2010/07/hill-climbing-procedure.htm (Hill Climbing in Artificial Intelligence)
- 17. Thank you!