Data Analysis and Algorithms Lecture 1: Introduction

Design and Analysis of
Algorithms

Course Outline
1. Introduction to Algorithms
2. Mathematical Preliminaries
3. Growth Functions
4. Recurrences
5. Overview of Data Structures
6. Sorting and Searching Algorithms
7. Tree Algorithms
8. Graph Algorithms
9. Strings and Pattern Matching
10. Dynamic Programming
11. Theory of NP-Completeness

Reference Books
1. T. Corman, E. Leiserson, L. Rivest, C. Stein, Introduction to
Algorithms, 3rd Edition, Prentice Hall
2. R. Sedgewik, P. Flajolet, An Introduction to the Analysis of
Algorithms, 2nd Edition, Addison-Wesley
3. A. Levitin, Introduction to Design and Analysis of Algorithms, 3rd
Edition, Pearson Education Inc

Algorithms - History
• Procedures for solving geometric and arithmetic problems were formulated by ancient
Greeks.
• Some two thousands years ago, the celebrated procedure for finding greatest
common divisor (g c d) was discovered by Euclid.
• In tenth century, Persian mathematician Muhammad- inbne- Musa Al lkowarzimi
stated procedures for solving algebraic equations. The procedure are described in his
famous book Kitab al Jabr wa’l muqabla (‘Rules of restoring and equating’).
The word algorithm is coined from Alkhowarizm
• In the mid- twentieth century D.E. Knuth undertook in depth study and analysis of
algorithms. His work is embodied in the comprehensive book ‘Art of Computer
Programming’, which serves as foundation for modern study of algorithms.

Algorithms – Formal Definition
An algorithm is an orderly step-by-step procedure, which has the
characteristics:
1) It accepts one or more input value
2) It returns at least one output value
3) It terminates after finite steps

Algorithms – Applications
• Sorting Algorithms (Elementary and Advanced)
• Searching Algorithms (Linear and Non-linear)
• Strings Processing ( Pattern matching, Parsing, Compression,
Cryptography)
• Image Processing ( Compression, Matching, Conversion)
• Mathematical Algorithms (Random number generator, matrix
operations, FFT, etc)

Algorithms – Classic Problems
• Hamiltonian Circuit Problem
• Traveling Salesperson Problem
• Graph Coloring Problem
• The Sum-Set Problem
• The n-Queen Problem

Hamiltonian Circuit Problem
Problem: Determine if a given graph has a Hamiltonian circuit.
• A Hamiltonian circuit , also called tour, is path that starts and ends at
the same vertex, and passes through all other vertices exactly once.
• A graph can have more than one Hamiltonian circuit, or none at all.

Traveling Salesperson Problem
Problem(a): A salesperson has to travel to n cities, such he starts at one city
and ends up at the same city, visits each city exactly once , and covers
minimum distance.
Problem(b): Determine minimum tour for a graph with n vertices.

Graph Coloring Problem
• Problem :. For given graph, find the minimum number of colors that can be
used to color the vertices so that no two adjacent vertices have the same
color.

Sum of Subset Problem
Problem : Given a set of set S = {s1, s2, s3, … sn} of n positive integers and an
integer w, find all subsets of S such that sum of integers, in each subset, equals
w.
Suppose S = {s1, s2, s3, s4, s5}
Let s1=5, s2 = 6, s3 = 10, s4 = 11 ,s5 = 16, and w = 21
• s3 + s4 = 10 + 11 = 21
In general, for a set of n integers, there are 2n subsets need to be examined.
For example, for n=20, the number of subsets is 220 = 1,048,576

Sum of Subset Problem
Problem : Given a set of set S = {s1, s2, s3, … sn} of n positive integers and an
integer w, find all subsets of S such that sum of integers, in each subset, equals
w.
Suppose S = {s1, s2, s3, s4, s5}
Let s1=5, s2 = 6, s3 = 10, s4 = 11 ,s5 = 16, and w = 21
• s1 + s2 + s3 = 5 + 6 + 10 =21
• s1 + s5 = 5 + 16 = 21
• s3 + s4 = 10 + 11 = 21
In general, for a set of n integers, there are 2n subsets need to be examined.
For example, for n=20, the number of subsets is 220 = 1,048,576

n-Queen Problem
Problem : n queens are to be placed on an n by n chessboard so that no two
queens threat each other. A queen threats another queen if is on the same row,
or same column, or same diagonal of the chess board.

Algorithm Design
• Divide-and-Conquer Algorithm
• Greedy Algorithm
• Backtracking Algorithm
• Dynamic Programming
• Brute Force
• Approximation Algorithm
• Randomized Algorithm

Divide-and-Conquer Algorithm
The divide and conquer algorithm has the following approach:
1) The problem is split (divided) into two distinct independent sub problems
2) The sub problems are solved (conquered) separately
3) The solutions to the sub problems are merged together to obtain solution to
main problem.

Problem: Sort the set of numbers {25,10, 8, 45, 67, 33, 20, 59} in ascending
order.

• Quick Sort
• Merge Sort
• Binary Search
• Multiplication of Large Integers
• Matrix Multiplication
• Fast Fourier Transform

Divide and conquer algorithm provides efficient sorting and searching methods.
It, however, has following limitations:
• The divide-and-conquer algorithm is implemented using recursive function
calls, which is not supported by some of the programming languages.
• Recursive calls need large stacks to store intermediate results, utilize
computer’s special memory called heap which may not be available on some
systems and decrease data access speed
• The divide-and-conquer algorithm is in-efficient when sub problems are not
of nearly equal size.

Greedy Algorithm
Greedy Algorithm is designed to solve optimization problems. The greedy algorithm
has the follows approach:
1) The problem is solved in steps. At each step, several choices are available for
partial solutions. A greedy choice is made by selecting the option with maximum
( or minimum) cost. This is referred to as locally optimum solution, as opposed
to globally optimum solution, which is expected to be the overall optimum
solution.
2) It is checked if the selected option satisfies problems constraint ( for example, the
cost does not exceed the prescribed limit ).This is referred to as feasible solution.
3) If current solution is not feasible, the next best option is selected.
4) Once a choice is made, it is not changed in subsequent steps. This condition is
referred to as irrevocability

Greedy Algorithm
Problem: ATM is to deliver a change for Rs. 98, with minimum number of
currency notes/coins.

Greedy Algorithm
Minimum Cabling in network
• Prim’s Algorithm
• Kruskal’s Algorithm
Shortest route in a network
• Dijkstara Algorithm
Huffman coding for text compression
Optimum Job Scheduling

Greedy Algorithm
The greedy algorithm provides is a simple and elegant solution to optimization
problems. It has, however, the following limitations:
• The greedy algorithm does not always produce a globally optimum solution ,
(for example, in case of traveling salesperson problem).
• In order to check whether or not the resulting solution is optimal, further
analysis is required.
• Often the analysis for globally optimum solution is quite complex.
• Analysis may requires additional resources.

Data Analysis and Algorithms Lecture 1: Introduction

More Related Content

Similar to Data Analysis and Algorithms Lecture 1: Introduction (20)

Recently uploaded (20)

Data Analysis and Algorithms Lecture 1: Introduction