19. Data Structures and Algorithm Complexity

SoftUni Team
Technical Trainers
Software University
http://softuni.bg
Data Structures,
Algorithms and Complexity
Analyzing Algorithm Complexity.
Asymptotic Notation

Table of Contents
1. Data Structures
 Linear Structures, Trees, Hash Tables, Others
2. Algorithms
 Sorting and Searching, Combinatorics,
Dynamic Programming, Graphs, Others
3. Complexity of Algorithms
 Time and Space Complexity
 Mean, Average and Worst Case
 Asymptotic Notation O(g)
2

4
 Examples of data structures:
 Person structure (first name + last name + age)
 Array of integers – int[]
 List of strings – List<string>
 Queue of people – Queue<Person>
What is a Data Structure?
“In computer science, a data structure is a particular
way of storing and organizing data in a computer so
that it can be used efficiently.” -- Wikipedia

5
Student Data Structure
struct Student {
string Name { get; set; }
short Age { get; set; } // Student age (< 128)
Gender Gender { get; set; }
int FacultyNumber { get; set; }
};
enum Gender
{
Male,
Female,
Other
}
Student
Name Maria Smith
Age 23
Gender Female
FacultyNumber SU2007333

6
Stack
data
data
data
data
data
data
data
top of the stack

7
Stack
class Stack<T>
{
// Pushes an elements at the top of the stack
void Push(T data);
// Extracts the element at the top of the stack
T Pop();
// Checks for empty stack
bool IsEmpty();
}
// The type T can be any data structure like
// int, string, DateTime, Student

8
Queue
data
data
data
data
data
data
data
Start of the queue: elements
are excluded from this position
End of the queue: new
elements are inserted here

9
Queue
class Queue<T>
{
// Appends an elements at the end of the queue
void Enqueue(T data);
// Extracts the element from the start of the queue
T Dequeue();
// Checks for empty queue
bool IsEmpty();
}
// The type T can be any data structure like
// int, string, DateTime, Student

10
Linked List
2
next
7
next
head
(list start)
4
next
5
next
null

11
Trees
17
15149
6 5 8
Project
Manager
Team
Leader
Designer
QA Team
Leader
Developer
1
Developer
2
Tester 1
Developer
3
Tester
2

13
Graphs 7
19
21
14
1
12
31
4
11
G
J
F
D
A
E C H
GA
H N
K

15
Hash-Table: Structure
h("Pesho") = 4
h("Kiro") = 2
h("Mimi") = 1
h("Ivan") = 2
h("Lili") = m-1
Ivan
null
null null
null Mimi Kiro null Pesho … Lili
0 1 2 3 4 … m-1
T
null
collision
Chaining the elements
in case of collision

16
 Data structures and algorithms are the foundation of computer
programming
 Algorithmic thinking, problem solving and data structures are
vital for software engineers
 C# developers should know when to use T[], LinkedList<T>,
List<T>, Stack<T>, Queue<T>, Dictionary<K, T>,
HashSet<T>, SortedDictionary<K, T> and SortedSet<T>
 Computational complexity is important for algorithm design and
efficient programming
Why Are Data Structures So Important?

 Primitive data types
 Numbers: int, float, double, decimal, …
 Text data: char, string, …
Simple structures
 A group of primitive fields stored together
 E.g. DateTime, Point, Rectangle, Color, …
 Collections
 A set / sequence of elements (of the same type)
 E.g. array, list, stack, queue, tree, hashtable, bag, …
Primitive Types and Collections
17

 An Abstract Data Type (ADT) is
 A data type together with the operations, whose properties are
specified independently of any particular implementation
 ADT is a set of definitions of operations
 Like the interfaces in C# / Java
 Defines what we can do with the structure
 ADT can have several different implementations
 Different implementations can have different efficiency, inner
logic and resource needs
Abstract Data Types (ADT)
18

 Linear structures
 Lists: fixed size and variable size sequences
 Stacks: LIFO (Last In First Out) structures
 Queues: FIFO (First In First Out) structures
 Trees and tree-like structures
 Binary, ordered search trees, balanced trees, etc.
 Dictionaries (maps, associative arrays)
 Hold pairs (key  value)
 Hash tables: use hash functions to search / insert
Basic Data Structures
19

20
 Sets, multi-sets and bags
 Set – collection of unique elements
 Bag – collection of non-unique elements
 Ordered sets and dictionaries
 Priority queues / heaps
 Special tree structures
 Suffix tree, interval tree, index tree, trie, rope, …
 Graphs
 Directed / undirected, weighted / unweighted,
connected / non-connected, cyclic / acyclic, …
Basic Data Structures (2)

22
 The term "algorithm" means "a sequence of steps"
 Derived from Muḥammad Al-Khwārizmī', a Persian mathematician
and astronomer
 He described an algorithm for solving quadratic equations in 825
What is an Algorithm?
“In mathematics and computer science, an algorithm is
a step-by-step procedure for calculations. An algorithm
is an effective method expressed as a finite list of well-
defined instructions for calculating a function.”
-- Wikipedia

23
 Algorithms are fundamental in programming
 Imperative (traditional, algorithmic) programming means to
describe in formal steps how to do something
 Algorithm == sequence of operations (steps)
 Can include branches (conditional blocks) and repeated logic (loops)
 Algorithmic thinking (mathematical thinking, logical thinking,
engineering thinking)
 Ability to decompose the problems into formal sequences of steps
(algorithms)
Algorithms in Computer Science

24
 Algorithms can be expressed as pseudocode, through flowcharts or
program code
Pseudocode and Flowcharts
BFS(node)
{
queue  node
while queue not empty
v  queue
print v
for each child c of v
queue  c
}
Pseudo-code Flowchart
public void DFS(Node node)
{
Print(node.Name);
for (int i = 0; i < node.
Children.Count; i++)
{
if (!visited[node.Id])
DFS(node.Children[i]);
}
visited[node.Id] = true;
}
Source code

 Sorting and searching
 Dynamic programming
 Graph algorithms
 DFS and BFS traversals
 Combinatorial algorithms
 Recursive algorithms
 Other algorithms
 Greedy algorithms, computational geometry, randomized
algorithms, parallel algorithms, genetic algorithms
25
Some Algorithms in Programming

Algorithm Complexity
Asymptotic Notation

27
 Why should we analyze algorithms?
 Predict the resources the algorithm will need
 Computational time (CPU consumption)
 Memory space (RAM consumption)
 Communication bandwidth consumption
 The expected running time of an algorithm is:
 The total number of primitive operations executed
(machine independent steps)
 Also known as algorithm complexity
Algorithm Analysis

28
 What to measure?
 CPU time
 Memory consumption
 Number of steps
 Number of particular operations
 Number of disk operations
 Number of network packets
 Asymptotic complexity
Algorithmic Complexity

29
 Worst-case
 An upper bound on the running time for any input of given size
 Typically algorithms performance is measured for their worst case
 Average-case
 The running time averaged over all possible inputs
 Used to measure algorithms that are repeated many times
 Best-case
 The lower bound on the running time (the optimal case)
Time Complexity

 Sequential search in a list of size n
 Worst-case:
 n comparisons
 Best-case:
 1 comparison
 Average-case:
 n/2 comparisons
 The algorithm runs in linear time
 Linear number of operations
30
Time Complexity: Example
… … … … … … …
n

31
 Algorithm complexity is a rough estimation of the number of steps
performed by given computation, depending on the size of the input
 Measured with asymptotic notation
 O(g) where g is a function of the size of the input data
 Examples:
 Linear complexity O(n)
 All elements are processed once (or constant number of times)
 Quadratic complexity O(n2)
 Each of the elements is processed n times
Algorithms Complexity

32
 Asymptotic upper bound
 O-notation (Big O notation)
 For a given function g(n), we denote by O(g(n)) the set of functions
that are different than g(n) by a constant
 Examples:
 3 * n2 + n/2 + 12 ∈ O(n2)
 4*n*log2(3*n+1) + 2*n-1 ∈ O(n * log n)
Asymptotic Notation: Definition
O(g(n)) = {f(n): there exist positive constants c and n0
such that f(n) <= c*g(n) for all n >= n0}

33
 О(n) means a function grows
linearly when n increases
 E.g.
 O(n2) means a function grows
exponentially when n increases
 E.g.
 O(1) means a function does not
grow when n changes
 E.g.
Functions Growth Rate
n
ƒ(n)
ƒ(n)=n+1
ƒ(n)=n2+2n+2
ƒ(n)=4 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0

34
Positive examples:
Examples
Negative examples:

36
Complexity Notation Description
constant O(1)
Constant number of operations, not
depending on the input data size, e.g.
n = 1 000 000  1-2 operations
logarithmic O(log n)
Number of operations proportional to
log2(n) where n is the size of the input
data, e.g.
n = 1 000 000 000  30 operations
linear O(n)
Number of operations proportional to the
input data size, e.g. n = 10 000  5 000
operations
Typical Complexities

37
Complexity Notation Description
quadratic O(n2)
Number of operations proportional to the
square of the size of the input data, e.g.
n = 500  250 000 operations
cubic O(n3)
Number of operations propor-tional to
the cube of the size of the input data, e.g.
n = 200  8 000 000 operations
exponential
O(2n),
O(kn),
O(n!)
Exponential number of operations, fast
growing, e.g. n = 20  1 048 576
operations
Typical Complexities (2)

39
Time Complexity and Program Speed
Complexity 10 20 50 100 1 000 10 000 100 000
O(1) < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s
O(log(n)) < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s
O(n) < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s
O(n*log(n)) < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s < 1 s
O(n2) < 1 s < 1 s < 1 s < 1 s < 1 s 2 s 3-4 min
O(n3) < 1 s < 1 s < 1 s < 1 s 20 s 5 hours 231 days
O(2n) < 1 s < 1 s 260 days hangs hangs hangs hangs
O(n!) < 1 s hangs hangs hangs hangs hangs hangs
O(nn) 3-4 min hangs hangs hangs hangs hangs hangs

40
 Complexity can be expressed as a formula of multiple variables, e.g.
 Algorithm filling a matrix of size n * m with the natural numbers 1,
2, … n*m will run in O(n*m)
 Algorithms to traverse a graph with n vertices and m edges will
take O(n + m) steps
 Memory consumption should also be considered, for example:
 Running time O(n) & memory requirement O(n2)
 n = 50 000  OutOfMemoryException
Time and Memory Complexity
Is it possible in
practice? In theory?

42
 A linear algorithm could be slower than a quadratic algorithm
 The hidden constant could be significant
 Example:
 Algorithm A performs 100*n steps  O(n)
 Algorithm B performs n*(n-1)/2 steps  O(n2)
 For n < 200, algorithm B is faster
 Real-world example:
 The "insertion sort" is faster than "quicksort" for n < 9
The Hidden Constant

43
Amortized Analysis
 Worst case: O(n2)
 Average case: O(n) – Why?
void AddOne(char[] chars, int m)
{
for (int i = 0; 1 != chars[i] && i < m; i++)
{
c[i] = 1 - c[i];
}
}

44
 Complexity: O(n3)
Using Math
sum = 0;
for (i = 1; i <= n; i++)
{
for (j = 1; j <= i*i; j++)
{
sum++;
}
}

45
 Just calculate and print sum = n * (n + 1) * (2n + 1) / 6
Using a Barometer
uint sum = 0;
for (i = 0; i < n; i++)
{
for (j = 0; j < i*i; j++)
{
sum++;
}
}
Console.WriteLine(sum);

46
 A polynomial-time algorithm has worst-case time complexity
bounded above by a polynomial function of its input size
 Examples:
 Polynomial time: 2n, 3n3 + 4n
 Exponential time: 2n, 3n, nk, n!
 Exponential algorithms hang for large input data sets
Polynomial Algorithms
W(n) ∈ O(p(n))

47
 Computational complexity theory divides the computational
problems into several classes:
Computational Classes

48
P vs. NP: Problem #1 in Computer Science Today

Analyzing the Complexity of Algorithms
Examples

 Runs in O(n) where n is the size of the array
 The number of elementary steps is ~ n
Complexity Examples
int FindMaxElement(int[] array)
{
int max = array[0];
for (int i = 1; i < array.length; i++)
{
if (array[i] > max)
{
max = array[i];
}
}
return max;
}
50

 Runs in O(n2) where n is the size of the array
 The number of elementary steps is ~ n * (n+1) / 2
Complexity Examples (2)
long FindInversions(int[] array)
{
long inversions = 0;
for (int i = 0; i < array.Length; i++)
for (int j = i + 1; j < array.Length; i++)
if (array[i] > array[j])
inversions++;
return inversions;
}
51

 Runs in cubic time O(n3)
 The number of elementary steps is ~ n3
decimal Sum3(int n)
{
decimal sum = 0;
for (int a = 0; a < n; a++)
for (int b = 0; b < n; b++)
for (int c = 0; c < n; c++)
sum += a * b * c;
return sum;
}
52

 Runs in quadratic time O(n*m)
 The number of elementary steps is ~ n*m
long SumMN(int n, int m)
{
long sum = 0;
for (int x = 0; x < n; x++)
for (int y = 0; y < m; y++)
sum += x * y;
return sum;
}
53

 Runs in quadratic time O(n*m)
 The number of elementary steps is ~ n*m + min(m,n)*n
long SumMN(int n, int m)
{
long sum = 0;
for (int x = 0; x < n; x++)
for (int y = 0; y < m; y++)
if (x == y)
for (int i = 0; i < n; i++)
sum += i * x * y;
return sum;
}
54

 Runs in exponential time O(2n)
 The number of elementary steps is ~ 2n
decimal Calculation(int n)
{
decimal result = 0;
for (int i = 0; i < (1<<n); i++)
result += i;
return result;
}
55

 Runs in linear time O(n)
 The number of elementary steps is ~ n
decimal Factorial(int n)
{
if (n == 0)
return 1;
else
return n * Factorial(n - 1);
}
56

 Runs in exponential time O(2n)
 The number of elementary steps is ~ Fib(n+1)
where Fib(k) is the kth Fibonacci's number
decimal Fibonacci(int n)
{
if (n == 0)
return 1;
else if (n == 1)
return 1;
else
return Fibonacci(n-1) + Fibonacci(n-2);
}
57

 fib(n) makes about fib(n) recursive calls
 The same value is calculated many, many times!
Fibonacci Recursion Tree
58

Lab Exercise
Calculating Complexity of Existing Code

60
 Data structures organize data in computer systems for efficient use
 Abstract data types (ADT) describe a set of operations
 Collections hold a group of elements
 Algorithms are sequences of steps for calculating / doing something
 Algorithm complexity is a rough estimation of the number of steps
performed by given computation
 Can be logarithmic, linear, n log n, square, cubic, exponential, etc.
 Complexity predicts the speed of given code before its execution
Summary

?
Data Structures, Algorithms and Complexity
https://softuni.bg/trainings/1308/data-structures-february-2016

License
 This course (slides, examples, labs, videos, homework, etc.)
is licensed under the "Creative Commons Attribution-
NonCommercial-ShareAlike 4.0 International" license
62
 Attribution: this work may contain portions from
 "Fundamentals of Computer Programming with C#" book by Svetlin Nakov & Co. under CC-BY-SA license
 "Data Structures and Algorithms" course by Telerik Academy under CC-BY-NC-SA license

Free Trainings @ Software University
 Software University Foundation – softuni.org
 Software University – High-Quality Education,
Profession and Job for Software Developers
 softuni.bg
 Software University @ Facebook
 facebook.com/SoftwareUniversity
 Software University @ YouTube
 youtube.com/SoftwareUniversity
 Software University Forums – forum.softuni.bg

19. Data Structures and Algorithm Complexity

More Related Content

What's hot (20)

Similar to 19. Data Structures and Algorithm Complexity (20)

More from Intro C# Book (20)

Recently uploaded (20)

19. Data Structures and Algorithm Complexity

Editor's Notes