Lecture 2 : Role of Algorithms in Computing

Jayavignesh T

Asst Professor

SENSE

What is an Algorithm?

• An algorithm is a set of rules for carrying out calculation either by hand or on a machine.

• An algorithm is a finite step-by-step procedure to achieve a required result.

• An algorithm is a sequence of computational steps that transform the input into the output.

• An algorithm is a sequence of operations performed on data that have to be organized in data structures.

• An algorithm is an abstraction of a program to be executed on a physical machine (model of Computation).

What is an Algorithm contd..?

• An algorithm is a sequence of unambiguous instructions for solving a problem,

– i.e., for obtaining a required output for any legitimate input in a finite amount of time

Requirements

• Recipe, process, method, technique, procedure, routine,… with following requirements:

1. Finiteness

– terminates after a finite number of steps

2. Definiteness

– rigorously and unambiguously specified

3. Input

– valid inputs are clearly specified

4. Output

– can be proved to produce the correct output given a valid input

5. Effectiveness

– steps are sufficiently simple and basic

Design and Analysis

• Algorithmic is a branch of computer science that consists of designing and analyzing computer algorithms

• The “design” pertain to

– The description of algorithm at an abstract level by means of a pseudo language, and

– Proof of correctness that is, the algorithm solves the given problem in all cases.

• The “analysis” deals with performance evaluation(complexity analysis).

Design and Analysis

• It is not depended on programming language, machine.

• Are mathematical entities, which can be thought of as running on some sort of idealized computer with an infinite random access memory

• Algorithm design is all about the mathematical theory behind the design of good programs.

Why Analyze?

• Why analyze algorithms?

– Evaluate Algorithm performance

– Compare Different Algorithms

• Analyze what about them?

– Running time, Memory usage, Solution quality

– Worst-case and “typical” case

• Computational complexity

– Understanding the intrinsic difficulty of computational problems – Classifying problems according to difficulty

– Algorithms provide upper bound to show problem is hard, must show that any algorithm to solve it requires at least a given amount of resources

Algorithm Problem solving

Understand the problem

Decision Making

Design an Algorithm

Prove Correctness

Analyze Algorithm

Code the Algorithm

Understand the problem

• Necessary Inputs to solve the problem?

• Input – Instances

• Decide range of inputs

• To Fix Boundary values

• Work correctly for all Valid inputs

Decision Making

• Analyze the required inputs and decide on

– Computational Means

– Exact vs Approximate Solving

– Data Structures

– Algorithm Design Technique

Computational Means

Computational capability of running devices

Sequential Algorithm ( Random Access Machine RAM)

Parallel Algorithm (Piece executed on multiple processing devices and put back together to get correct result)

Proper choice of Computational device (Space/Time efficient)

Decision Making contd..

• Exact vs Approximate Solving

• Exact Algorithm

– Problem needs to be solved exactly or correctly

• Approximation Algorithm

– Solve a complex problem, we will not get exact solution

– Ex : Travelling salesman problem

• Data Structures

– Algorithm + Data Structures – Programming Language

– Proper Choice of Data Structures required before designing actual algorithm

Algorithm Design Technique

• Strategy or Paradigm

– General approach to solve program algorithmically

– Different problems from Different areas of computing

• Brute Force :

– Straight forward technique with naïve approach

• Divide and Conquer :

– Problem is divided into smaller instances

Algorithm Design Technique contd..

• Decrease and Conquer :

– Instance size is decreased to solve the problem

• Transform and Conquer :

– Instance is modified and then solved

• Dynamic Programming :

– Results of smaller, reoccurring instances are obtained to solve problem

• Greedy Technique :

– Solve the problem by making locally optimal decisions

Specification of Algorithm

• Natural Language

– Drawback : Specification by this means not clear

• Pseudo Code

– Mixture of Natural Language + Programming constructs

ALGORITHM Sum(a,b)

// Problem Description : This algorithm performs addition of two numbers

// Input : Two integers a and b

// Output : Addition of two integers

c a + b

return c

• Flow Chart

– Using Geometric Shapes containing descriptions of algorithm’s steps.

Proving Algorithm’s Correctness

• Prove algorithm yields required results

• For Every Legitimate Input in finite amount of time

• If one instance of valid input, the algorithm performs incorrectly !!

– Ex : Mathematical Induction

Analysis of Algorithm

• Time Efficiency

– Indicates how fast algorithm runs

• Space Efficiency

– How much Extra memory the algorithm needs to complete its execution

• Simplicity

– Generating Sequence of instructions which are easy to understand

• Generality

– Range of inputs it can accept

– Generality of problem which algorithm can solve

Coding an Algorithm

• Implementation of algorithm done by suitable programming language

– C

– C++

– Java,

– Verilog etc etc…

Problem Types to solve

• Sorting Algorithms

–Arranging elements in increasing (ascending) or decreasing (descending) order

–Numbers, Characters, Strings

– Sorting makes easier to search elements

– Ex : Dictionary, Telephone books, Class lists, Employee database etc..

Problem Types to solve

• Sorting Algorithms

–Properties

• Stable Property

–Preserves the relative order of any two equal elements. Position i , j where i < j , sorted list i’ and j’ such that i’ < j’

• In Place Property

–Does not require extra memory, except for possibly a few memory unit

Searching

• To find the desired element from the list

• Element to be Searched – Search Key

• Sequential Search

• Binary Search etc.

• Some Search algorithms work faster than others but

– Requires more memory

– Works only in sorted arrays etc..

Linear Search Algorithm

• Input: An array of n numbers and an element which is required to be searched in the given list

• Output: Number exists in the list or not. Algorithm: 1. Input the size of list i.e. n 2. Read the n elements of array A 3. Input the item/element to be searched in the given list. 4. for each element in the array i=1 to n 5. if A[i]==item 6. Search successful, return 7. if i==n+1 8. Search unsuccessful. 9. Stop

Graph Problems

• Collection of points called vertices

• Some are connected by line segments – edges

• Graph traversal algorithms, shortest path algorithms etc…

• String Processing

– String – sequence of characters

– Text Strings = Letters, numbers, special characters, bit strings (0 & 1)

Combinatorial Problems

• Computing Permutations and combinations

• Complex computing due to

– As problem size grows the combinatorial objects grow rapidly and reach to huge value

– No algorithm available which solves these problems in finite amount of time.

– Many fall under unsolvable problems

Geometric Problems

• Deal with geometric objects such as points, lines and polygons

• Applications in Computer Graphics, Robotics and topography

Time Complexity

• Amount of computer time required by an algorithm to run on completion

• Difficult to compute time complexity in terms of physically clocked time.

• Drawbacks of measuring running time in-terms of seconds, millisecond etc are

– Dependence of speed of a underlying hardware

– Number of other programs running (System load)

– Dependence of compiler used in generating machine code

How to calculate running time then?

• Time complexity given in terms of FREQUENCY COUNT

• Count denoting number of times of execution of statement.

For (i=0; i <n;i++) { // St1 : 1, St 2 : n+1 , St 3 : n times

sum = sum + a[i]; // n times

}

3n + 2 ; O(n) neglecting constants and lower order terms

How to calculate running time then?

for (i=0; i < n ; i ++) // 1 ; n+1 ; n times

{

for (j=0; j < n ; j ++) // n ; n(n+1) ; n(n)

{

c[i][j] = a[i][j] + b[i][j];

}

} 3n2+4n+ 2 = O(n2)

How to calculate running time then?

• All Algorithms run longer on larger inputs

• Algorithm’s efficiency - f(n)

• Identify the most important operations of the algorithm – BASIC Operation

• Basic operation – contributing to most of total running time

• Compute the number of times basic operation is executed (mostly in inner loop)

Ex : Sorting Algorithms – Comparison (< >)

Matrix Multiplication, Polynomial evaluation – Arithmetic Operations ( *, +)

= (assignment), ==(equality) etc..

Order of Growth of Algorithm

• Measuring the performance of an algorithm in relation with input size n

Rate of Growth of Algorithm as fn of i/p size

Efficiency comparisons