Algorithm Analysis Techniques Handout

Use the handout about mergesort, provide the recurrance formula, expand it to get a closed form equation, and then prove that it is correct.

The handout (e.g. Mergesort) was an example of the use of the general algorithm type of divide and conquer

Since we are breaking in two, for ease of calculations, suppose that n is a power of 2.

Recurrence Formula

T (n)    <    c1                 for n < 1
                  2 T(n/2)+c2n   for n > 1

Expand for Closed Form Equation

Expand up (without loss of generality, let n = 2k):

T(1) < c1
T(2) < 2T(1) + 2c2 = 2c1 + 2c2
T(4) < 2T(2) + 4c2 = 2(2c1 + 2c2) + 4c2 = 4c1 + 8c2
T(8) < 2T(4) + 8c2 = 2(4c1 + 8c2) + 8c2 = 8c1 + 242
T(16) < 2T(8) + 16c2 = 2T(4) + 16c2 = 2(8c1 + 242) + 16c2 = 16c1 + 642

Do we see a pattern? This is important. Either try to determine the pattern or try to bring to bear all the information about the problem that you know. Remember that we know something about mergesort and expect nlogn

n=2     2c1 + 2c2     2 * 1
n=4    4c1 + 8c2     4 * 2
n=8    8c1 + 24c2     8 * 3
n=16   16c1 + 64c2     16 * 4
AH HA!

T(n) < nc1 + n*logn c2

OR another technique:

T(n) = 2 T(n/2)+c2n   for n > 1
              T(n/2) = 2 T(n/4)+c2n/2 - so plug it in
T(n) = 2[ 2 T(n/4)+c2n/2 ] + c2n   = 4T(n/4) + c2n + c2n = 4T(n/4) + 2c2n
               T(n/4) = 2 T(n/8)+c2n/4 - so plug it in
T(n) = 4 [ 2 T(n/8)+c2n/4 ] + 2c2n = 8T(n/8) + c2n + 2c2n = 8T(n/8) + 3c2n
               T(n/8) = 2 T(n/16)+c2n/8 - so plug it in
T(n) = 8 [ 2 T(n/16)+c2n/8 ] + 3c2n = 16T(n/16) + c2n + 3c2n = 16T(n/16) + 4c2n
...
T(n) = 2k T(n/2k) + k*n c2
as 2k approaches n
T(n) < nc1 + n*logn c2

Verify using proof by induction

THEOREM:
T(n)    <    c1                 for n < 1
                    2 T(n/2)+c2n   for n > 1
has a closed form equation of T(n) < c1n + c2 nlogn

PROOF: By induction ( surprise!)

Base: T(n) < c1n + c2 nlogn so T(1) < c1 + 0 = c1
The recurrance relation states T(n)  < c1 for n < 1; so T(1) < c1
OK .

Induction hypothesis: Assume THM is true: T(n) < c1n + c2 nlogn for all n < n'.

Induction step:

Suppose n = n' + 1

then, T (n) < 2 T(n/2)+c2n   for n > 1 (by definition of T(n))

since n/2 < n', by induction hypothesis, T (n/2) = c1n/2 + c2 n/2log(n/2)

Ok, let's look at T(n) again, Remember definition, T(n) < 2T (n/2) + c2n so use inductive hypothesis on T(n/2)

T (n) < 2(c1n/2 + c2n/2 log n/2) + c2n

< c1n + c2n(log n - 1) + c2n

< c1n + c2nlog n

By induction, theorem is true for all n.

One could verify that it is O(nlogn)

DEFN.: f(n) = 0(g(n)) iff there exist two positive constants c and n0 such that f(n) < c* g(n) for all n > n0.

Is it was O(n)?

The answer is no and the proof is as follows (using the def. of Big O as specified)
Assume it is. We need to show:
c1n + c2nlog n < c3n for some c3 and all n > n0
divide by n
c1 + c2log n < c3
subtract c1 and divide by c2
log n < (c3 - c1)/ c2
No matter what c3 and n0 we choose, log n will eventually grow greater as n gets large.
Contradiction, hence the algorithm is NOT O(n)