-
Notifications
You must be signed in to change notification settings - Fork 485
/
04-arrays-bigoh.html
683 lines (599 loc) · 30.3 KB
/
04-arrays-bigoh.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>CS 2150: 04-arrays-bigoh slide set</title>
<meta name="description" content="A set of slides for a course on Program and Data Representation">
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, minimal-ui">
<link rel="stylesheet" href="../slides/reveal.js/dist/reset.css">
<link rel="stylesheet" href="../slides/reveal.js/dist/reveal.css">
<link rel="stylesheet" href="../slides/reveal.js/dist/theme/black.css" id="theme">
<link rel="stylesheet" href="../slides/css/pdr.css">
<!-- Code syntax highlighting -->
<link rel="stylesheet" href="../slides/reveal.js/plugin/highlight/zenburn.css">
<!-- Printing and PDF exports -->
<script>
var link = document.createElement( 'link' );
link.rel = 'stylesheet';
link.type = 'text/css';
link.href = window.location.search.match( /print-pdf/gi ) ? '../slides/reveal.js/css/print/pdf.scss' : '../slides/reveal.js/css/print/paper.scss';
document.getElementsByTagName( 'head' )[0].appendChild( link );
</script>
<!--[if lt IE 9]>
<script src="../slides/reveal.js/lib/js/html5shiv.js"></script>
<![endif]-->
<style>.reveal li { font-size:93%; line-height:120%; }</style>
</head>
<body>
<div class="reveal">
<!-- Any section element inside of this container is displayed as a slide -->
<div class="slides">
<section data-markdown><script type="text/template">
# CS 2150
### Program and Data Representation
<p class='titlep'> </p>
<div class="titlesmall"><p>
<a href="http://www.cs.virginia.edu/~mrf8t">Mark Floryan</a> ([email protected])<br>
<a href="http://www.cs.virginia.edu/~asb">Aaron Bloomfield</a> ([email protected])<br>
<a href="http://github.com/uva-cs/pdr">@github</a> | <a href="index.html">↑</a> | <a href="./04-arrays-bigoh.html?print-pdf"><img class="print" width="20" src="../slides/images/print-icon.png" style="top:0px;vertical-align:middle"></a>
</p></div>
<p class='titlep'> </p>
## Arrays & Big-Oh
</script></section>
<section>
<h2>CS 2150 Roadmap</h2>
<table class="wide">
<tr><td colspan="3"><p class="center">Data Representation</p></td><td></td><td colspan="3"><p class="center">Program Representation</p></td></tr>
<tr>
<td class="top"><small> <br> <br>string<br> <br> <br> <br>int x[3]<br> <br> <br> <br>char x<br> <br> <br> <br>0x9cd0f0ad<br> <br> <br> <br>01101011</small></td>
<!-- image adapted from http://openclipart.org/detail/3677/arrow-left-right-by-torfnase -->
<td><img class="noborder" src="images/red-double-arrow.png" height="500" alt="vertical red double arrow"></td>
<td class="top"> <br>Objects<br> <br>Arrays<br> <br>Primitive types<br> <br>Addresses<br> <br>bits</td>
<td> </td>
<td class="top"><small> <br> <br>Java code<br> <br> <br>C++ code<br> <br> <br>C code<br> <br> <br>x86 code<br> <br> <br>IBCM<br> <br> <br>hexadecimal</small></td>
<!-- image adapted from http://openclipart.org/detail/3677/arrow-left-right-by-torfnase -->
<td><img class="noborder" src="images/green-double-arrow.png" height="500" alt="vertical green double arrow"></td>
<td class="top"> <br>High-level language<br> <br>Low-level language<br> <br>Assembly language<br> <br>Machine code</td>
</tr>
</table>
</section>
<section data-markdown><script type="text/template">
# Contents
[Arrays in C/C++](#/arrays)
[Orders of Growth](#/growth)
[Determining Running Times](#/runningtimes)
</script></section>
<section>
<section id="arrays" data-markdown class="center"><script type="text/template">
# Arrays in C/C++
</script></section>
<section data-markdown><script type="text/template">
## Primitive Arrays in C++
- Two ways to declare arrays:
- Specify name, type, and size
```
int someInts[3];
```
- Specify name, type, and initialization list
```
int someInts[] = {4, 37, 18};
```
- Size not needed in this case
- Either way, this creates 3 ints:
```
someInts[0], someInts[1], someInts[2]
```
</script></section>
<section data-markdown><script type="text/template">
## Things to remember about arrays in C++
1. No index checking!
- Compiler and runtime system will not catch accesses out of array index bounds
- Out of bounds access will cause unpredictable behavior
2. When passed as an argument to a function
- Function does not know the size of the array
- Need to pass another argument containing the size
3. Cannot be copied with =
- Must be copied element by element
4. Cannot be compared with ==
- It won't do what you want it to do
</script></section>
<section>
<h2>C++ Implementation of Arrays</h2>
<ul>
<li>Array names are pointers to beginning of array
<pre><code>int someInts[3];</code></pre><ul>
<li>The size is only used by compiler to set aside memory for the array; it is not stored anywhere</li>
<li><code>someInts</code> is set to <code>&someInts[0]</code><ul>
<li>Also called the base address</li></ul></li>
<li>Local constant that cannot be changed</li></ul></li>
<li>Note that the previous declaration is almost the same as:
<pre><code>int* someInts = new int[3];</code></pre><ul>
<li>But since this second declaration is declared with <code>new</code>, the programmer must delete it: <code>delete []</code></li></ul></li>
</ul>
</section>
<section>
<h2>int someInts[3] = {2, 4, 6};</h2>
<table class="transparent">
<tr><th>Array part</th><th>Address</th><th>Memory</th></tr>
<tr><td><code>&someInts[0]</code></td><td>1000</td><td class="border">2</td></tr>
<tr><td><code>&someInts[1]</code></td><td>1004</td><td class="border">4</td></tr>
<tr><td><code>&someInts[2]</code></td><td>1008</td><td class="border">6</td></tr>
<tr><td></td><td></td><td class="border">...</td></tr>
<tr><td><code>someInts</code></td><td>???</td><td class="border">1000</td></tr>
<tr><td></td><td></td><td class="border" style="border-bottom:medium solid;">...</td></tr>
</table>
<p>This assumes that ints are 4 bytes in size</p>
</section>
<section data-markdown><script type="text/template">
## Where is someInts[2] located in memory?
- Remember
- `someInts` is a pointer to address of `someInts[0]` (i.e., `&someInts[0]`)
- Array elements are laid out sequentially in contiguous memory addresses
```
&someInts[i] = {addr of someInts} + (sizeof(int) * i)
```
</script></section>
<section data-markdown><script type="text/template">
## Operations on Arrays
- `someInts = someOtherArray;` is illegal
- Array base addresses are considered constant (cannot be reassigned)
- This is one example of when pointers and arrays are NOT the same
- Is `someInts == someOtherArray` allowed?
- Yes, but will only evaluate to true if `someInts` and `someOtherArray` point at the same memory location
</script></section>
<section data-markdown><script type="text/template">
## Function Calls and Arrays
(no external source code)
```
void someFunc(int arrayOfInts[]) { /* ... */ }
// ...
int main() {
int someInts[3];
someFunc(someInts); // no brackets
return 0;
}
```
- `someInts` is a memory address
- Size of `someInts[]` unknown
- Is this pass by reference? Pass by value?
</script></section>
<section data-markdown><script type="text/template">
## Multidimensional Arrays
(no external source code)
```
int m[2][3]; // 2 rows, 3 columns
int n[2][3] = // dimensions optional here
{ {1,2,3}, {4,5,6} };
// initialize row 0: 1,2,3
// initialize row 1: 4,5,6
```
</script></section>
<section data-markdown><script type="text/template">
## C++ Implementation
Multi-dimensional arrays in C++ are stored in ***row major*** order
![2-d array diagram](images/04-arrays-bigoh/2d-array-layout.png)
</script></section>
<section data-markdown><script type="text/template">
## Command-line parameters
- You can define the `main()` function as either:
- `int main()`
- `int main (int argc, char **argv)`
- Or: `int main (int argc, char* argv[])`
- The second version gives us two parameters:
- The number of command line parameters
- Note that the program executable is one of them (`argv[0]`)
- An array of C-style strings (`char*`) containing those parameters
</script></section>
<section data-markdown id="cmdlineparams"><script type="text/template">
## Command-line parameters
Source code: [cmdlineparams.cpp](code/04-arrays-bigoh/cmdlineparams.cpp.html) ([src](code/04-arrays-bigoh/cmdlineparams.cpp))
```
int main (int argc, char* argv[]) {
// the command-line parameters are stored as C-style
// strings in the argv[] array. argc contains 1 more
// than the number of command line parms specified.
// The 0th command line parameter is the program
// name. We can print a C-style string via cout:
cout << "This program is called '" << argv[0]
<< "'" << endl;
cout << "The following are the command "
<< "line parameters you specified: " << endl;
// this for loop starts at 1 to avoid printing the
// name of the program
for ( int i = 1; i < argc; i++ ) {
// we can convert the C-style strings into
// C++-style strings, and then print them:
string s(argv[i]);
cout << "\t" << s << endl;
}
return 0;
}
```
</script></section>
</section>
<section>
<section id="growth" data-markdown class="center"><script type="text/template">
# Orders of Growth
</script></section>
<section data-markdown><script type="text/template">
## Classifying functions by their<br>Asymptotic Growth Rates
- Asymptotic growth rate or asymptotic order
- Comparing and classifying functions that ignores constant factors and small inputs.
- The sets are <span class='red'>big-omega</span>, big-theta, and <span class='skyblue'>big-oh</span>:
- <span class='red'>Ω</span>(*g*): functions that grow ***at least as fast*** as g
- Θ(*g*): functions that grow ***at the same rate*** as g
- <span class='skyblue'>O</span>(*g*): functions that grow ***no faster than*** g
![venn diagram](images/04-arrays-bigoh/venn-diagram.png)
</script></section>
<section data-markdown><script type="text/template">
## Why do we care?
- Some data structures are faster than others
- Each data structure has some operations that are fast, and some operations that are slow
- We need a way to compare them
- This allows us to:
- Better choose the data structures that we will use
- Better design additional data structures
</script></section>
<section data-markdown><script type="text/template">
## Input sizes
- Your algorithm does not matter if you have 10 elements
- A [bogosort](http://en.wikipedia.org/wiki/Bogosort) will work just fine
- Consider big input sizes:
- UVa's e-mail probably has about 100,000 e-mail addresses
- [OpenStreetMap](http://www.openstreetmap.org), for driving routes, has over 4.7 billion nodes and 6.4 million GPS points ([ref](http://www.openstreetmap.org/stats/data_stats.html)) (as of Sep 2018)
- A 'node' is a point on the Earth's surface, and not necessarily an intersection
- Google's index has almost 47 billion items ([ref](http://www.worldwidewebsize.com/)) (as of Sep 2018)
</script></section>
<section data-markdown><script type="text/template">
## Even for smaller input sizes...
- This is from the first lecture set
- All times are in ms (1/1000<sup>th</sup> of a second)
- Source code: [DataStructureComparison.java](../uva/code/introduction//DataStructureComparison.java.html) ([src](../uva/code/introduction//DataStructureComparison.java))
| Data Structure | Total time | Insert time | Search time | Delete time |
|----------------|------------|-------------|-------------|-------------|
| Vector | 17,311 | 30 | 12,620 | 4,661 |
| ArrayList | 17,281 | 28 | 12,609 | 4,644 |
| LinkedList | 24,255 | 54 | 17,934 | 6,267 |
| HashSet | 122 | 103 | 9 | 10 |
| TreeSet | 119 | 93 | 14 | 12 |
| Vector, sorted | 294 | 36 | 19 | 239 |
</script></section>
<section data-markdown><script type="text/template">
## Assumptions
- We have measured the running time of our program with different input sizes, and that result is encapsulated in some function *f*(*n*)
- *n* is the input size, and is always a positive integer
- We may not know *f*(*n*) exactly, but we still will want to classify it
- We have some other function *g* that we want to compare our program to
- So we will compare *f*(*n*) to *g*(*n*), such as:
- *f*(*n*) ∈ O(*g*(*n*))
- We could shorten this: *f* ∈ O(*g*)
- *f*(*n*) ∉ Ω(*g*(*n*))
- We could shorten this: *f* ∉ Ω(*g*)
</script></section>
<section data-markdown><script type="text/template">
## Worst-case Scenario
- We always analyze the ***worst case*** run-time
- It makes no sense to analyze the best case, as that is rarely likely to happen
- And the *average* case (if you could even define what that is) may not be representative and is not used in these analyses either
- A more formal definition of *worst case*, should you be interested, can be found [here](http://en.wikipedia.org/wiki/Worst_case)
</script></section>
<section>
<h2>The Sets O(<i>g</i>), Θ(<i>g</i>), Ω(<i>g</i>)</h2>
<ul>
<li>Let <i>f</i> and <i>g</i> be a functions from the non-negative integers into the positive real numbers</li>
<li>For some real constant <i>c</i> > 0 and some non-negative integer constant <i>n</i><sub>0</sub><ul>
<li>O(<i>g</i>) is the set of functions <i>f</i>, such that:<ul>
<li><i>f</i>(<i>n</i>) <span class='red'>≤</span> c * <i>g</i>(<i>n</i>) for all <i>n</i> ≥ <i>n</i><sub>0</sub></li></ul></li>
<li>Ω(<i>g</i>) is the set of functions <i>f</i>, such that:<ul>
<li><i>f</i>(<i>n</i>) <span class='red'>≥</span> c * <i>g</i>(<i>n</i>) for all <i>n</i> ≥ <i>n</i><sub>0</sub></li></ul></li>
<li>Θ(<i>g</i>) = O(<i>g</i>) ∩ Ω(<i>g</i>)<ul>
<li>Θ(<i>g</i>) is the asymptotic order of g or the <i>order</i> of g</li>
<li><i>f</i> ∈ Θ(<i>g</i>) read as "<i>f</i> is asymptotic order <i>g</i>" or "<i>f</i> is order <i>g</i>"</li></ul></li>
</ul></li>
</ul>
<!-- <script type="text/javascript">insertCanvas();</script> -->
</section>
<section data-markdown><script type="text/template">
## Asymptotic Bounds
- For the sets big-oh O(*g*), big-theta Θ(*g*), and big-omega Ω(*g*), remember these meanings:
- O(*g*): functions that grow ***no faster than*** *g*, or ***asymptotic upper bound***
- Ω(*g*): functions that grow ***at least as fast as*** *g*, or ***asymptotic lower bound***
- Θ(*g*): functions that grow ***at the same rate as*** *g*, or ***asymptotic tight bound***
</script></section>
<section>
<h2>Big-Oh Examples</h2>
<p><i>f</i>(<i>n</i>) ∈ O(<i>g</i>(<i>n</i>)) means that there are positive constants <i>c</i> and <i>n</i><sub>0</sub> such that <i>f</i>(<i>n</i>) ≤ <i>c</i>*<i>g</i>(<i>n</i>) for all values <i>n</i> ≥ <i>n</i><sub>0</sub></p>
<ul>
<li>Is <i>n</i> ∈ O(<i>n</i><sup>2</sup>)?
<ul><li class="fragment">Yes, <i>c</i> = 1, <i>n</i><sub>0</sub> = 2 works fine</li></ul></li>
<li>Is 10<i>n</i> ∈ O(<i>n</i>)?
<ul><li class="fragment">Yes, <i>c</i> = 11, <i>n</i><sub>0</sub> = 2 works fine</li></ul></li>
<li>Is <i>n</i><sup>2</sup> ∈ O(<i>n</i>)?
<ul><li class="fragment">No; no matter what values for <i>c</i> and <i>n</i><sub>0</sub> we pick, <i>n</i><sup>2</sup> > <i>c</i>*<i>n</i> for big enough <i>n</i></li></ul></li>
</ul>
<script type="text/javascript">insertCanvas();</script>
</section>
<section>
<h2>Given <i>f</i> ∈ O(<i>h</i>) and <i>g</i> ∉ O(<i>h</i>),<br>which of these are true?</h2>
<ol>
<li>For <b><i>all</i></b> positive integers <i>m</i>, <i>f</i>(<i>m</i>) < <i>g</i>(<i>m</i>).</li>
<li>For <b><i>some</i></b> positive integer <i>m</i>, <i>f</i>(<i>m</i>) < <i>g</i>(<i>m</i>).</li>
<li>For <b><i>some</i></b> positive integer <i>m</i><sub>0</sub>, and <b><i>all positive</i></b> integers <i>m</i> > <i>m</i><sub>0</sub>, <i>f</i>(<i>m</i>) < <i>g</i>(<i>m</i>).</li>
<li>1 and 2</li>
<li>2 and 3</li>
<li>1 and 3</li>
</ol>
<script type="text/javascript">insertCanvas();</script>
</section>
<section data-markdown><script type="text/template">
## 1 is false: Prove by Counter-Example
- Recall that *f* ∈ O(*h*) and *g* ∉ O(*h*)
- We are disproving: for ***all*** positive integers m, *f*(*m*) < *g*(*m*):
- To disprove by counter-example, you have to find just ***one*** example that does not fulfill the stated claim
- Pick *f*(*n*) = 5\**n*, *g*(*n*) = *n*<sup>3</sup>, *h*(*n*) = *n*<sup>2</sup>
- *f* ∈ O(*h*), as 5\**n* ≤ *c* \* *n*<sup>2</sup> ∀ *n* > *n*<sub>0</sub> when *c* = 6 and *n*<sub>0</sub> = 2
- *g* ∉ O(*h*), as *n*<sup>3</sup> ≤ *c* \* *n*<sup>2</sup> ∀ *n* > *n*<sub>0</sub> cannot be satisifed, for large *n*, with any values of *c* and *n*<sub>0</sub>
- For *m*=2, *f*(*m*) = 10 > 8 = *g*(*m*)
- Therefore, this is false
- *f*(*n*) ∈ O(*g*(*n*)) means there are positive constants *c* and *n*<sub>0</sub> such that *f*(*n*) ≤ *c*\**g*(*n*) for all values *n* ≥ *n*<sub>0</sub> (but not ***all*** values of *n*, which is waht this was trying to show)
</script></section>
<section data-markdown><script type="text/template">
## 2 is true: Intuition
- Recall that *f* ∈ O(*h*) and *g* ∉ O(*h*)
- Claim: for ***some*** positive integer *m*, *f*(*m*) < *g*(*m*)
- We know:
- *g* must grow faster than *h*, otherwise *g* would be in O(*h*)
- *f* must grow no faster than *h*, since *f* ∈ O(*h*)
- Rephrased: *f* grows as slow as, or slower than, *h*
- So, if *g* grows faster than *h*, but *f* grows as slow or slower than *h*, eventually, then:
- *g* will grow faster than *f*
- Or *g*(*n*) > *f*(*n*) for some large *n*
- Thus, for some *m*, *f*(*m*) < *g*(*m*)
</script></section>
<section data-markdown><script type="text/template">
## 3 is true: Deduction
- Recall that *f* ∈ O(*h*) and *g* ∉ O(*h*)
- Claim: for ***some*** positive integer *m*<sub>0</sub>, and ***all positive*** integers *m* > *m*<sub>0</sub>, *f*(*m*) < *g*(*m*)
- We know, from the last slide, that *g* will grow faster than *f*
- This is essentially the definition of big-oh, which is what the claim is basically stating: that *f* ∈ O(*g*)
</script></section>
<section data-markdown><script type="text/template">
## Given *f* ∈ O(*h*) and *g* ∉ O(*h*),<br>which of these are true?
1. For ***all*** positive integers *m*, *f*(*m*) < *g*(*m*).
2. For ***some*** positive integer *m*, *f*(*m*) < *g*(*m*).
3. For ***some*** positive integer *m*<sub>0</sub>, and ***all positive*** integers *m* > *m*<sub>0</sub>, *f*(*m*) < *g*(*m*).
4. 1 and 2
5. 2 and 3
6. 1 and 3
</script></section>
<section data-markdown><script type="text/template">
## Lower Bound: Ω (Omega)
- *f*(*n*) ∈ Ω (*g*(*n*)) means:
- There are positive constants *c* and *n*<sub>0</sub> such that *f*(*n*) <span class='red'>≥</span> *c*\*g*(*n*) for all *n* ≥ *n*<sub>0</sub>
- The difference from big-oh is the ≥ in big-omega versus ≤ in big-oh
- This is a *lower* bound
</script></section>
<section>
<img class="stretch" alt="big-omega" src="images/04-arrays-bigoh/omega-diagram-1.png">
</section>
<section data-transition="fade" data-transition-speed="fast">
<img class="stretch" alt="big-omega" src="images/04-arrays-bigoh/omega-diagram-2.png">
</section>
<section data-markdown><script type="text/template">
## Theta ("Order of")
- Intuition: the set Θ(*f*) is the set of functions that grow ***as fast as*** *f*
- Definition: *f*(*n*) ∈ Θ(*g*(*n*)) if and only if both:
1. *f*(*n*) ∈ O (*g*(*n*))
2. *f*(*n*) ∈ Ω (*g*(*n*))
- Note that we do not have to pick the same *c* and *n*<sub>0</sub> values for cases 1 and 2
- When we say, "*f* is order *g*" that means *f*(*n*) ∈ Θ (*g*(*n*))
</script></section>
<section>
<img class="stretch" alt="big-theta" src="images/04-arrays-bigoh/theta-diagram.png">
</section>
<section data-markdown><script type="text/template">
## Little-oh
- Let *f*(*n*) ∈ <span class='red'>o</span>(*g*(*n*))
- Any function that is o(*g*) is also O(*g*)
- So both act as an upper bound
- But a O(*g*) bound can also be Ω(*g*) and thus Θ(*g*)
- Meaning the big-oh bound can be tight
- Little-oh means the bound can not be tight
- Thus, it's O(*g*) but not Θ(*g*)
- Or, that *f*(*n*) will ***always*** be ***strictly*** less than *g*(*n*):
- *f*(*n*) <span class='red'><</span> *c* \* *g*(*n*) for all *n* ≥ *n*<sub>0</sub>, for some constants *c* and *n*<sub>0</sub>
- Rarely used in computer science...
</script></section>
<section data-markdown><script type="text/template">
## Little-omega
- Analogous to little-oh
- It's a lower bound that cannot be a tight lower bound
- If something is ω(*g*), then it's Ω(*g*) but not Θ(*g*)
- Rarely used in computer science...
- What does this mean for little-theta?
</script></section>
<section>
<h2>Another Way to Define Order Classes</h2>
<ul>
<li>Comparing <i>f</i>(<i>n</i>) and <i>g</i>(<i>n</i>) as <i>n</i> approaches infinity...</li>
<li>If \( \lim{n \to \infty}\frac{f(n)}{g(n)} \) is:
<ul>
<li>< ∞, including the case in which the limit is 0, then <i>f</i> ∈ O(<i>g</i>)</li>
<li>> 0, including the case in which the limit is ∞, then <i>f</i> ∈ Ω(<i>g</i>)</li>
<li>= <i>c</i> and 0 < <i>c</i> < ∞ then <i>f</i> ∈ Θ(<i>g</i>)</li>
<li>= 0 then <i>f</i> ∈ o(<i>g</i>)<ul><li>read as "little-oh of <i>g</i>"</li></ul></li>
<li>= ∞ then <i>f</i> ∈ ω(<i>g</i>)<ul><li>read as "little-omega of <i>g</i>"</li></ul></li>
</ul></li></ul>
</section>
<section data-markdown><script type="text/template">
## A note about notation
- Stating *f*(*n*) = O(*g*(*n*)) implies an equality
- This is a bit of an abuse of the notation, as O(*g*(*n*)) is a ***set***
- It should be *f*(*n*) ∈ O(*g*(*n*))
</script></section>
<section data-markdown><script type="text/template">
## Some Properties of O(*g*), Θ(*g*), Ω(*g*)
- Reversing big-oh: *f* ∈ O(*g*) ⇔ *g* ∈ Ω(*f*)
- Adding big-oh: O(*f* + *g*) = O(max(*f*, *g*))
- Similar equations hold for Ω and Θ
- Transitive: If *f* ∈ O(*g*) and *g* ∈ O(*h*), then *f* ∈ O(*h*)
- Big-Oh is transitive, as are all the others (big-theta, big-omega, little-oh, and little-omega)
- Reflexive: *f* ∈ Θ(*f*)
- As are big-oh and big-omega
- Symmetric: If *f* ∈ Θ(*g*), then *g* ∈ Θ(*f*)
- Big-Oh is not symmetric! (neither are Ω, o, or ω)
- Θ defines an ***equivalence relation*** on the functions
- Each set Θ(*f*) is an equivalence class (complexity class)
</script></section>
<section data-markdown><script type="text/template">
## Classification of functions
- Θ(1) denotes the set of functions bounded by a constant (for large *n*)
- *f* ∈ Θ(log *n*) means *f* is logarithmic
- *f* ∈ Θ(*n*) means *f* is linear
- *f* ∈ Θ(*n* log *n*) means *f* is log-linear
- *f* ∈ Θ(*n*<sup>2</sup>) means *f* is quadratic
- *f* ∈ Θ(*n*<sup>3</sup>) means *f* is cubic
- *f* ∈ Θ(2<sup>*n*</sup>) means *f* is exponential
- *n*<sup>*k*</sup> ∈ <span class='red'>o</span>(*c*<sup>*n*</sup>) for any *k* > 0 and any *c* > 1
- Powers of *n* grow slower than any exponential function *c*<sup>*n*</sup>
- As long as *n* ≥ 1 and *c* ≥ 1, which is always the case with input sizes
</script></section>
<section data-markdown><script type="text/template">
## A note about logs
- The difference between log<sub>10</sub>*n* and log<sub>2</sub>*n* is always a constant
- Specifically, about 3.322
- Since we don't care about constants in these analyses, we'll ignore the log base
- Most things in computer science are log base 2 anyway...
</script></section>
<section data-markdown><script type="text/template">
## Typical Growth Rates
| Function | Name |
|----------|------|
| *c* | constant |
| log *n* | logarithmic |
| *n* | linear |
| *n* log *n* | log-linear |
| *n*<sup>2</sup> | quadratic |
| *n*<sup>3</sup> | cubic |
| 2<sup>*n*</sup> | exponential |
</script></section>
<section data-markdown><script type="text/template">
## Does Order Class Matter?
- No, not for small inputs
- Yes, for many real problems
</script></section>
<section data-markdown><script type="text/template">
## Practical Complexity
![graph](images/04-arrays-bigoh/graph-1.png)
</script></section>
<section data-markdown data-transition="fade" data-transition-speed="fast">><script type="text/template">
## Practical Complexity
![graph](images/04-arrays-bigoh/graph-2.png)
</script></section>
<section data-markdown data-transition="fade" data-transition-speed="fast">><script type="text/template">
## Practical Complexity
![graph](images/04-arrays-bigoh/graph-3.png)
</script></section>
<section data-markdown data-transition="fade" data-transition-speed="fast">><script type="text/template">
## Practical Complexity
![graph](images/04-arrays-bigoh/graph-4.png)
</script></section>
<section data-markdown data-transition="fade" data-transition-speed="fast">><script type="text/template">
## Practical Complexity
![graph](images/04-arrays-bigoh/graph-5.png)
</script></section>
<section data-markdown><script type="text/template">
## General Rules for Running<br>Time Calculations (Big-Oh)
- For loops
- The number of times the for loop runs times the total running time of the statements inside the for loop
- Nested loops
- Analyze from inside to out
- Runtime of the statement \* product of the sizes of the loops
- Consecutive statements
- Take the max runtime of the individual statements
- if/else
- Time for the test plus the longer of the runtimes
</script></section>
</section>
<section>
<section id="runningtimes" data-markdown class="center"><script type="text/template">
# Determining Running Times
</script></section>
<section data-markdown><script type="text/template">
## How to tell the running time...
- You need to imagine how long the algorithm would run given varying input sizes
- We will assume our input size is *n*
- You should ask yourself...
- How long will this algorithm take when *n* = 1?
- When *n* = 100?
- When *n* = 1,000,000,000?
- Recall that we always analyze the worst case!
</script></section>
<section data-markdown><script type="text/template">
## Running time: constant
- Constant (Θ(1)) time means that the running time does NOT depend on the size of the input
- Examples:
- Getting a vector's size
- Linked list insert or delete (at the known end)
- Finding the k<sup>th</sup> element in an array or vector
</script></section>
<section data-markdown><script type="text/template">
## Running time: logarithmic
- Logarithmic (Θ(log *n*)) running time is when the search space is cut in half in each iteration
- Or when you are iterating up or down a data structure, such as a tree, that has Θ(log *n*) height
- Examples:
- Binary search in a sorted array or vector
- Search (or insert or delete) with a balanced tree
- Balanced tree means an AVL tree or Red-black tree
</script></section>
<section data-markdown><script type="text/template">
## Running time: linear
- Linear (Θ(*n*)) running time means that we have to process each element, and there is one step (or a constant number of steps) for each one
- Examples:
- Printing (or otherwise iterating through) a list
- Finding an element in an unsorted array or vector
- Finding an element in a sorted or unsorted linked list
- Doubling the size of a vector's underlying array
</script></section>
<section data-markdown><script type="text/template">
## Running time: Log-linear
- Log-linear (Θ(*n* log *n*)) typically occurs when you are going to take a linear number of steps, but each one takes log *n* time
- Or log *n* steps, each one of which takes *n* time
- Examples:
- Fast sorts: merge sort and heap sort
- Quicksort, on a good day (but this is not guaranteed!)
- Inserting *n* elements into a data structure where each insert takes log *n* time
</script></section>
<section data-markdown><script type="text/template">
## Running time: quadratic
- A quadratic (Θ(*n*<sup>2</sup>)) running time occurs when, for each input, you have to search through the entire input again
- (among other ways it can occur)
- Examples:
- Slow sorts: insertion sort, bubble sort, selection sort
- Quicksort, on a bad day
- Some graph algorithms we'll see later this semester
- Doubly nested for loops (where each loop goes 1 to *n*)
</script></section>
<section data-markdown><script type="text/template">
## Running time: exponential
- An exponential (Θ(2<sup>*n*</sup>)) running time typically means you are going to try every possible solution, and there are 2<sup>*n*</sup> of them
- Examples (we have not seen any yet)
- Trying to crack a binary combination lock by trying every single possibility
- Traveling salesperson problem (we'll see that later this semester)
- Satisfiability of a Boolean expression
</script></section>
</section>
</div>
</div>
<script src='../slides/reveal.js/dist/reveal.js'></script><script src='../slides/reveal.js/plugin/zoom/zoom.js'></script><script src='../slides/reveal.js/plugin/notes/notes.js'></script><script src='../slides/reveal.js/plugin/search/search.js'></script><script src='../slides/reveal.js/plugin/markdown/markdown.js'></script><script src='../slides/reveal.js/plugin/highlight/highlight.js'></script><script src='../slides/reveal.js/plugin/math/math.js'></script>
<script src="js/settings.js"></script>
</body>
</html>