Algorithms
Master fundamental algorithms with step-by-step explanations and practical examples
Level 1: Foundation - Algorithms
Linear Search
Linear search examines each element in a collection one by one until the target is found or all elements are checked.
š Real-Life Analogy:
Think of looking for a specific card in a deck. You go through each card one by one from top to bottom until you find the card you're looking for.
TypeScript
// Linear search implementation function linearSearch(arr: number[], target: number): number { for (let i = 0; i < arr.length; i++) { if (arr[i] === target) { return i; // Return index if found } } return -1; // Return -1 if not found } // Example usage const numbers = [64, 34, 25, 12, 22, 11, 90]; console.log(linearSearch(numbers, 22)); // Output: 4 // Linear search for strings function linearSearchString(arr: string[], target: string): number { for (let i = 0; i < arr.length; i++) { if (arr[i] === target) { return i; } } return -1; } const names = ["Alice", "Bob", "Charlie", "David"]; console.log(linearSearchString(names, "Charlie")); // Output: 2
Python
# Linear search implementation def linear_search(arr: list[int], target: int) -> int: for i in range(len(arr)): if arr[i] == target: return i # Return index if found return -1 # Return -1 if not found # Example usage numbers = [64, 34, 25, 12, 22, 11, 90] print(linear_search(numbers, 22)) # Output: 4 # Linear search for strings def linear_search_string(arr: list[str], target: str) -> int: for i, item in enumerate(arr): if item == target: return i return -1 names = ["Alice", "Bob", "Charlie", "David"] print(linear_search_string(names, "Charlie")) # Output: 2
OCaml
(* Linear search implementation *) let rec linear_search arr target index = if index >= Array.length arr then -1 else if arr.(index) = target then index else linear_search arr target (index + 1) (* Helper function to start search from index 0 *) let search arr target = linear_search arr target 0 (* Example usage *) let numbers = [|64; 34; 25; 12; 22; 11; 90|];; let result = search numbers 22;; (* 4 *) (* Linear search for lists *) let rec linear_search_list lst target index = match lst with | [] -> -1 | head :: tail -> if head = target then index else linear_search_list tail target (index + 1) let names = ["Alice"; "Bob"; "Charlie"; "David"];; let name_result = linear_search_list names "Charlie" 0;; (* 2 *)
Key Takeaways:
- Time Complexity: O(n) - worst case checks every element
- Space Complexity: O(1) - uses constant extra space
- Works on both sorted and unsorted arrays
- Simple to implement and understand
Basic Recursion
Recursion is when a function calls itself to solve a smaller version of the same problem. Every recursive function needs a base case to stop the recursion.
šŖ Real-Life Analogy:
Think of Russian nesting dolls. To count all the dolls, you open the current doll, count it as 1, then count all the dolls inside. You keep doing this until you reach the smallest doll (base case).
TypeScript
// Factorial: n! = n Ć (n-1) Ć (n-2) Ć ... Ć 1 function factorial(n: number): number { if (n <= 1) return 1; // Base case return n * factorial(n - 1); // Recursive case } // Fibonacci sequence function fibonacci(n: number): number { if (n <= 1) return n; // Base cases return fibonacci(n - 1) + fibonacci(n - 2); } // Sum of array elements using recursion function sumArray(arr: number[], index: number = 0): number { if (index >= arr.length) return 0; // Base case return arr[index] + sumArray(arr, index + 1); // Recursive case } // Examples console.log(factorial(5)); // 120 console.log(fibonacci(6)); // 8 console.log(sumArray([1, 2, 3, 4, 5])); // 15
Python
# Factorial: n! = n Ć (n-1) Ć (n-2) Ć ... Ć 1 def factorial(n: int) -> int: if n <= 1: return 1 # Base case return n * factorial(n - 1) # Recursive case # Fibonacci sequence def fibonacci(n: int) -> int: if n <= 1: return n # Base cases return fibonacci(n - 1) + fibonacci(n - 2) # Sum of array elements using recursion def sum_array(arr: list[int], index: int = 0) -> int: if index >= len(arr): return 0 # Base case return arr[index] + sum_array(arr, index + 1) # Recursive case # Examples print(factorial(5)) # 120 print(fibonacci(6)) # 8 print(sum_array([1, 2, 3, 4, 5])) # 15
OCaml
(* Factorial: n! = n Ć (n-1) Ć (n-2) Ć ... Ć 1 *) let rec factorial n = if n <= 1 then 1 (* Base case *) else n * factorial (n - 1) (* Recursive case *) (* Fibonacci sequence *) let rec fibonacci n = if n <= 1 then n (* Base cases *) else fibonacci (n - 1) + fibonacci (n - 2) (* Sum of list elements using recursion *) let rec sum_list = function | [] -> 0 (* Base case *) | head :: tail -> head + sum_list tail (* Recursive case *) (* Examples *) let fact5 = factorial 5;; (* 120 *) let fib6 = fibonacci 6;; (* 8 *) let sum_result = sum_list [1; 2; 3; 4; 5];; (* 15 *)
šÆ Recursion Visualization: Factorial(5)
Call Stack Visualization: š factorial(5) Memory Stack: āā 5 * factorial(4) āāāāāāāāāāāāāāāāāāā ā āā 4 * factorial(3) ā factorial(5) ā ā Current call ā ā āā 3 * factorial(2)ā factorial(4) ā ā ā ā āā 2 * factorial(1)ā factorial(3) ā ā ā ā ā āā returns 1ā factorial(2) ā ā ā ā āā returns 2 ā factorial(1) ā ā ā āā returns 6 āāāāāāāāāāāāāāāāāāā ā āā returns 24 āā returns 120 Step-by-Step Execution: 1ļøā£ factorial(5) ā 5 * factorial(4) 2ļøā£ factorial(4) ā 4 * factorial(3) 3ļøā£ factorial(3) ā 3 * factorial(2) 4ļøā£ factorial(2) ā 2 * factorial(1) 5ļøā£ factorial(1) ā returns 1 (BASE CASE) 6ļøā£ factorial(2) ā 2 * 1 = 2 7ļøā£ factorial(3) ā 3 * 2 = 6 8ļøā£ factorial(4) ā 4 * 6 = 24 9ļøā£ factorial(5) ā 5 * 24 = 120 ā
š³ Fibonacci Recursion Tree: fib(5)
fib(5) / \ fib(4) fib(3) / \ / \ fib(3) fib(2) fib(2) fib(1) / \ / \ / \ ā fib(2) fib(1) fib(1) fib(0) fib(1) fib(0) 1 / \ ā ā ā ā ā fib(1) fib(0) 1 1 0 1 0 ā ā 1 0 Execution Order & Results: š¢ fib(0) = 0 (base case) š¢ fib(1) = 1 (base case) š¢ fib(2) = fib(1) + fib(0) = 1 + 0 = 1 š¢ fib(3) = fib(2) + fib(1) = 1 + 1 = 2 š¢ fib(4) = fib(3) + fib(2) = 2 + 1 = 3 š¢ fib(5) = fib(4) + fib(3) = 3 + 2 = 5 ā ļø Notice: Some values are calculated multiple times! This is why naive fibonacci is inefficient.
š Debugging Recursion Techniques
1. Add Debug Prints
TypeScript - Debug Version
function factorialDebug(n: number, depth: string = ""): number { console.log(`${depth}ā factorial(${n}) called`); if (n <= 1) { console.log(`${depth}ā BASE CASE: returning 1`); return 1; } const result = n * factorialDebug(n - 1, depth + " "); console.log(`${depth}ā factorial(${n}) returning ${result}`); return result; } // Usage: factorialDebug(4) // Output: // ā factorial(4) called // ā factorial(3) called // ā factorial(2) called // ā factorial(1) called // ā BASE CASE: returning 1 // ā factorial(2) returning 2 // ā factorial(3) returning 6 // ā factorial(4) returning 24
2. Trace Execution Path
- Draw the call stack: Visualize function calls as they're pushed/popped
- Track variables: Note how parameters change with each call
- Identify the base case: Ensure it's reached and returns correctly
- Verify progress: Each call should move closer to the base case
3. Common Recursion Debugging Questions
ā Base case missing? ā Stack overflow
ā Base case wrong? ā Incorrect results
ā No progress toward base? ā Infinite recursion
ā Wrong recursive call? ā Logic errors
ā Stack too deep? ā Consider iterative solution
Key Principles of Recursion:
- Base Case: A condition that stops the recursion
- Recursive Case: The function calls itself with a smaller problem
- Progress: Each recursive call should move closer to the base case
- Stack Usage: Each call uses memory on the call stack
Two-Pointer Technique
The two-pointer technique uses two pointers that move through the data structure to solve problems efficiently. This technique is particularly useful for array and string problems.
š„ Real-Life Analogy:
Imagine two people walking toward each other from opposite ends of a hallway to meet in the middle. They can check things along the way and stop when they meet or find what they're looking for.
TypeScript
// Check if string is palindrome function isPalindrome(s: string): boolean { let left = 0; let right = s.length - 1; while (left < right) { if (s[left] !== s[right]) { return false; } left++; right--; } return true; } // Find pair with target sum in sorted array function findPairWithSum(arr: number[], target: number): number[] | null { let left = 0; let right = arr.length - 1; while (left < right) { const sum = arr[left] + arr[right]; if (sum === target) { return [left, right]; } else if (sum < target) { left++; } else { right--; } } return null; } // Examples console.log(isPalindrome("racecar")); // true console.log(isPalindrome("hello")); // false console.log(findPairWithSum([1, 2, 3, 4, 6], 6)); // [1, 3] (indices)
Python
# Check if string is palindrome def is_palindrome(s: str) -> bool: left = 0 right = len(s) - 1 while left < right: if s[left] != s[right]: return False left += 1 right -= 1 return True # Find pair with target sum in sorted array def find_pair_with_sum(arr: list[int], target: int) -> list[int] | None: left = 0 right = len(arr) - 1 while left < right: current_sum = arr[left] + arr[right] if current_sum == target: return [left, right] elif current_sum < target: left += 1 else: right -= 1 return None # Examples print(is_palindrome("racecar")) # True print(is_palindrome("hello")) # False print(find_pair_with_sum([1, 2, 3, 4, 6], 6)) # [1, 3] (indices)
OCaml
(* Check if string is palindrome *) let is_palindrome s = let len = String.length s in let rec check left right = if left >= right then true else if s.[left] <> s.[right] then false else check (left + 1) (right - 1) in check 0 (len - 1) (* Find pair with target sum in sorted array *) let find_pair_with_sum arr target = let len = Array.length arr in let rec search left right = if left >= right then None else let current_sum = arr.(left) + arr.(right) in if current_sum = target then Some (left, right) else if current_sum < target then search (left + 1) right else search left (right - 1) in search 0 (len - 1) (* Examples *) let palindrome_result = is_palindrome "racecar";; (* true *) let pair_result = find_pair_with_sum [|1; 2; 3; 4; 6|] 6;; (* Some (1, 3) *)
When to Use Two-Pointer Technique:
- Palindrome checking: Compare characters from both ends
- Pair finding: Find two elements that meet a condition
- Array reversal: Swap elements from both ends
- Sorted array problems: Take advantage of the sorted order
Basic Sorting Algorithms
Sorting algorithms arrange elements in a specific order (ascending or descending). These basic algorithms are fundamental to understanding more complex sorting techniques.
š Real-Life Analogy:
Think of organizing books on a shelf by height, or arranging playing cards in your hand from lowest to highest. Each sorting algorithm has a different strategy for achieving this organization.
Bubble Sort
Bubble sort repeatedly steps through the list, compares adjacent elements, and swaps them if they're in the wrong order. The largest elements "bubble" to the end.
š Visual Step-by-Step Process:
Initial Array: [64, 34, 25, 12, 22, 11, 90] Pass 1: Compare adjacent elements, swap if needed [64, 34, 25, 12, 22, 11, 90] ā Compare 64 & 34 [34, 64, 25, 12, 22, 11, 90] ā Swap! Compare 64 & 25 [34, 25, 64, 12, 22, 11, 90] ā Swap! Compare 64 & 12 [34, 25, 12, 64, 22, 11, 90] ā Swap! Compare 64 & 22 [34, 25, 12, 22, 64, 11, 90] ā Swap! Compare 64 & 11 [34, 25, 12, 22, 11, 64, 90] ā Swap! Compare 64 & 90 [34, 25, 12, 22, 11, 64, 90] ā No swap needed ā Largest element "bubbled" to end Pass 2: Repeat for remaining elements [25, 34, 12, 22, 11, 64, 90] ā After swaps ā Second largest in position Pass 3: Continue until sorted [25, 12, 22, 11, 34, 64, 90] ā After swaps ā Third largest in position Final: [11, 12, 22, 25, 34, 64, 90] ā Sorted!
š Real-World Applications:
š Educational Systems: Teaching sorting concepts due to its simplicity and clear visualization
š§ Embedded Systems: Small datasets where simplicity matters more than efficiency
š Data Validation: Quick checks on small arrays to verify if data is already sorted
š® Game Development: Sorting small leaderboards or inventory items (< 50 elements)
ā” Performance Benchmarks:
Array Size | Bubble Sort | Quick Sort | Use Case |
---|---|---|---|
10 elements | ~0.001ms | ~0.002ms | ā Bubble sort acceptable |
100 elements | ~0.1ms | ~0.01ms | ā ļø Quick sort better |
1,000 elements | ~10ms | ~0.1ms | ā Avoid bubble sort |
10,000 elements | ~1000ms | ~1ms | ā Never use bubble sort |
TypeScript
// Bubble Sort Implementation function bubbleSort(arr: number[]): number[] { const n = arr.length; const result = [...arr]; // Create a copy to avoid modifying original for (let i = 0; i < n - 1; i++) { // Flag to optimize - if no swaps occur, array is sorted let swapped = false; for (let j = 0; j < n - i - 1; j++) { if (result[j] > result[j + 1]) { // Swap elements [result[j], result[j + 1]] = [result[j + 1], result[j]]; swapped = true; } } // If no swapping occurred, array is sorted if (!swapped) break; } return result; } // Example with step-by-step visualization function bubbleSortWithSteps(arr: number[]): void { const n = arr.length; console.log(`Initial array: [${arr.join(', ')}]`); for (let i = 0; i < n - 1; i++) { console.log(`\nPass ${i + 1}:`); let swapped = false; for (let j = 0; j < n - i - 1; j++) { if (arr[j] > arr[j + 1]) { [arr[j], arr[j + 1]] = [arr[j + 1], arr[j]]; swapped = true; console.log(` Swapped ${arr[j + 1]} and ${arr[j]}: [${arr.join(', ')}]`); } } if (!swapped) { console.log(` No swaps needed - array is sorted!`); break; } } } // Example usage const numbers = [64, 34, 25, 12, 22, 11, 90]; const sorted = bubbleSort(numbers); console.log('Sorted:', sorted); // [11, 12, 22, 25, 34, 64, 90]
Python
# Bubble Sort Implementation def bubble_sort(arr): n = len(arr) result = arr.copy() # Create a copy to avoid modifying original for i in range(n - 1): # Flag to optimize - if no swaps occur, array is sorted swapped = False for j in range(n - i - 1): if result[j] > result[j + 1]: # Swap elements result[j], result[j + 1] = result[j + 1], result[j] swapped = True # If no swapping occurred, array is sorted if not swapped: break return result # Example with step-by-step visualization def bubble_sort_with_steps(arr): n = len(arr) print(f"Initial array: {arr}") for i in range(n - 1): print(f"\nPass {i + 1}:") swapped = False for j in range(n - i - 1): if arr[j] > arr[j + 1]: arr[j], arr[j + 1] = arr[j + 1], arr[j] swapped = True print(f" Swapped {arr[j + 1]} and {arr[j]}: {arr}") if not swapped: print(" No swaps needed - array is sorted!") break # Example usage numbers = [64, 34, 25, 12, 22, 11, 90] sorted_numbers = bubble_sort(numbers) print('Sorted:', sorted_numbers) # [11, 12, 22, 25, 34, 64, 90]
OCaml
(* Bubble Sort Implementation *) let bubble_sort arr = let n = Array.length arr in let result = Array.copy arr in let rec bubble_pass i swapped = if i >= n - 1 then swapped else if result.(i) > result.(i + 1) then ( let temp = result.(i) in result.(i) <- result.(i + 1); result.(i + 1) <- temp; bubble_pass (i + 1) true ) else bubble_pass (i + 1) swapped in let rec sort_passes pass = if pass >= n - 1 then result else let swapped = bubble_pass 0 false in if not swapped then result else sort_passes (pass + 1) in sort_passes 0 (* Example usage *) let numbers = [|64; 34; 25; 12; 22; 11; 90|];; let sorted = bubble_sort numbers;; (* sorted = [|11; 12; 22; 25; 34; 64; 90|] *) (* Functional approach using lists *) let rec find_min_and_remove lst = match lst with | [] -> failwith "Empty list" | [x] -> (x, []) | x :: xs -> let (min_val, remaining) = find_min_and_remove xs in if x <= min_val then (x, xs) else (min_val, x :: remaining) let rec selection_sort_list = function | [] -> [] | lst -> let (min_val, remaining) = find_min_and_remove lst in min_val :: selection_sort_list remaining (* Example usage *) let numbers = [|64; 34; 25; 12; 22; 11; 90|];; let sorted = selection_sort numbers;; (* sorted = [|11; 12; 22; 25; 34; 64; 90|] *) let numbers_list = [64; 34; 25; 12; 22; 11; 90];; let sorted_list = selection_sort_list numbers_list;;
Selection Sort Characteristics:
- Time Complexity: O(n²) in all cases
- Space Complexity: O(1) - sorts in place
- Not Stable: May change relative order of equal elements
- Minimum Swaps: Makes at most n-1 swaps
Insertion Sort
Insertion sort builds the sorted array one element at a time by repeatedly taking an element from the unsorted portion and inserting it into its correct position in the sorted portion.
TypeScript
// Insertion Sort Implementation function insertionSort(arr: number[]): number[] { const n = arr.length; const result = [...arr]; // Create a copy for (let i = 1; i < n; i++) { const key = result[i]; let j = i - 1; // Move elements greater than key one position ahead while (j >= 0 && result[j] > key) { result[j + 1] = result[j]; j--; } // Insert key at its correct position result[j + 1] = key; } return result; } // Step-by-step visualization function insertionSortWithSteps(arr: number[]): void { const n = arr.length; console.log(`Initial array: [${arr.join(', ')}]`); for (let i = 1; i < n; i++) { const key = arr[i]; let j = i - 1; console.log(`\nPass ${i}: Inserting ${key}`); console.log(` Sorted portion: [${arr.slice(0, i).join(', ')}]`); // Move elements greater than key one position ahead while (j >= 0 && arr[j] > key) { arr[j + 1] = arr[j]; j--; } // Insert key at its correct position arr[j + 1] = key; console.log(` After insertion: [${arr.join(', ')}]`); } } // Binary insertion sort (optimization) function binaryInsertionSort(arr: number[]): number[] { const result = [...arr]; for (let i = 1; i < result.length; i++) { const key = result[i]; // Find location to insert using binary search let left = 0, right = i; while (left < right) { const mid = Math.floor((left + right) / 2); if (result[mid] <= key) { left = mid + 1; } else { right = mid; } } // Shift elements and insert for (let j = i; j > left; j--) { result[j] = result[j - 1]; } result[left] = key; } return result; } // Example usage const numbers = [64, 34, 25, 12, 22, 11, 90]; const sorted = insertionSort(numbers); console.log('Sorted:', sorted); // [11, 12, 22, 25, 34, 64, 90]
Python
# Insertion Sort Implementation def insertion_sort(arr): n = len(arr) result = arr.copy() # Create a copy for i in range(1, n): key = result[i] j = i - 1 # Move elements greater than key one position ahead while j >= 0 and result[j] > key: result[j + 1] = result[j] j -= 1 # Insert key at its correct position result[j + 1] = key return result # Step-by-step visualization def insertion_sort_with_steps(arr): n = len(arr) print(f"Initial array: {arr}") for i in range(1, n): key = arr[i] j = i - 1 print(f"\nPass {i}: Inserting {key}") print(f" Sorted portion: {arr[:i]}") # Move elements greater than key one position ahead while j >= 0 and arr[j] > key: arr[j + 1] = arr[j] j -= 1 # Insert key at its correct position arr[j + 1] = key print(f" After insertion: {arr}") # Binary insertion sort (optimization) def binary_insertion_sort(arr): result = arr.copy() for i in range(1, len(result)): key = result[i] # Find location to insert using binary search left, right = 0, i while left < right: mid = (left + right) // 2 if result[mid] <= key: left = mid + 1 else: right = mid # Shift elements and insert for j in range(i, left, -1): result[j] = result[j - 1] result[left] = key return result # Example usage numbers = [64, 34, 25, 12, 22, 11, 90] sorted_numbers = insertion_sort(numbers) print('Sorted:', sorted_numbers) # [11, 12, 22, 25, 34, 64, 90]
OCaml
(* Insertion Sort Implementation *) let insertion_sort arr = let n = Array.length arr in let result = Array.copy arr in for i = 1 to n - 1 do let key = result.(i) in let j = ref (i - 1) in (* Move elements greater than key one position ahead *) while !j >= 0 && result.(!j) > key do result.(!j + 1) <- result.(!j); j := !j - 1 done; (* Insert key at its correct position *) result.(!j + 1) <- key done; result (* Functional approach using lists *) let rec insert x = function | [] -> [x] | y :: ys when x <= y -> x :: y :: ys | y :: ys -> y :: insert x ys let rec insertion_sort_list = function | [] -> [] | x :: xs -> insert x (insertion_sort_list xs) (* Tail-recursive version *) let insertion_sort_tail_rec lst = let rec sort_acc acc = function | [] -> List.rev acc | x :: xs -> sort_acc (insert x acc) xs in sort_acc [] lst (* Example usage *) let numbers = [|64; 34; 25; 12; 22; 11; 90|];; let sorted = insertion_sort numbers;; (* sorted = [|11; 12; 22; 25; 34; 64; 90|] *) let numbers_list = [64; 34; 25; 12; 22; 11; 90];; let sorted_list = insertion_sort_list numbers_list;;
Insertion Sort Characteristics:
- Time Complexity: O(n²) worst/average case, O(n) best case
- Space Complexity: O(1) - sorts in place
- Stable: Maintains relative order of equal elements
- Adaptive: Efficient for small datasets and nearly sorted arrays
- Online: Can sort a list as it receives it
Algorithm | Best Case | Average Case | Worst Case | Space | Stable |
---|---|---|---|---|---|
Bubble Sort | O(n) | O(n²) | O(n²) | O(1) | Yes |
Selection Sort | O(n²) | O(n²) | O(n²) | O(1) | No |
Insertion Sort | O(n) | O(n²) | O(n²) | O(1) | Yes |
Ready to practice?
Now that you understand fundamental algorithms, let's focus on practice strategies!
Continue to Practice Focus ā