Stochastic Entity

Uniform Sampling

Wed, 04 Sep 2024 00:00:00 +0000

To uniformly sample points over a volume one needs to consider the infinitesimal volume element corresponding to that specific geometry in a given coordinate system.

For example, let’s consider uniformly sampling points over a linear geometry between a and b. The infinitesimal volume (in this case line) element is simply given by $dV = dx$. In this case, one can trivially get the corresponding points using: $pt = \mathcal{U}[0, 1] * (b - a) + a$.

import numpy as np
pts = np.random.rand(100) * 3 + 2  # 100 pts between 2 and 5

What if we have to sample uniformly over a square in the range $x \in [1, 3)$ and $y \in [2, 5)$. The infinitesimal volume (in this case area) element in cartesian coordinate system is given by $dV = dx\ dy$. Since it can be written as a product of infinitesimal line elements for each dimension, one can randomly sample points independently across each dimension.

pts_x = np.random.rand(1000) * 2 + 1
pts_y = np.random.rand(1000) * 3 + 2

Now let’s say we have to sample points uniformly within a circular region of radius $r_{max}$. If working in cartesian coordinate system, one can first sample point uniformly in the 2D square as above with the extents $[-r_{max}, r_{max}]$ and then only keep the point if it lies within the circular region i.e. $x^2 + y^2 \le r_{max}^2$. This will give the correct result; however, its a little wasteful in the sense that on average we would have to sample $n_{pts} \left[ 1 + \frac{\pi}{4}\right]$ points before we reach the required number of points.

A better and more efficient approach would to be work in the circular coordinate system $(r, \theta)$. Naively one might think to uniformly sample points with $r \in [0, r_{max}]$ and $\theta \in [0, 2 \pi]$; and them transforming them to $x$ and $y$ using:

\[x = r \cos \theta \qquad y = r \sin \theta\]

Here is the result with $r_{max} = 4$

However, using this strategy we don’t get uniform distribution. Points are over-sampled as one goes radially inwards. A correct approach is to look at the infinitesimal volume element in spherical coordinate system, which is

\[dV = r\ dr \ d\theta = d \left(\frac{r^2}{2} \right)\ d\theta = dr'\ d\theta\]

One can then uniformly sample in $r’$ where $r’ = r^2 / 2$ and in $\theta$. In the case of $r_{max} = 4$, this implies sampling $r’$ uniformly over the range $r’ \in [0, 16)$.

pts_r_prime = np.random.random(500) * 16
pts_theta = np.random.random(500) * 2 * np.pi

pts_r = np.sqrt(2 * pts_r_prime)

pts_x = pts_r * np.sin(pts_theta)
pts_y = pts_r * np.cos(pts_theta)

What if we go up a dimension and have to sample points uniformly within a spherical volume till $r = r_{max}$. In spherical coordinate system, the infinitesimal volume element is given by:

\[dV = r^2 \sin \theta\ dr\ d\theta\ d\phi = [ d(r^3/3)]\ [d\cos \theta]\ [d\phi]\]

Thus we have to:

Uniformly sample in $\phi \in [0, 2 \pi]$
Uniformly sample in $\cos \theta$. Since $\theta$ goes from 0 to $\pi$, we have to sample $\cos \theta \in [1, -1]$.
Uniformly sample in $r’ \in [0, r_{max}^3 / 3]$, where $r’ = r^3 / 3$

$\rightarrow$ If we have a coordinate system $(u_1, u_2, u_3)$ that is related to the cartesian coordinate system via some transforms $x = f(u_1, u_2, u_3), y = g(u_1, u_2, u_3), z=h(u_1, u_2, u_3)$; then the infinitesimal volume element in the new coordinate system is given by:

\[dV = \left| \frac{\partial(x, y, z)}{\partial(u_1, u_2, u_3)}\right| du_1\ du_2\ du_3\]

More SQL

Mon, 01 Apr 2024 17:18:23 +0000

Let’s say we have the following table Users, where user_id is the column with unique values for this table. The goal is to find for each user, the minimum of the two values for col_a and col_b.

+-------------+----------+
| Column Name | Type     |
+-------------+----------+
| user_id     | int      |
| col_a       | int      |
| col_b       | int      |
+-------------+----------+

To do this, we can use the LEAST function (and its counterpart GREATEST):

SELECT
  user_id,
  LEAST(col_a, col_b) AS min_val
FROM Users

What if we want to cap a column to some minimum value cap_val?

SELECT
    user_id,
    GREATEST(10, col_a) AS cap_val
FROM Users

Window functions

Let’s say we have a table called Purchases, where each row contains a user with user_id purchasing product product_name on buy_date of quantity.

| user_id | product_name | buy_date | quantity |
| ------- | ------------ | -------- | -------- |
| 1       | A            | 5        | 10       |
| 1       | B            | 6        | 12       |
| 2       | C            | 2        | 21       |
| 2       | D            | 2        | 42       |
| 2       | F            | 1        | 16       |
| 2       | E            | 6        | 18       |
| 3       | E            |          |          |

NOTE that the frame used for calculating window function results for a given row depends on the presence of ORDER BY as stated here. To state here:

If ORDER BY present: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
If ORDER BY not present: RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING

Also, these window functions don’t use frame but instead the entire partition: DENSE_RANK(), LAG(), LEAD(), RANK(), ROW_NUMBER().

Here are some common window functions applied with OVER(PARTITION BY user_id ORDER BY buy_date DESC), as per the query (notice especially the behavior of LAST_VALUE and SUM):

SELECT 
	*,
    ROW_NUMBER() OVER(PARTITION BY user_id ORDER BY buy_date) AS row_num, 
    RANK() OVER(PARTITION BY user_id ORDER BY buy_date) AS `rank`, 
    DENSE_RANK() OVER(PARTITION BY user_id ORDER BY buy_date) AS `dense_rank`,
    FIRST_VALUE(product_name) OVER(PARTITION BY user_id ORDER BY buy_date) AS `first_product`,
    LAST_VALUE(product_name) OVER(PARTITION BY user_id ORDER BY buy_date) AS `last_product`,
    LEAD(product_name) OVER(PARTITION BY user_id ORDER BY buy_date) AS `lead_product`,
    LAG(product_name) OVER(PARTITION BY user_id ORDER BY buy_date) AS `lag_product`,
    SUM(quantity) OVER(PARTITION BY user_id ORDER BY buy_date) AS `total_quantity`
FROM Purchases;

The output is:

| user_id | product_name | buy_date | quantity | row_num | rank | dense_rank |
| ------- | ------------ | -------- | -------- | ------- | ---- | ---------- |
| 1       | A            | 5        | 10       | 1       | 1    | 1          |
| 1       | B            | 6        | 12       | 2       | 2    | 2          |
| 2       | F            | 1        | 16       | 1       | 1    | 1          |
| 2       | C            | 2        | 21       | 2       | 2    | 2          |
| 2       | D            | 2        | 42       | 3       | 2    | 2          |
| 2       | E            | 6        | 18       | 4       | 4    | 3          |
| 3       | E            |          |          | 1       | 1    | 1          |

| user_id | product_name | buy_date | quantity | first_product | last_product | lead_product | lag_product | total_quantity |
| ------- | ------------ | -------- | -------- | ------------- | ------------ | ------------ | ----------- | -------------- |
| 1       | A            | 5        | 10       | A             | A            | B            |             | 10             |
| 1       | B            | 6        | 12       | A             | B            |              | A           | 22             |
| 2       | F            | 1        | 16       | F             | F            | C            |             | 16             |
| 2       | C            | 2        | 21       | F             | D            | D            | F           | 79             |
| 2       | D            | 2        | 42       | F             | D            | E            | C           | 79             |
| 2       | E            | 6        | 18       | F             | E            |              | D           | 97             |
| 3       | E            |          |          | E             | E            |              |             |                |

Window function with GROUP BY

Let’s say we have a table called Orders; where each row (unique order_id) is an order for a certain product along with the quantity of that product bought.

+-------------+----------+
| Column Name | Type     |
+-------------+----------+
| order_id    | int      |
| product_id  | int      |
| quantity    | int      |
+-------------+----------+

We would like to rank the products based on their total quantity purchased.

To do this one would first do an aggregation to count the number of items of each product bought and then rank them using a window function as follows:

SELECT
  product_id,
  RANK() OVER(ORDER BY total_quantity DESC) AS `rank`
FROM (
  SELECT product_id, SUM(quantity) AS total_quantity
  FROM Orders
  GROUP BY 1
) AS k

However, the window and the group by can be combined as follows:

SELECT product_id,
       RANK() OVER(ORDER BY SUM(quantity)) AS `rank`
FROM Orders
GROUP BY 1

The reason being that window function are evaluated in the SELECT clause; which executes after the GROUP BY clause.

Common data structures

Sun, 31 Mar 2024 00:00:00 +0000

Stack

A stack is a data structure based on the Last In First Out principle. A common example is a stack of cards or CD-ROM’s, where the last item that was added to the stack will be the first one out when removing from the stack.

$\rightarrow$ In python, the default list class serves as an implementation of a stack; as you append to the end of the list, and the pop operation removes the last added item.

In this post, we saw a recursive implementation of pre-order traversal of a binary tree. We can convert it into a iterative algorithm using a stack as follows:

def pre_order_traversal(node):
    my_stack = [node]

    while my_stack:
        curr_node = my_stack.pop()
        print(curr_node.val)

        if curr_node.right is not None:
            my_stack.append(curr_node.right)
        if curr_node.left is not None:
            my_stack.append(curr_node.left)

Note that under the hood, recursive algorithms are evaluated using a stack. Whenever a recursive step is encountered, the corresponding function is pushed onto the stack. After the function has returned with some value, the item is popped out, so that the execution can proceed to the next step.

Given a string s containing just the characters $(, ), [, ], \{, \}$, determine if the input string is valid.

We maintain a stack to keep track of the latest paranthesis scope. Any time we encounter an open paranthesis, we push it onto the stack. When we encounter a closing paranthesis, we pop from the stack and check if the open-close pair matches or not. If it doesn’t match, then we have an invalid string.

def valid_paranthesis(s):
    my_stack = []

    pair_hashmap = {
        '[' : ']',
        '{': '}',
        '(', ')',
    }

    for char in s:
        if char in ['(', '[', '{']:
            my_stack.append(char)
        else:
            last_open = my_stack.pop()
            if pair_hashmap[last_open] != char:
                return False
    return True

Queue

A queue is a data structure that follows the First In First Out principle. A common example is people lined up in a queue at a bus stop. The person who came first will be the first to board the bus.

$\rightarrow$ In python, use deque from collections module for queue implementation.

In this post, we saw how a queue can be used to obtain a level-order traversal of a binary tree. In general, if we want to perform a BFS over an unweighted graph, we can use a queue. The following example demonstrates this:

Given a binary matrix, return the shortest path that starts from the top-left corner and ends in the bottom-right corner. Only cells with value 0 can be visited. From a cell you can move in any of the 4 directions sharing an edge.

The binary matrix can be thought of as a graph, with each cell as a node, sharing an edge with its adjacent cells. To find the shortest path from a source to destination, we can start a BFS from the start node. As stated above, we will use a queue for this task. We also need to make sure that we keep track of cells that we have already visited to avoid loops and to avoid going down a path that had(or is) already been(being) traversed.

Since we also want to explicitly print out the shortest path and not just the length, we need to, while visiting a node keep track of where we came from. To do this we will make a separate data structure for the node.

from dataclasses import dataclass

@dataclass
class Node:
    pos_x: int
    pos_y: int
    came_from: Node

def shortest_path_binary(grid):
    m, n = len(grid), len(grid[0])

    visited = set()
    
    # Start position
    start_node = Node(0, 0, None)
    if grid[0][0] == 1:  # edge case: start position itself can't be visited
        return None

    queue = deque([start_node])
    visited.add((0, 0))

    while queue:
        node = queue.popleft()

        # stopping criteria
        if (node.pos_x == m - 1) and (node.pos_y == n - 1):
            return node

        for (step_x, step_y) in directions:
            new_pos_x = node.pos_x + step_x
            new_pos_y = node.pos_y + step_y

            # criteria to visit connected nodes (cells)
            if ((new_pos_x >= 0) and (new_pos_x < m) and  
                (new_pos_y >= 0) and (new_pos_y < n) and 
                (grid[new_pos_x][new_pos_y] == 0) and 
                ((new_pos_x, new_pos_y) not in visited)):
                visited.add((new_pos_x, new_pos_y))
                next_node = Node(new_pos_x, new_pos_y, came_from=node)
                queue.append(next_node)

    # We only reach this if no path present 
    return None

Once we have the destination node returned using the above function, we can simply back-track using the came_from variable as follows:

curr_node = shortest_path_binary(grid)

res = []
while curr_node is not None:
    res.append((curr_node.pos_x, curr_node.pos_y))
    curr_node = curr_node.came_from
    
print(res[::-1])

Sliding window

Sat, 30 Mar 2024 00:00:00 +0000

This iterative technique is useful when dealing with problems that involve finding a (longest/shortest) subsequence that satisfies a given criteria in a string or an array. We maintain two pointers, left and right; initially located at the first index.

$\rightarrow$ For the case of finding the longest subarray/substrig, the right pointer is moved as long as the subarray defined by the range [left, right] satisfies the window (subarray) criterion. Once the window condition is broken, the left pointer is moved until the window condition gets satisfied or it catches up to the right pointer. A typical logic will look like this:

def solution(nums, *args, **kwargs):
    left, res = 0, 0
    for right in range(len(nums)):
        while (window_condition_broken) and (left < right):
            # do some logic here
            left += 1
        # update res
    return res

$\rightarrow$ For the case of finding shortest subsequence, the strategy is exactly reversed. The right pointer is moved until the subarray condition is satisfied; at which point the left pointer is incremented until the condition is broken.

Let’s apply the sliding window technique to this problem: Given a string find the longest substring without repeating characters.

We will maintain a data structure to keep track of characters present in the substring defined by [left, right]. An obvious choice here is a set.
We then keep moving the right pointer increasing the subarray length.
The moment we reach a position where the character has already been encountered, the condition of non-repeatedness is broken.
We then move the left pointer until we reach a position such that the character at that left location is the one causing the broken window condition.
As we move the left pointer, we also need to make sure that we remove the previous characters from the set.

def longest_non_repeating(s):
    left, max_length = 0, 0
    encountered = set()
    for right in range(len(s)):
        while (s[right] in encountered) and (left < right):
            encountered.remove(s[left])
            left += 1

        encountered.add(s[right])
        max_length = max(max_length, right - left + 1)
    return max_length

Let’s consider another problem: Given an array of positive integers nums and a positive integer target, return the minimal length of a subarray whose sum is greater than or equal to target. If there is no such subarray, return 0 instead.

The window condition here is that the elements of the window should sum to greater or equal to the target.

What if the window at given moment sums to less than the target? The only way to make the window sum greater or equal to target is to move the right pointer.
What id the window at given moment sums to greater than target? In this case, since our goal is to find the shortest window, the only way to shorten the window is to move the left pointer, in the hopes that the new window also satisfies the window condition.

def min_subarray_sum(nums, target):
    left = 0

    curr_sum, min_length = 0, float('inf')
    for right in range(len(nums)):
        curr_sum += nums[right]
        while (curr_sum >= target) and (left <= right):
            min_length = min(min_length, right - left + 1)

            curr_sum -= nums[left]
            left += 1
        
    return 0 if min_length == float('inf') else min_length

The questions we have considered above were windows that had variable size. Let us now consider a problem, where the size of the window is fixed - given two strings s and p, return an array of all the start indices of p’s anagrams in s. Both s and p only consists of lowercase english letters.

An anagram of a string is another string of equal length but with the characters jumbled up. Every though the ordering changes, the count map, i.e. the number of times each character appears remains unchanged. Thus our sliding window will always have the length equal to that of the target string, in this case p. We will check if the count map of the window matches that of the target, if yes we have a match; if not we slide the window one step to the right and keep going.

Note that when we slide the window the only characters that are being affected is the newly added rightmost character and the leftmost dropped character. Hence for checking the match we don’t need to scan the entire hashmap.

The count map will be maintained across each of the lowercase letter; hence for the string abbad, the map will look like a:2, b:2, c:0, d:1, e:0, ..... . We will maintain a variable called matches that counts how many out of the 26 characters match in their counts between the target and the current window. Let’s say that for the current window, the count of letter f matches and that when we slide the window the newly added character was also f. In that case the match count will decrease. Conversely, if the count of letter f did not match previously, then we need to check if after adding, it matches or not. If it still doesn’t match, we do nothing, else if it matches, we increase the match by 1.

def findAnagrams(self, s: str, p: str) -> List[int]:
    if len(p) > len(s):
        return []
        
    length = len(p)

    hashmap_p = defaultdict(int)
    hashmap_window = defaultdict(int)
    for i in range(len(p)):
        hashmap_p[p[i]] += 1
        hashmap_window[s[i]] += 1

    # check matches for the first window.
    matches = 0
    for ele in string.ascii_lowercase:
        if hashmap_p[ele] == hashmap_window[ele]:
            matches += 1

    res = []
    if matches == len(string.ascii_lowercase):
        res.append(0)
    
    for ptr in range(len(p), len(s)):
        # handle newly added rightmost element
        right_element = s[ptr]
        right_already_match = hashmap_p[right_element] == hashmap_window[right_element]

        hashmap_window[right_element] += 1
        if right_already_match:
            matches -= 1
        else:
            if hashmap_p[right_element] == hashmap_window[right_element]:
                matches += 1

        # handle leftmost dropped element
        left_element = s[ptr - length]
        left_already_match = hashmap_p[left_element] == hashmap_window[left_element]

        hashmap_window[left_element] -= 1
        if left_already_match:
            matches -= 1
        else:
            if hashmap_p[left_element] == hashmap_window[left_element]:
                matches += 1

        # Check if this new window matches
        if matches == len(string.ascii_lowercase):
            res.append(ptr - length + 1)

    return res

Recursion

Sat, 30 Mar 2024 00:00:00 +0000

A recursive algorithm is one which calls itself, on a subset of input. To setup a correct recursion and to avoid infinite loops, one must take the following into account:

setup the base case
recursive calls should lead towards the base case

A classical example of recursion is the Tower of Hanoi. There are 3 poles, with the first pole containing stack of discs with decreasing diameter. The goal is to recreate this structure in the third pole, using the second pole as helper. At any time one can only move 1 disc and one can never stack a disc of higher diameter on top of a lower diameter one.

What is the base case?

What if we had only one disc, what do we do? Simple, we just move it to the destination pole in one move. What if we had no disc, well, in that case we don’t do anything and simply stop.

What is the recursive setup?

Assume that somehow we were able to get the top $n-1$ discs to the second pole, what do we do now. In that case, we move the last remaining disc from the first pole to the third pole and then somehow get the $n-1$ discs from the second pole to the third pole. But who is that somehow? It is the algorithm itself!

def tower_of_hanoi(n, start, middle, end):
    if n > 0:
        tower_of_hanoi(n-1, start, end, middle)
        print(f"Move disc from {start} to {end}")
        tower_of_hanoi(n-1, middle, start, end)

As we saw previously, a depth-first traversal of a binary tree can also be implemented recursively.

Recusive/iterative duality

Note: A recursive algorithm can also be implemented as an iterative algorithm and vice-versa. Sometimes, one approach is much easier to help solve the problem than the other. So, if you feel stuck or having a hard time with one approach, using the complementary approach might help.

Most of the graph and trees problem are easier to solve with the recursive approach; while problems involving arrays and dynamic programming are mostly easier with the iterative approach. The reason being that an array can be traversed from left to right or from right to left with the same effort. However, trees for example have a natural ordering; it is easier to go from node to its children, rather than from children to its parent. The base case in this type of data structure are the leaf children. Hence a recusive approach works best.

In this post we saw an iterative implementation of Kadane’s algorithm. Here we turn it into a recursion. The strategy is the same - we find, for each index, the max sum subarray that ends on that index, and then we choose the max out of these. However the direction of traversal is opposite; whereas in the iterative case, we started filling from left to right; for the recursive case we start the recursive from the last index and recurse to the base case - the base case being the single first element.

def max_sum_subarray_rec(nums):
    def util(i):
        if i == 0:  # base case
            curr_sum = nums[i]
        else:  # recursive case
            curr_sum = max(nums[i], nums[i] + util(i-1))
        util.res = max(util.res, curr_sum)
        return curr_sum
    
    util.res = float('-inf')
    util(len(nums) - 1)
    return util.res

Depth Sum

Given a list where each location can contain either an integer or a list, compute the depth sum?

E.g. $nums = [1, 2, [3, 4], 5]$, the depth sum is $ 1*1 + 1*2 + 2*3 + 2*4 + 1*5 = 22 $.

For $nums = [1, [2, [3]], 4]$, the depth sum is $1*1 + 2*2 + 3*3 + 1*4 = 18$.

Let’ say we already have an algorithm that computes the depth sum; how will it look like. We can iterate over the input argument, in this case a list. If the element is an integer we can simply update the sum. If however, the element is a list, we can use the algorithm itself to get the depth sum of the element; with depth one level deeper.

What will the base case? The base case will be when the input consists of all integers and no nested lists.

def depth_sum(nums):
    def _depth_sum(nums, depth=0):
        curr_sum = 0
        for ele in nums:
            if isinstance(ele, list):
                curr_sum += _depth_sum(ele, depth+1)  # recurse one level deeper
            else:
                curr_sum += ele
        return curr_sum
    return _depth_sum(nums)

Merge Sort

Let’s take the classic problem of sorting an array and see how we can apply recursion to solve the task.

We basically want a function that takes an input and sorts it. Let’s say we already have a function to do this; how will it look like. We can break the input array into parts and sort each of the part. This is the recursive aspect - we can use the function itself to accomplish this. The base case being when there is a single element, in which case nothing needs to be done.

Now that we have two sorted arrays, how to put them together into one sorted array? We can use the pointer technique as discussed here. We keep two pointers at the start of each of the two sorted arrays. We then compare the values at these pointer locations and choose the smallest one and keep going. This leads to the mergesort algorithm as follows:

def merge_sort(nums)
    def _merge_sort(nums, start, end):
        if len(nums) == 0:
            raise ValueError("Empty input array!") 
        
        if start == end:  # base case
            return [nums[start]]

        mid = (start + end) // 2
        _merge_sort(nums, start, mid)  # recurse on the left
        _merge_sort(nums, mid + 1, end)  # recurse on the right

        # merge two sorted arrays, in-place
        i = 0
        while i < mid + 1:
            if nums[i] > nums[mid + 1]:
                nums[mid + 1], nums[i] = nums[i], nums[mid + 1]
            i += 1

Split BST

Let us take the following problem. We want to split a BST into two separate BSTs given a target. One subtree has nodes that are all smaller or equal to the target value, while the other subtree has all nodes that are greater than the target value.

Again, let’s apply the recursive formulation:

What is the base case?
- Well, if the root itself is NULL, then we can simply return [None, None].
- What if the root itself is equal to the target? Since its a BST, we know that the right subtree starting at root will all have values greater than the root. Hence we can simply detach the right subtree and return [root, root.right] as the result.
What is the recursive case? Depending on the value of the root, we recurse on the left subtree or the right subtree.

def split_bst(root, target):
    if not root:
        return None, None
    if root.val == target:
        rightchild = root.right
        root.right = None
        return root, rightchild
    elif root.val < target:
        small, large = split_bst(root.right, target)
        root.right = small
        return root, large
    else:
        small, large = split_bst(root.left, target)
        root.left = large
        return small, root

Max Sum Path BST

Given a binary tree, find the path with the maximum sum. The path can start at any node and end at any node.

Here we will apply the recusive version of Kadane’s algorithm. Let’s say, for a given node, we have the two max paths, one ending on the left child and one ending on the right child. Now what is the max path ending on the node? We have three options: either we extend the left path (green), or we extend the right path (blue); or we simply go solo and do not extend either of the paths (red).

But what about the max path in general, that can start at any node and end at any node. Well, for this case we also need to take the path that can be obtaining the left and the right path (shown in orange). That is:

\[max sum = max(max sum, green, blue, red, orange)\]

def max_sum_path_util(root):
    """ Returns max sum path ending on the root.
    """
    if root is None: # Base case
        return 0
    
    left_end_max_sum = max_sum_path_util(root.left)  # recursive case
    right_end_max_sum = max_sum_path_util(root.right)  # recursive case

    max_single_left_right = max(
        left_end_max_sum + root.val,    # green path
        right_end_max_sum + root.val,   # blue path
        root.val,                       # red path
    )

    # global max path
    max_sum = max(
        left_end_max_sum + right_end_max_sum + root.val, # orange path
        max_single_left_right
    )
    max_sum_path_util.res = max(max_sum_path_util.res, max_sum)

    return max_single_left_right

def max_sum_path(root):
    max_sum_path_util.res = float('-inf')

    max_sum_path_util(root)
    return max_sum_path_util.res

Binary Trees

Sat, 30 Mar 2024 00:00:00 +0000

A binary tree is a tree data structure where every node has at most two children. Each node can be of three categories - the root which has no parent, the leaf which have no children, and the inner nodes, which have at least one child. A node can be defined as follows:

class TreeNode:
    def __init__(self, val, left=None, right=None):
        self.val = val
        self.left = left
        self.right = right

Here is an example binary tree:

A traversal can be defined as a strategy in which to visit every node of the tree. We can either go wide or go deep.

$\rightarrow$ For the case where we want to go deep first (depth-first traversal), we can define the following three recursive ways depending on when the current node is marked as visited.

Pre-order traversal

def pre_order_traversal(node) -> None:
  if node is not None:
      print(node.val)
      pre_order_traversal(node.left)
      pre_order_traversal(node.right)

In-order traversal

def in_order_traversal(node) -> None:
  if node is not None:
      in_order_traversal(node.left)
      print(node.val)
      in_order_traversal(node.right)

Post-order traversal

def post_order_traversal(node) -> None:
  if node is not None:
      post_order_traversal(node.left)
      post_order_traversal(node.right)
      print(node.val)

$\rightarrow$ For the case where we want to go wide first (breadth-first search), we can use a queue for the traversal. For more details on queue, refer to this page. Note that this traversal basically prints the values on a per-level basis, hence it is also called as Level-order traversal.

from collections import deque

def bfs(root) -> None:
    queue = deque([root])

    while queue:
        node = queue.popleft()
        print(node.val)

        if node.left:
            queue.append(node.left)
        if node.right:
            queue.append(node.right)

Binary Search tree

A binary search tree (BST) is a binary tree where for every node, the value of the left child is less than or equal to the current node value, and the value of the right child is greater than the current node value. Here is one example:

Note: An in-order traversal of a BST will lead to values in sorted order.

Shortest paths

A BFS is very useful when computing shortest paths. Let us consider a binary matrix where the goal is to start at the top-leftmost position and reach the bottom-rightmost position in the shortest amount of steps. A cell can be visited only if the matrix value at that location is 0 and all the adjacent cells of the path are 8-directionally connected (share an edge or a corner)

from collections import deque

def shortest_path_binary(grid):
    n, m = len(grid), len(grid[0])

    # trivial case: if the starting location itself is invalid
    if grid[0][0] == 1:
        return -1

    queue = deque()
    queue.append((0, 0))
    len_queue = deque([1])
    seen = set((0, 0))

    while queue:
        node = queue.popleft()
        prev_length = len_queue.popleft()

        # logic for reaching the destination
        if node == (n-1, m-1):
            return prev_length

        for x, y in [[0, 1], [0, -1], [1, 0], [-1, 0],
                     [-1, -1], [-1, 1], [1, -1], [1, 1]]:
            neighbor = (node[0] + x, node[1] + y)
            if (
                (neighbor[0] >= 0) and  # boundry condition
                (neighbor[0] < n) and 
                (neighbor[1] >= 0) and 
                (neighbor[1] < m) and 
                (grid[neighbor[0]][neighbor[1]] == 1) and  # visit condition
                (neighbor not in seen)  # avoid looping back
            ):
                seen.add(neighbor)
                queue.append(neighbor)
                len_queue.append(prev_length + 1)
    
    # if we reach here, we haven't found any path
    return -1

from collections import deque

def shortest_path(start):
    """ Shortest path from start to every other vertex.
    """
    shortest_path_map = {start: 0}

    queue = deque([start])
    while queue:
        curr_node = node.popleft()
        for child, weight in zip(curr_node.children, curr_node.weights):
            connceted_path_length = weight + shortest_path_map[curr_node]
            if child not in shortest_path_map:
                shortest_path_map[child] = connected_path_length
                queue.append(child)
            else:
                if shortest_path_map[child] > connected_path_length:
                    shortest_path_map[chid] = connected_path_length
                    queue.append(child)
    return shortest_path_map

SQL Patterns

Thu, 28 Mar 2024 17:18:23 +0000

NOTE: When solving sql questions always make sure that all the required components listed below are taken care of:

SELECT {columns},
    AGG_FUNC {column(s)}
FROM {table(s)}
JOIN_TYPE {other_table(s)}
ON {column_names}
WHERE {conditions}
GROUP BY {columns}
HAVING {condition on aggregates}
ORDER BY {column(s)}
LIMIT {num rows} OFFSET {num}

Keep In Mind

$\rightarrow$ When using RANK() always think if the ordering is ASC or DESC.

Check

Check if values could be NULL
For questions with largest/smallest, check what to do if two or more have the same values
Check the final column names and any column name typos
Check that the final answer does need rounding or not (if numerical), or ordering (if not)
Whenever there is a GROUP BY think about NULL
Whenever doing UNION check if a simple OR in the where clause would work if disjoint sets
Ask if the foreign key can be NULL

Caveat

Let’s say we have two tables Sellers and Orders as given below and the goal is to find all sellers who did not make any sells in the month of Apr.

+-------------+---------+
| Column Name | Type    |
+-------------+---------+
| seller_id   | int     |
| seller_name | int     |
+-------------+---------+

+-------------+---------+
| Column Name | Type    |
+-------------+---------+
| seller_id   | int     |
| sale_month  | Enum    |
+-------------+---------+

One query that you would think will get the job done is:

SELECT s.seller_id
FROM Sellers AS s
LEFT JOIN Orders AS o
    ON s.seller_id = o.seller_id
    WHERE o.sale_month = 'Apr'
GROUP BY s.seller_id
HAVING COUNT(o.seller_id) = 0

However, the above query will give wrong result. This is because in SQL, WHERE is evaluated after the LEFT JOIN. We can understand this with the following example:

+-------------+-------------+
| seller_id   | seller_name |
+-------------+-------------+
| 1           | A           |
| 2           | B           |
+-------------+-------------+

+-------------+-------------+
| seller_id   | sale_month  |
+-------------+-------------+
| 1           | Apr         |
| 1           | May         |
+-------------+-------------+

$\rightarrow$ From the above example, we can see that seller_id=2 did not make any sale in the month of Apr, which is the output that we want.

Now let’s analyze the query we have written. The output after the LEFT JOIN will be:

+-------------+---------------+-------------+--------------+
| s.seller_id | s.seller_name | o.seller_id | o.sale_month |
+-------------+---------------+-------------+--------------+
|      1      |       A       |      1      |     Apr      |
|      1      |       A       |      1      |     May      |
|      2      |       B       |    NULL     |     NULL     |
+-------------+---------------+-------------+--------------+

And after the filtering with the WHERE o.sale_month = 'Apr' we will get:

+-------------+---------------+-------------+--------------+
| s.seller_id | s.seller_name | o.seller_id | o.sale_month |
+-------------+---------------+-------------+--------------+
|      1      |       A       |      1      |     Apr      |
+-------------+---------------+-------------+--------------+

Notice that the row with seller_id=2 has disappeared altogether. So we have to careful with the usage of ON and WHERE when LEFT JOIN or RIGHT JOIN is involved.

A correct query is:

SELECT s.seller_id
FROM Sellers AS s
LEFT JOIN Orders AS o
    ON s.seller_id = o.seller_id
    AND o.sale_month = 'Apr'
GROUP BY s.seller_id
HAVING COUNT(o.seller_id) = 0

By using this, the filtering of the month condition happens in the joining process and not after. The output after the LEFT JOIN will now look like:

+-------------+---------------+-------------+--------------+
| s.seller_id | s.seller_name | o.seller_id | o.sale_month |
+-------------+---------------+-------------+--------------+
|      1      |       A       |      1      |     Apr      |
|      2      |       B       |    NULL     |     NULL     |
+-------------+---------------+-------------+--------------+

The above table, after the groupby and having clause will give the right answer, which is seller_id=2.

Activity

Let’s say we have a table called Logins that contains the following columns:

Logins
user_id
login_date
time_spent

What is the total time that each user spent for the month of Feb, 2024?

SELECT user_id, SUM(time_spent) AS total_time_spent
FROM Logins
WHERE DATE_FORMAT(login_date, '%Y-%m') = '2024-02'
GROUP BY 1;

Which user(s) have logged in the most number of times for a given day?

WITH LoginsPerDay AS (
SELECT user_id, login_date, COUNT(1) AS num_logins
FROM Logins 
GROUP BY 1, 2;
)
SELECT DISTINCT(user_id) AS user_id
FROM LoginsPerDay
WHERE num_logins = (SELECT MAX(num_logins) FROM LoginsPerDay)

Which users have logged in for at least 5 consecutive days? To answer this, we need to do a self-join where for any given row in the first table, we check if there are rows in the second table that are within 5 days (including itself). Now there can be multiple logins on some day; hence we need to dedup using DISTINCT.
```
SELECT DISTINCT l1.user_id,
FROM Logins l1
JOIN Logins l2
ON l1.user_id = l2.user_id
  AND l2.login_date BETWEEN l1.login_date AND DATE_ADD(l1.login_date, INTERVAL 4 DAY)
GROUP BY l1.user_id, l1.login_date
HAVING(COUNT(DISTINCT l2.login_date)) = 5
```
Which users are active? An active user is a user that has made a second login within 3 days of any other of their login. The strategy is the same as that used for the above question. The only difference is we don’t have to worry about distinct; since even a second login on the same day counts.
```
SELECT DISTINCT l1.user_id,
FROM Logins l1
JOIN Logins l2
ON l1.user_id = l2.user_id
  AND l2.login_date BETWEEN l1.login_date AND DATE_ADD(l1.login_date, INTERVAL 3 DAY)
GROUP BY l1.user_id, l1.login_date
HAVING(*) > 1;
```

Friends and Relations

Let us now consider a table called Friends that contains the following columns:

Friends
user1_id
user2_id

Each row is ordered such that user1_id < user2_id.

Calculate number of friends each user has? Since a given row (1, 2) implies that both user1_id and user2_id have increased their friend count by 1; we need to somehow capture the information present in both these columns into a single column. We can do this using UNION.
```
WITH Connections AS (
  SELECT user1_id AS person FROM Friends
  UNION ALL
  SELECT user2_id FROM Friends
)
SELECT person, COUNT(1) AS num_friends
FROM Connections
GROUP BY 1
```
Calculate number of mutual friends for each friend pair? A mutual friend of a pair $A \leftrightarrow B$ is someone such that $A \leftrightarrow C$ and $C \leftrightarrow B$. Thus to find the mutual friends we need to do self-join twice. Since the given table only contains a directed edge $A \rightarrow B$, to create undirected edges we simply replicate the table swapping the user_id as follows:
```
  WITH Connections AS (
      SELECT user1_id, user2_id FROM Friends
      UNION
      SELECT user2_id, user1_id FROM Friends
  )
```

Now we can apply our mutual friend criteria. Note that the first table in the join is the Friends table and not the Connections table, to avoid double counting and maintain the ordering user1_id < user2_id.

SELECT c1.user1_id, c1.user2_id, COUNT(1) AS num_mutual_friends
FROM Friends c1, Connections c2, Connections c3
    WHERE c1.user1_id = c2.user1_id
    AND c2.user2_id = c3.user1_id
    AND c3.user2_id = c1.user2_id
GROUP BY 1, 2;

Now let’s say we have another table called Likes that contains pages that are liked by users.

+-------------+---------+
| Column Name | Type    |
+-------------+---------+
| user_id     | int     |
| page_id     | int     |
+-------------+---------+

We would like to recommend to any given user, pages that their friends liked which they haven’t liked yet; along with the count of the friends that liked that particular page. To do this we will join the Connections table with the Likes table; while making sure that the page in the Likes table row being considered is something that we haven’t liked yet.

NOTE: We can use both NOT IN and NOT EXISTS; however for some reason NOT EXISTS is more optimal.

SELECT f.user1_id AS user_id, l.page_id, COUNT(l.user_id) AS friends_likes
FROM (
    SELECT user1_id, user2_id FROM Friendship
    UNION
    SELECT user2_id, user1_id FROM Friendship
) AS f
JOIN Likes l
ON f.user2_id = l.user_id
AND NOT EXISTS (SELECT page_id FROM Likes WHERE user_id = f.user1_id AND page_id = l.page_id)
GROUP BY f.user1_id, l.page_id

A very similar concept to that of Friends is that of Follower. Let’s say we have a table as given below, where every row implies that a person whose id is follower_id is following a person whose id is user_id. Unlike the Friends table, where any entry implies a undirected relation, in this case every row is a directed relation.

+-------------+------+
| Column Name | Type |
+-------------+------+
| user_id     | int  |
| follower_id | int  |
+-------------+------+

We want to find all pairs of users (user1_id, user2_id) that have the most number of common followers. Notice that C is a common follower for a pair of users $(A, B)$ if there exists two rows in the table $(user\_id=A, follower\_id=C)$ and $(user\_id=B, follower\_id=C)$.

WITH AggTable AS (
    SELECT r1.user_id AS user1_id, r2.user_id AS user2_id, COUNT(1) AS common
    FROM Relations r1, Relations r2
        WHERE r1.user_id < r2.user_id 
        AND r1.follower_id = r2.follower_id
    GROUP BY 1, 2
)

SELECT user1_id, user2_id
FROM AggTable
WHERE common = (SELECT MAX(common) FROM AggTable)

Pivoting

Let’s consider a table Visits that contains entries for a particular month.

Visits
tenant_id
pet_category
visit_date
amount

Each entry (row) corresponds to a visit made by some tenant to supermarket to buy food for one of their pets along with the amount spent. Compute the total amount that each tenant spends on each pet_category, where pet_category is an Enum with values cat, dog, fish. The output should look like:

tenant_id	cat	dog	fish
1	20	40	0
2	34	0	25

We can get the total amount spent by each user per each pet category by simply doing a groupby. However, this will lead to a long-format table, which we will have to pivot to make the wide-format as required by the problem.

WITH ReqTable AS (
    SELECT tenant_id, pet_category, SUM(amount) AS total_amount
    FROM 
    GROUP BY 1, 2;
)
SELECT
    tenant_id,
    SUM(IF(pet_category = 'cat', total_amount, 0)) AS 'cat',
    SUM(IF(pet_category = 'dog', total_amount, 0)) AS 'dog',
    SUM(IF(pet_category = 'fish', total_amount, 0)) AS 'fish'
FROM ReqTable
GROUP BY 1;

Other scenarions

We are given two tables Trips and Users; which could be a log from a ride-share service.

+-------------+----------+
| Column Name | Type     |
+-------------+----------+
| id          | int      |
| client_id   | int      |
| driver_id   | int      |
| city_id     | int      |
| status      | enum     |
| request_at  | date     |     
+-------------+----------+

+-------------+----------+
| Column Name | Type     |
+-------------+----------+
| users_id    | int      |
| banned      | enum     |
| role        | enum     |
+-------------+----------+

The cancellation rate is computed by dividing the number of canceled (by client or driver) requests with unbanned users by the total number of requests with unbanned users on that day. Find the cancellation rate of unbanned users (both client and driver must not be banned) each day between “2013-10-01” and “2013-10-03”. Round Cancellation Rate to two decimal points.

Since we only care about unbanned users we first check that neither client_id nor driver_id are banned.

SELECT 
    request_at AS Day,
    ROUND(AVG(IF(status = 'completed', 0, 1)), 2) AS `Cancellation Rate`
FROM Trips t
    WHERE client_id NOT IN (SELECT users_id FROM Users WHERE banned = 'Yes')
    AND driver_id NOT IN (SELECT users_id FROM Users WHERE banned = 'Yes')
    AND request_at BETWEEN '2013-10-01' AND '2013-10-03'
GROUP BY 1;

Given two tables Customers and Orders, get the customer_id and customer_name of customers who bought products “A”, “B” but did not buy the product “C”.

+---------------------+---------+
| Column Name         | Type    |
+---------------------+---------+
| customer_id         | int     |
| customer_name       | varchar |
+---------------------+---------+

+---------------+---------+
| Column Name   | Type    |
+---------------+---------+
| order_id      | int     |
| customer_id   | int     |
| product_name  | varchar |
+---------------+---------+

We first get the count of each product A, B and C for each customer_id. Then we filter those that have a non-zero values for products A and B, but have zero value for C.

WITH AggTable AS (
    SELECT
        customer_id,
        SUM(IF(product_name = 'A', 1, 0)) AS product_a_count,
        SUM(IF(product_name = 'B', 1, 0)) AS product_b_count,
        SUM(IF(product_name = 'C', 1, 0)) AS product_c_count
    FROM Orders
    GROUP BY customer_id
    HAVING product_a_count > 0 AND product_b_count > 0 AND product_c_count = 0
)
SELECT customer_id, customer_name
FROM AggTable
JOIN Customers USING(customer_id)

Write a solution to report the number of grand slam tournaments won by each player. Do not include the players who did not win any tournament. We have two tables named Players and Championships as follows:

+----------------+---------+
| Column Name    | Type    |
+----------------+---------+
| player_id      | int     |
| player_name    | varchar |
+----------------+---------+

+---------------+---------+
| Column Name   | Type    |
+---------------+---------+
| year          | int     |
| Wimbledon     | int     |
| Fr_open       | int     |
| US_open       | int     |
| Au_open       | int     |
+---------------+---------+

We first CROSS JOIN between the two tables to get all possible combinations of (year, player_id). We then check the number of grand slams won by that player_id for that particular year and then sum that count across all years.

WITH AggTable AS (
    SELECT 
        p.player_id,
        SUM(IF(Wimbledon = player_id, 1, 0) + 
            IF(Fr_open = player_id, 1, 0) + 
            IF(US_open = player_id, 1, 0) + 
            IF(Au_open = player_id, 1, 0)) AS grand_slams_count
    FROM Championships c, Players p
    GROUP BY p.player_id
    HAVING grand_slams_count > 0
)
SELECT player_id, player_name, grand_slams_count
FROM AggTable
JOIN Players USING(player_id)

Let us consider a call log table Calls as follows:

+--------------+----------+
| Column Name  | Type     |
+--------------+----------+
| caller_id    | int      |
| recipient_id | int      |
| call_time    | datetime |
+--------------+----------+

Find the id of the users whose first and last calls on any day were with the same person.

It is important to note that just like the Friends table, any given call log row $A \rightarrow B$ implies a call happened between A and B. Hence first we duplicate the table flipping the caller_id and recipient_id. We then compute the first call and the last call for every user for every day, and then check if the recipients match for at least any day.

WITH Connections AS (
    SELECT caller_id AS user_id, recipient_id, call_time
    FROM Calls
    UNION
    SELECT recipient_id AS user_id, caller_id AS recipient_id, call_time
    FROM Calls
),
FirstCall AS (
    SELECT user_id, CAST(call_time AS DATE) AS day, recipient_id
    FROM (
        SELECT *,
            ROW_NUMBER() OVER(PARTITION BY user_id, CAST(call_time AS DATE) ORDER BY call_time DESC) AS first_call
        FROM Connections
    ) AS k
    WHERE k.first_call = 1
), LastCall AS (
    SELECT user_id, CAST(call_time AS DATE) AS day, recipient_id
    FROM (
        SELECT *,
            ROW_NUMBER() OVER(PARTITION BY user_id, CAST(call_time AS DATE) ORDER BY call_time) AS last_call
        FROM Connections
    ) AS k
    WHERE k.last_call = 1
)

SELECT DISTINCT(f.user_id)
FROM FirstCall AS f
JOIN LastCall AS l
    ON f.user_id = l.user_id
    AND f.day = l.day
    AND f.recipient_id = l.recipient_id;

ML Design

Sun, 10 Mar 2024 00:00:00 +0000

A ML design task can broadly be classified into the following components, in order of priority:

Defining success metrics
Data pipelines
Backend model design
Online model design
Model evaluation
Model monitoring
Serving

Defining success metrics

The most important aspect when building a ML system for a specific task is to be able to identify the success criteria. For example, for a recommendation system, this could be user click-rate or engagement, while for a predictive system, this could be as simple as mean abosolute error. Another thing to keep in mind is the logistical requirements of the model, like throughput, latency, scalability and availability.

Data pipelines

These essentially define how data for training or inference is flowing in, how it is ingested, what pipelines are setup for ETL. At this stage, we also need to think about the refresh rate, how the data is stored, what’s an optimal way to fetch the data, etc. For example for training a recommendation system, one might need past click data, and some useful features might be user specific metadata like gender, ethnicity, occupation, income, age group, etc and environment specific attributes like time, location, history of usage, platform, etc; each of which can be stored separtely and have their associated pipelines for data cleaning and normalization. Note that not all these features would be available in the raw data and trade-offs would have to be made to either collect these attributes or abandon them altogether.

If data is collected in batches and refreshed with some cadence one needs to also think about data versioning.
Another really important point to think about is ethical issues with respect to data anonymity and privacy.

For the usage history for a given users we can an array of 100 most recently watched/viewed/played items. Each item can itself be an embedding into a high-dimensional space. Each item can also be weighted depending on the delta between the user’s engagement with the item as compared to the average user engagement. For example, if a user watches a certain video and his watch fraction is 95%; while an average user’s watch fraction is only 50%, then there is a high signal that the user like such and such kind of videos and hence this particular array item should be given a higher weight.

Backend model design

If the task at hand involves user engagement and personalization, then the ML task can be broken into two component - a backend model component that operates at a lower refresh rate (high latency) and an online model component that needs to have a small latency. This latency requirements impose the model architechtures that can be used. For example, for online task, a small light-weight model would be the right choice. The backend model usually will process data in batch, i.e. for thousands of users at a time.

It is always useful to have a simple model that can be treated as a baseline against which to measure any other complex model.
One needs to also think of developing some kind of model registry to keep track of model requirements and versioning.

Online model design

The online model only needs to process a single item batch_size=1 at a time. At the start of each session the current state is simply the output of the last run of the backend model $A^{[l-1]}$. Each interaction can be thought of as a new datapoint token $x^{[l]}$. The job of the online model is to then compute the new state given the previous state and the new data point in some fashion. In the simplest case, this could just be a linear model: $A^{[l]} = W (A^{[l-1]}, x^{[l]})$. The interaction data-point is backed up into a database or an object store of some kind to be able to provide users history of their interactions.

Model evaluation

Segmenting the metrics based on certain features like country, age group, new vs old users is required to make sure that the model is performing on-par across user types.

Serving

Model monitoring

Model deteriorate over time, either due to covariate shifts or biases. It is therefore important to track relevant monitoring metrics. This allows us to identify when a if either a re-training needs to be done or if the model architechture as a whole needs to be updated.

A useful approach to handle catastrophic model quality deterioration is to have a separate benchmark model that too can be continuously trained. Once the model performance decreases that a certain threshold, one can quickly switch out to the benchmark model.

IMDB movie reviews

Fri, 08 Mar 2024 00:00:00 +0000

The goal of this notebook is to understand some details about Huggingface’s dataset and transformer libraries and also as a reference point for fine-tuning a LLM model for a classification task.

!pip install -q transformers datasets

[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m510.5/510.5 kB[0m [31m7.7 MB/s[0m eta [36m0:00:00[0m
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m116.3/116.3 kB[0m [31m8.8 MB/s[0m eta [36m0:00:00[0m
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m134.8/134.8 kB[0m [31m13.1 MB/s[0m eta [36m0:00:00[0m
[?25h

# For NN development, we will be using Pytorch-Lightning package
!pip install lightning --quiet

[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m2.1/2.1 MB[0m [31m7.9 MB/s[0m eta [36m0:00:00[0m
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m840.4/840.4 kB[0m [31m8.7 MB/s[0m eta [36m0:00:00[0m
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m800.9/800.9 kB[0m [31m9.8 MB/s[0m eta [36m0:00:00[0m
[?25h

import numpy as np
import torch
import lightning as L

from sklearn import metrics
from collections import Counter
from datasets import load_dataset, load_dataset_builder, get_dataset_split_names
from transformers import AutoModel, AutoTokenizer
from torch.utils.data import DataLoader
from torch import nn, optim
from torchmetrics.classification import BinaryAccuracy

# Setting up the device for GPU usage
device = 'cuda' if torch.cuda.is_available() else 'cpu'
print(device)

cuda

Data

# Inspect the dataset
ds_builder = load_dataset_builder("imdb")
print(ds_builder.info.description)
print(ds_builder.info.features)

/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: 
The secret `HF_TOKEN` does not exist in your Colab secrets.
To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.
You will be able to reuse this secret in all of your notebooks.
Please note that authentication is recommended but still optional to access public models or datasets.
  warnings.warn(

Downloading readme:   0%|          | 0.00/7.81k [00:00<?, ?B/s]

{'text': Value(dtype='string', id=None), 'label': ClassLabel(names=['neg', 'pos'], id=None)}

# Let's inspect the splits
get_dataset_split_names("imdb")

['train', 'test', 'unsupervised']

# Now let's load the train and the test dataset splits
dataset_train = load_dataset(path="imdb", split="train")
dataset_test = load_dataset(path="imdb", split="test")

Downloading data:   0%|          | 0.00/21.0M [00:00<?, ?B/s]

Downloading data:   0%|          | 0.00/20.5M [00:00<?, ?B/s]

Downloading data:   0%|          | 0.00/42.0M [00:00<?, ?B/s]

Generating train split:   0%|          | 0/25000 [00:00<?, ? examples/s]

Generating test split:   0%|          | 0/25000 [00:00<?, ? examples/s]

Generating unsupervised split:   0%|          | 0/50000 [00:00<?, ? examples/s]

# Let's now define some key variables
TRAIN_SAMPLE_SIZE = 1000
TEST_SAMPLE_SIZE = 1000
TRAIN_BATCH_SIZE = 8
TEST_BATCH_SIZE = 4
EPOCHS = 1
LEARNING_RATE = 1e-5
MAX_LEN = 200

# For the sake of this tutorial I will subsample to a size of 1000
train_indices = np.random.choice(len(dataset_train),
                                 size=TRAIN_SAMPLE_SIZE, replace=False)
data_train = dataset_train.select(train_indices)

test_indices = np.random.choice(len(dataset_test),
                                size=TEST_SAMPLE_SIZE, replace=False)
data_test = dataset_test.select(test_indices)

# Let's make sure we have good distribution of class labels
from collections import Counter

Counter(data_train['label'])

Counter({0: 513, 1: 487})

Toknenization

We can now start to tokenize the input data.

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

tokenizer_config.json:   0%|          | 0.00/48.0 [00:00<?, ?B/s]

config.json:   0%|          | 0.00/570 [00:00<?, ?B/s]

vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]

tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]

# We apply the BERT tokenizer with a max_len of 200 and padding or truncating
# text if needed. Since each `review` is a single entity, we don't require the
# token_type_ids which should all be just 0's.
def tokenization(datarow):
  return tokenizer(datarow["text"], max_length=MAX_LEN,
                   return_token_type_ids=False, padding='max_length',
                   truncation=True)

# As per this (https://huggingface.co/docs/datasets/en/use_dataset),
# this is a good way to apply tokenization to the entire dataset.
data_train = data_train.map(tokenization, batched=True)
data_test = data_test.map(tokenization, batched=True)

Map:   0%|          | 0/1000 [00:00<?, ? examples/s]

Map:   0%|          | 0/1000 [00:00<?, ? examples/s]

The dataset returns lists in the output. However, we would like to use it in Pytorch and want it to return tensors instead. To do this, we will set its format as described here

data_train = data_train.with_format("torch", device=device)
data_test = data_test.with_format("torch", device=device)

# Remove unwanted column 'text' since we have already tokenized it.
data_train = data_train.remove_columns(column_names=["text"])
data_test = data_test.remove_columns(column_names=["text"])

We can pass these Huggingface dataset directly to the Torch Dataloaders.

train_params = {
    "batch_size": TRAIN_BATCH_SIZE,
    "shuffle": True,
}

test_params = {
    "batch_size": TEST_BATCH_SIZE,
    "shuffle": False
}

train_dataloader = DataLoader(data_train, **train_params)
test_dataloader = DataLoader(data_test, **test_params)

Model

class FineTuneBert(L.LightningModule):
    def __init__(self, bert_backbone) -> None:
        super().__init__()
        self.bert_backbone = bert_backbone
        self.dropout = nn.Dropout(p=0.3)

        # In the loss function, there are multiple ways we can go:
        # 1. Use CrossEntropyLoss, in which case the out_features = 2
        # 2. Use BCELoss, in which case the out_features = 1, but we need an additional
        #    sigmoid layer at the end.
        # 3. BCEWithLogitsLoss, in which case the out_features = 1, but we do not need
        #    an additional sigmoid layer - WE WILL BE USING THIS.
        self.fc1 = nn.Linear(in_features=768, out_features=1)

    def forward(self, input_ids, attention_mask):
        # pass the inputs through the backbone
        # The output of BERT is last_hidden_state and pooler_output. Here we are
        # concerned about the latter.
        _, bert_output = self.bert_backbone(input_ids=input_ids, attention_mask=attention_mask, return_dict=False)
        output = self.dropout(bert_output)
        output = self.fc1(output)
        output = output.view(-1)
        return output

    def configure_optimizers(self):
        optimizer = optim.AdamW(params=self.parameters(), lr=LEARNING_RATE)
        return optimizer

    def training_step(self, batch, batch_idx):
        label, input_ids, attention_mask = (
            batch['label'],
            batch['input_ids'],
            batch['attention_mask']
        )

        preds = self.forward(input_ids=input_ids, attention_mask=attention_mask)
        loss = nn.functional.binary_cross_entropy_with_logits(preds, label.float())
        self.log('train_loss', loss, prog_bar=True)
        return loss

    def test_step(self, batch, batch_idx):
        label, input_ids, attention_mask = (
            batch['label'],
            batch['input_ids'],
            batch['attention_mask'],
        )
        preds = self(input_ids, attention_mask)
        test_loss = nn.functional.binary_cross_entropy_with_logits(preds, label.float())
        self.log("test_loss", test_loss, prog_bar=True)

        # Compute accuracy
        acc_metric = BinaryAccuracy().to(device)
        self.log("test_acc", acc_metric(preds, label), on_epoch=True)

bert = AutoModel.from_pretrained("bert-base-uncased")

model = FineTuneBert(bert_backbone=bert)
model.to(device)

FineTuneBert(
  (bert_backbone): BertModel(
    (embeddings): BertEmbeddings(
      (word_embeddings): Embedding(30522, 768, padding_idx=0)
      (position_embeddings): Embedding(512, 768)
      (token_type_embeddings): Embedding(2, 768)
      (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
      (dropout): Dropout(p=0.1, inplace=False)
    )
    (encoder): BertEncoder(
      (layer): ModuleList(
        (0-11): 12 x BertLayer(
          (attention): BertAttention(
            (self): BertSelfAttention(
              (query): Linear(in_features=768, out_features=768, bias=True)
              (key): Linear(in_features=768, out_features=768, bias=True)
              (value): Linear(in_features=768, out_features=768, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
            (output): BertSelfOutput(
              (dense): Linear(in_features=768, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (intermediate): BertIntermediate(
            (dense): Linear(in_features=768, out_features=3072, bias=True)
            (intermediate_act_fn): GELUActivation()
          )
          (output): BertOutput(
            (dense): Linear(in_features=3072, out_features=768, bias=True)
            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
            (dropout): Dropout(p=0.1, inplace=False)
          )
        )
      )
    )
    (pooler): BertPooler(
      (dense): Linear(in_features=768, out_features=768, bias=True)
      (activation): Tanh()
    )
  )
  (dropout): Dropout(p=0.3, inplace=False)
  (fc1): Linear(in_features=768, out_features=1, bias=True)
)

trainer = L.Trainer(max_epochs=5)
trainer.fit(model=model, train_dataloaders=train_dataloader)

INFO: GPU available: True (cuda), used: True
INFO:lightning.pytorch.utilities.rank_zero:GPU available: True (cuda), used: True
INFO: TPU available: False, using: 0 TPU cores
INFO:lightning.pytorch.utilities.rank_zero:TPU available: False, using: 0 TPU cores
INFO: IPU available: False, using: 0 IPUs
INFO:lightning.pytorch.utilities.rank_zero:IPU available: False, using: 0 IPUs
INFO: HPU available: False, using: 0 HPUs
INFO:lightning.pytorch.utilities.rank_zero:HPU available: False, using: 0 HPUs
INFO: LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
INFO:lightning.pytorch.accelerators.cuda:LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
INFO: 
  | Name          | Type      | Params
--------------------------------------------
0 | bert_backbone | BertModel | 109 M 
1 | dropout       | Dropout   | 0     
2 | fc1           | Linear    | 769   
--------------------------------------------
109 M     Trainable params
0         Non-trainable params
109 M     Total params
437.932   Total estimated model params size (MB)
INFO:lightning.pytorch.callbacks.model_summary:
  | Name          | Type      | Params
--------------------------------------------
0 | bert_backbone | BertModel | 109 M 
1 | dropout       | Dropout   | 0     
2 | fc1           | Linear    | 769   
--------------------------------------------
109 M     Trainable params
0         Non-trainable params
109 M     Total params
437.932   Total estimated model params size (MB)



Training: |          | 0/? [00:00<?, ?it/s]


INFO: `Trainer.fit` stopped: `max_epochs=5` reached.
INFO:lightning.pytorch.utilities.rank_zero:`Trainer.fit` stopped: `max_epochs=5` reached.

trainer.test(model=model, dataloaders=test_dataloader)

INFO: LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
INFO:lightning.pytorch.accelerators.cuda:LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

Testing: |          | 0/? [00:00<?, ?it/s]

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃        Test metric        ┃       DataLoader 0        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│         test_acc          │    0.8849999904632568     │
│         test_loss         │    0.39432957768440247    │
└───────────────────────────┴───────────────────────────┘

[{'test_loss': 0.39432957768440247, 'test_acc': 0.8849999904632568}]

Arrays

Fri, 08 Mar 2024 00:00:00 +0000

One of the most basic things that one can do with an array is to have a pointer or a set of pointers that traverse the array in some fashion. The number of pointers to book-keep depends on the question at hand. Let’s look at few sample problems to understand this concept:

Given an array nums compute the largest element.

To find the largest element, we have to at-least visit each element in the array. Thus we need a single pointer to traverse the array. We will reserve a constant memory space that holds the value of the largest element or at least the index of the position that holds the largest element.

def get_largest(nums: List[int]) -> int:
    largest_num = float('-inf')
    for num in nums:
        largest_num = max(largest_num, num)

    return -1 if largest_num == float('-inf') else largest_num

Given an array nums and a target check if target exists in the array.

Since we are not given any extra information as to the structure of the array, e.g. sorted or not, we should write a solution for the generic case. Similar to the previous case, we need a pointer that scans through the array and at each step check for equality of the element with the target. This gives us $O(n)$ time complexity and $O(1)$ space complexity algorithm as follows:

def find_target(nums: List[int], target: int) -> bool:
    for num in nums:
        if num == target:
            return True
    return False

Here is a slightly complicated problem, where we have a single traversal pointer: given a linked list, reverse it.

def rev_linked_list(head):
    new_head = None
    while head is not None:
        next_in_line = head.next
        head.next, new_head = new_head, head
        head = next_in_line
    return new_head

Given a sorted array nums and a target check if there exists $(i, j)$ such that $nums[i] + nums[j] = target$ and $i \ne j$.

Since we need to find two elements from the array that sum to the target, we should probably use two pointers. But the questions is where should they be placed initially. Ideally we would like to, on each iteration, update one or both the pointers and the updates should be such that we march towards the breaking or end-of-iteration criteria. In this case, we should place one pointer at the beginning and the other at the end. We will call them left and right respectively. If the sum of the elements at these positions is greater than the target, then the only way to bring down the sum is to decrease the right pointer. Conversly, if the sum is smaller than the target, the only way to increase the sum is to update the left pointer. Note that the we can make the above two statements only in this specific case of sorted array. If they sum to the target, we have found the pair and simply exit the iteration.

def two_sum_sorted(nums: List[int], target: int) -> bool:
    left, right = 0, len(nums) - 1
    while left < right:
        curr_sum = nums[left] + nums[right]
        if curr_sum == target:
            return True
        elif curr_sum > target:
            right -= 1
        else:
            left += 1
    return False

You are given an integer array nums of length n; such that each element represent a height of a column placed at that index, with the base resting on the $x-axis$. Find two lines that together with the x-axis form a container, such that the container contains the most water and return the maximum amount of water a container can store.

Since we need to create a container, we need two columns and hence two pointers, the left and the right column of the container places at indices $i$ and $j$, such that $i < j$. Now the question is what should be the initial placement of these pointers.

We actually only have 2 sensible choices:

One after other $i=0, j=1$
Opposite ends $i=0, j=len(nums) - 1$.

At any given configuration $(i, j)$, the volume of the water contained is: $V = width \times height = (j - i) * min(nums[i], nums[j])$.

If we place the pointers at opposite ends, then in every iteration we will either increment $i$ or decrement $j$, i.e. the width will always decrease. Hence the only hope of finding the container with more water is to increase the height. Since the height of the water volume is controlled by the smaller of the two columns, the pointer of the column which is smallest should be updated.

If instead we had chosen the one-after-other configuration, then in each iteration, the width will either decrease or increase and we won’t be able to make a decision as to which pointer to update as we did above.

def most_water_container(heights: List[int]) -> int:
    max_water_content = float('-inf')

    left, right = 0, len(heights) - 1
    while left < right:
        # Water content = width * (bar with the lower height of the two)
        water_content = (right - left) * min(heights[left], heights[right])
        max_water_content = max(max_water_content, water_content)

        # Which pointer to update? We will be making the width smaller with
        # the next pointer update, so the only hope for increasing the water content
        # is to increse the minimum height. Hence we should update the pointer
        # of the bar with the lower height of the two.
        if heights[left] < heights[right]:
            left += 1
        else:
            right -= 1

    return max_water_content

Kadane’s Algorithm

Let’s say we are given an array of integers; the goal is to find a subarray with the maximum sum.

To do this we find the subarray that starts at any given index, but ends on the index being considered; and we do this for every index. Let’s say that we have a such a subarray that ends on index $i-1$; shown in green. Now what would be the max subarray that ends on index $i$. Well we have two options: either we extend the previous subarray (blue) or we go solo i.e. start anew (red). Once we have the max subarrays ending at each index, the result will be the one that has the maximum value.

def max_sum_subarray(nums):
    curr_sum, max_sum = nums[0], nums[0]
    for i in range(1, len(nums)):
        curr_sum = max(curr_sum + nums[i], nums[i])
        max_sum = max(max_sum, curr_sum)
    return max_sum

The above algorithm will give the max subarray sum. However, if we also wanted the subarray [left index, right index] that had that sum, we have to modify the algorithm a little bit. Basically we keep track of the ending index of the max subarray. Then we traverse the array in the backward direction starting at the ending index, until we find the element that started the subarray.

def max_sum_subarray(nums):
    ending_idx = 0
    curr_sum, max_sum = nums[0], nums[0]
    for i in range(1, len(nums)):
        curr_sum = max(curr_sum + nums[i], nums[i])

        if curr_sum > max_sum:
            ending_idx = i
            max_sum = curr_sum

    # traverse back to find the start index
    last_sum, start_idx = max_sum, ending_idx
    while last_sum != nums[start_idx]:
        last_sum -= nums[start_idx]
        start_idx -= 1
    return start_idx, ending_idx, max_sum

Combinatorics, Tail sum for expectation and Simplices

Sun, 15 Dec 2019 17:23:00 +0000

1. How many different ways are possible to have $k-tuple$ with non-negative integer values such that they sum to a given value $n$?

$\to $To arrive at the solution, one can use the $stars-and-bars$ method. Let’s draw a sequence of n stars. There are $n-1$ spaces between the stars. What we need to do now to create $k$ numbers, is to place $k-1$ bars in those spaces. One such possibility for $n=6$ and $k=3$ is as follows:

\[\star\ | \star\ \star\ \star\ | \star\ \star \quad \quad \to 3-tuple\ (1, 3, 2)\]

Thus we need to choose $k-1$ locations out of the $n-1$ spaces available. The number of such combinations possible is thus $\binom{n-1}{k-1}$.

2. What if we allowed for the values to be 0? How many combinations are possible then?

$\to$ In the above case when all is done we placed $n+k-1$ objects(stars + bars). We were restricted by the fact that the values had to be non-negative. So we had to place the bars in the gaps between stars. However, now we do not have any such restriction. We can first choose $k-1$ slots out of $n+k-1$ slots available to place the bars. Once that is done, we just place the $n$ stars in the $n$ slots remaining. One such possibility for $n=6$ and $k=3$ is as follows:

\[\underset{\_}{|}\ \underset{\_}{\star}\ \underset{\_}{\star}\ \underset{\_}{\star}\ \underset{\_}{|}\ \underset{\_}{\star}\ \underset{\_}{\star}\ \underset{\_}{\star} \quad \quad \to 3-tuple\ (0, 3, 3)\]

Given this discussion, the number of combinations of $k-tuple$ possible is $\binom{n+k-1}{k-1} = \binom{n+k-1}{n}$. This number is also known as the multiset coefficient.

3. We roll a fair dice $n$ times. What is the probability that all faces have appeared?

Lets $\mathcal{I_i}$ be the number of times a face $i$ has appeared after the dice has been rolled $n$ times. What we need is a $6-tuple\ (\mathcal{I_1}, \mathcal{I_2}, \mathcal{I_3}, \mathcal{I_4}, \mathcal{I_5}, \mathcal{I_6})$ such that the elements sum to $n$, and none of the values $\mathcal{I_i}$ are 0. The probability that all faces have appeared is then the result given in Que. 1 divided by the results given in Que 2.

\begin{align} prob = \binom{n-1}{k-1} / \binom{n+k-1}{k-1} &= \frac{n!\ (n-1)!}{(n-k)!\ (n+k-1)!}\\ &= \frac{n!\ (n-1)!}{(n-6)!\ (n+5)!} \qquad \qquad k = 6 \end{align}

The tail sum for expectation formula for a non-negative integer random number is given as:

\[E[X] = \sum_{x=0}^\infty x\ P(X = x) = \sum_{x=0}^\infty P(X > x)\]

Proof: To show this, one can use an interesting identity for any non-negative integer given by:

\[x = \sum_{k=0}^\infty \mathcal{I}(x > k),\]

where $\mathcal{I}(condition)$ is an indicator function that evaluates to $1$ if condition is true, else 0. The well known formula for expectation can then becomes:

\[E[X] = \sum_{x=0}^\infty x\ P(X = x) = \sum_{x=0}^\infty \sum_{k=0}^\infty \mathcal{I}(x > k)\ P(X = x).\]

Switching the order of summation, we get the required result:

\[E[X] = \sum_{k=0}^\infty \sum_{x=0}^\infty \mathcal{I}(x > k)\ P(X = x) = \sum_{k=0}^\infty P(X > k).\]

4. We draw a random number from Uniform distribution $\mathcal{U}[0, 1]$ and keep drawing till the sum of the draws is greater than or equal 1. On average how many samples would we need to draw?

If we independently draw $d$ times from Uniform distribution $\mathcal{U}[0, 1]$, the state-space of all possibilities correspond to the region $S$ in the following diagram (shown for $d=2$ and $d=3$):

Each point in the region has equal probability. The state-space where the sum of the two draw is less than $1$ is then given by the region $R$. The probability that the sum of the samples drawn is less than $1$ is then the volume of $R$ divided by the volume of $S$.

The region $S$ is nothing but a hypercube in $d$-dimensions where $d$ is the number of draws. For $d=2$, it is a square, for $d=3$ it is a cube and so on. Since each side of this hypercube has length $1$, the volume of $S$ is trivially $1$.

The region $R$ in $d$-dimensions is nothing but a standard $d-simplex$.

The volume of such a simplex can be shown to be $\frac{1}{d!}$.

We can easily verify this for a couple of dimensions.

\begin{align} Vol(2-simplex) &= \int_{x=0}^1 \int_{y=0}^{1-x} dx\ dy = 1/2! \\ Vol(3-simplex) &= \int_{x=0}^1 \int_{y=0}^{1-x} \int_{z=0}^{1-x-y} dx\ dy\ dz = 1/3! \end{align}

Let us now define the random variable whose expectation we require as $X$. In other words, $X=x$ means that we require $x$ draws from the distribution such that the cumulative sum of the draws is greater than or equal to $1$.

\[E[X] = \sum_{x=0}^\infty x\ P(X = x)\]

Using the tail sum for expectation, we can write:

\[E[X] = \sum_{x=0}^\infty P(X > x)\]

$P(X > x)$ means the probability that one requires more than $x$ draws to reach a sum greater than $1$. This can also be thought as the probability that in $x$ rolls, one obtained a sum less than $1$. Interestingly, this value is nothing but the volume of region $R$. Thus we have

\[E[X] = \sum_{x=0}^\infty P(X > x) = \sum_{x=0}^\infty \frac{1}{x!} = e \quad \quad (surprise!!!)\]

Thus, the average number of draws required till the cumulative sum of the draws is greater than or equal to $1$ is $e$.

Absorbing Markov Chains

Fri, 13 Dec 2019 17:23:00 +0000

A Markov chain containing absorbing states is known as an absorbing Markov chain. So what is an absorbing state. In simple words, if you end up on an absorbing state you can’t go anywhere else; you are stuck there for all eternity. In other words, the probability of transition from an absorbing state $i$ to any other non-absorbing state, also called transient states, is 0.

At this point, we can make an observation:

0. The probability of eventually begin absorbed into any of the absorbing states given that one starts from anywhere is 1.

Let’s say we have $n$ transient states and $m$ absorbing states in the state space of any system. The transition matrix for such a system can be written as:

$$T = \left[ \begin{matrix} Q & R \\ 0 & I \end{matrix} \right],$$

where $Q$ of shape ($n \times n$) gives the probability of transitioning from transient state $i$ to transient state $j$. The matrix $R$ of shape $n \times m$ gives the probability of transitioning from transient state $i$ to absorbing state $j$, and $I$ of identity matrix of shape $m$.

We can compute the respective probability weights of the various states given a current probability weights by applying the transition matrix following the Markov property:

\[v_{k+1} = T\ v_{k}\]

Recursively expanding the above equation to the starting state $v_0$, we have

\[v_{k} = T^k\ v_{0}\]

It is interesting to compute what $T^k$ is:

$$ \begin{align} T^2 &= \begin{bmatrix} Q & R \\ 0 & I \end{bmatrix} \begin{bmatrix} Q & R \\ 0 & I \end{bmatrix} = \begin{bmatrix} Q^2 & QR + RI \\ 0 & I \end{bmatrix} \\ T^3 &= \begin{bmatrix} Q & R \\ 0 & I \end{bmatrix} \begin{bmatrix} Q^2 & QR + RI \\ 0 & I \end{bmatrix} = \begin{bmatrix} Q^3 & Q^2R + QR + RI \\ 0 & I \end{bmatrix} \end{align} $$

Given the pattern we can write

$$T^k = \begin{bmatrix} Q^k & R(I + Q + Q^2 + ... + Q^k) \\ 0 & I \end{bmatrix}$$

Using the summation formula for a geometric sequence we can write:

$$ T^k = \begin{bmatrix} Q^k & R \left(\frac{I - Q^k}{I - Q}\right) \\ 0 & I \end{bmatrix} $$

From the above formula, we note that:

1. the probability the system is in transient state $j$ given that it started in transient state $i$ after $k$ steps is given by the $(i, j)^{th}$ entry of the matrix $Q^k$

The entries of $Q$ are all smaller than 1, given that these are probabilities. Hence we have $\lim_{k \to \infty} Q^k = 0$. This allows us to write

$$T^\infty = \begin{bmatrix} 0 & R \left(\frac{1}{I - Q}\right) \\ 0 & I \end{bmatrix} = \begin{bmatrix} 0 & NR \\ 0 & I \end{bmatrix},$$

where $N = [I - Q]^{-1}$ is known as the fundamental matrix. The $0’s$ in the first column of the above matrix proves statement given in blockquote 0.

From the above formula, we note that:

2. the probability the system ends up in absorbing state $j$ given that it started in transient state $i$ is the $(i, j)^{th}$ entry of the matrix $NR$.

The fundamental matrix $N$ has a very interesting interpretation:

3. its $(i, j)^{th}$ entry corresponds to the expected number of times the system visits transient state $j$ given that it started in transient state $i$, before being absorbed.

To see this, let us use a random indicator variable $\mathcal{I}_k$ which is 1 if the system is in state $j$ during $k^{th}$ step, else 0. The expected value of this indicator variable is:

\[E[\mathcal{I}_k] = 1 \times prob(\mathcal{I}_k = 1) + 0 \times prob(\mathcal{I}_k = 1) = Q^k_{i, j} \quad ......... \text{refer blockquote 1}\]

The total number of visits we make to state $j$ in $n$ steps of the Markov chain is simply summing up all these indicator variables (one for each step).

\[\text{total_visits after $n$ steps} = \mathcal{I}_0 + \mathcal{I}_1 + \mathcal{I}_2 + .....+\ \mathcal{I}_n\]

Taking the expectation we get

$$ \begin{align} E[\text{total_visits after $n$ steps}] &= E[\mathcal{I}_0] + E[\mathcal{I}_1] + E[\mathcal{I}_2] + .....+\ E[\mathcal{I}_n] \\ &= Q^0_{i, j} +\ Q^1_{i, j} +\ ...+\ Q^n_{i, j} \end{align} $$

Taking $n \to \infty$, we have $E[\text{total_visits}] = Q^0_{i, j} +\ Q^1_{i, j} +\ ...+\ Q^\infty_{i, j} = [I - Q]^{-1}_{i, j} = N_{i, j}$

Summing this over all the possible transient states, we get the expected number of steps the Markov chain runs for, starting at state $i$, before getting absorbed into one of the absorbing states. This is nothing but adding the entries of the $i^{th}$ row of $N_{i, j}$:

\[t_i = \sum_j N_{i, j}\]

Q1. On average, how many times do we have to roll a fair dice before seeing two 6’s in a row?

Solution: One can represent the state space as $S = [6, E, 66]$, with $66$ being the absorbing state.

where $E = \{\phi, 1, 2, 3, 4, 5\}$. $\phi$ represents the null state $\to$ the state before we even began rolling the dice.

The transition matrix can be written as:

$$T = \begin{bmatrix} 0 & 5/6 & 1/6 \\ 1/6 & 5/6 & 0 \\ 0 & 0 & 1\end{bmatrix},$$

where the labels corresponds to the index as given in $S$. The fundamental matrix $N$ is given by:

$$ \begin{align} N = [I - Q]^{-1} &= \left(\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} - \begin{bmatrix} 0 & 5/6 \\ 1/6 & 5/6 \end{bmatrix}\right)^{-1}\\ &= \begin{bmatrix} 6 & 30 \\ 6 & 36 \end{bmatrix} \end{align} $$

Since we begin in state $\phi$ which is at index $1$ (assuming 0 indexing), the average number of dice rolls before seeing two 6’s in a row is

\[t_1 = N_{1,0} + N_{1, 1} = 42\]

Q2. You keep on tossing a fair coin until you see HHT or THT. Which of these combination is more likely to occur?

The trick in most of such problems is to correctly identify the relevant state space. For this problem, the state space that we can use is $S = [\phi, HH, HT, TH, TT, HHT, THT]$, where $HHT$ and $THT$ are the absorbing state. The transition diagram then becomes:

The red arrows indicates rolling a heads on the next roll, while blue arrows indicate rolling a tails on the next turn. Given this, the transition matrix becomes:

where the labels corresponds to the index as given in $S$. The fundamental matrix then becomes:

$$N = [I - Q]^{-1} = \begin{bmatrix} 1 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 0 & 1\\ \end{bmatrix}, $$

where the $Q$ matrix is shown in green box. Following blockquote 2. the required matrix is given by:

$$P = NR = \begin{bmatrix} 5/8 & 3/8 \\ 1 & 0 \\ 1/2 & 1/2 \\ 1/2 & 1/2 \\1/2 & 1/2 \end{bmatrix},$$

where $R$ is the matrix shown in blue. Thus the probability of starting from $\phi$ and ending up in the $HHT$ state is $5/8$, while ending up in the $THT$ state is $3/8$.

Q3. On average, how many times must a 6-sided die be rolled until all sides appear at least once? What about for an n-sided die?

Let $n_i$ be the number of times face $i$ has shown up since we started rolling the dice. We now define an indicator variable $\mathcal{I}_i = \mathcal{I} (n_i > 0)$ and a random variable $X$ that is the sum of these indicator variables $\mathcal{I}_1$ to $\mathcal{I}_6$. The possible values of $X$ serves as the states of the system, i.e. $S = [1, 2, 3, 4, 5, 6]$.

The state $X=6$ can then be identified as the absorbing state, which happens when we see all the faces of the dice. The state diagram is given as:

The transition matrix corresponding to this can be written as:

$$T = \begin{bmatrix} 1/6 & 5/6 & 0 & 0 & 0 &0 \\ 0 & 2/6 & 4/6 & 0 & 0 &0\\ 0 & 0 & 3/6 & 3/6 & 0 &0\\ 0 & 0 & 0 & 4/6 & 2/6 &0\\ 0 & 0 & 0 & 0 & 5/6 &1/6\\ 0 & 0 & 0 & 0 & 0 & 1 \end{bmatrix},$$

where the labels corresponds to the index as given in $S$. The fundamental matrix is given by:

$$N = [I - Q]^{-1} = \begin{bmatrix} 1.2 & 1.5 & 2 & 3 & 6 \\ 0 & 1.5 & 2 & 3 & 6\\ 0 & 0 & 2 & 3 & 6\\ 0 & 0 & 0 & 3 & 6\\ 0 & 0 & 0 & 0 & 6 \end{bmatrix}$$

Since we begin in state $1$ (after one dice roll) which is at index $0$, the average number of dice rolls before seeing all sides appear is:

\[t_{\phi} = 1 + \sum_{j} N_{0j} = 1 + (1.2 + 1.5 + 2 + 3 + 6) = 14.7\]

The $1$ in the above equation is needed since initially we start with no dice roll (state $\phi$). The above summation can be written in an interesting way:

\[t_{\phi} = 6\left(\frac{1}{6} + \frac{1}{5} + \frac{1}{4} + \frac{1}{3} + \frac{1}{2} + 1 \right)\]

The quantity in the bracket is nothing but the $6^{th}$ Harmonic number $H_6$. Hence generalizing to arbitrary sided dice, the expected number of rolls required before seeing all the faces of the $n-sided$ dice is:

\[t_n = n\ H_n\]

Logistic Regression

Tue, 12 Nov 2019 17:23:00 +0000

One of the most common test case in supervised machine learning is that of classification: Given a data point, classify it into one of the many labels available

The simplest case of this is binary classification. For e.g. classifying emails as spam or ham, classifying objects as star or galaxy, classifying images as a cat or a dog, etc. In all these cases, to be able to apply the mathematical modeling of machine learning one needs to abstract away some feature(s) of the object. Most often choosing or engineering the right features is what gives more power to predictive modeling rather than sophisticated models.

If your features are crappy, no amount of sophisticated models can come to the rescue.

However, in this post we are going to assume that we already have the features in hand. Given this, there are broadly two different ways of approaching classification problems - parametric and non-parametric. The topic we are going to discuss falls in the non-parametric approaches; more specifically as a linear model.

Logistic regression aims to creates a linear boundary separating objects of one class from the other. If the input features live in a $\mathcal{R}^d$ dimensional space, we want to create a $\mathcal{R}^{d-1}$ hyperplane optimally separating the two classes. The predicted class for any new point can then be inferred depending on which side of the hyperplane it lies. We will refer to this hyperplane as boundry.

\[y = sign(w^T x + b), \qquad y \in \{0, 1\}.\]

We will take the class with label 1 as the positive class.

Conceptually, one can also think of the boundry as a set of points in the feature space whose probability of belonging to class $y=1$ is half. As we move away from the boundry, on one side the probability of belonging to $y=1$ diminishes, while on the other end it approaches 1.

$\rightarrow$ Mathematically, what we want is a specific function of the perpendicular distance of any point from the boundary given by $z= w^Tx + b$, whose value on the boundary be 0.5 and should range from 0 to 1 as we move from one side of the boundary to the other.

There are many such functions that we could choose as shown below:

The functional form used by Logistic regression is the sigmoid function:

\[P(y=1|x) = \hat{y} = \sigma (z) = \frac{1}{1 + e^{-z}}\]

One common terminology used in this regards is that of Odds. It is defined as:

\[\mathrm{Odds} = \frac{P(y = 1)}{P(y = 0)} = \frac{\hat{y}}{1 - \hat{y}}\]

The domain of Odds lies in the range $[0, \infty)$. The natural logarithm of the above quantity is called Log Odds which lies in the range $(-\infty, \infty)$. This can be shown to be

\[\mathrm{Log Odds} = w^T x + b\]

Thus we can interpret coefficient corresponding to a given predicting variable as: the change in log odds for a unit change in a given variable keeping the rest variables fixed.

$\rightarrow$ LogOdds = perpendicular distance; and the value of LogOdds on the boundary is 0.

Our goal in now to estimate the value of the parameters of the model, $w$, from training data. However, to assess which values of the parameters are good, we have to first define the likelihood. Under the assumption that each data-point is independent of every other data-point, we have:

\[\mathcal{L} = \prod_{i=1}^n p(y_i | x_i, w) = \prod_{i=1}^n [\mathcal{I}(y_i=1)\ \hat{y_i}][\mathcal{I}(y_i=0) (1 - \hat{y_i})]\]

Remember, $\hat{y_i}$ gives the probability of a datapoint $i$ belonging to class $y=1$. So if a data-point belongs to $y=0$ we need to use $1 - \hat{y_i}$. Here $\mathcal{I}$ is an indicator variable to denote if a particular datapoint belongs to positive class or not. Writing in terms of log-likelihood and knowing that log of multiplication is equal to sum of the logs, this becomes:

\[\begin{align} \log\ \mathcal{L} &= \sum_{i=1}^n \left[\log[\mathcal{I}(y_i=1) \hat{y_i}] + \log [\mathcal{I}(y_i=0) (1 - \hat{y_i})] \right] \\ &= \sum_{i=1}^n \left[y_i \log \hat{y_i} + (1-y_i)\log (1 - \hat{y_i}) \right] \end{align}\]

The value of $w$ we are looking for them becomes:

\[w_{ML} = \underset{w}{\mathrm{argmax}} (\log\ \mathcal{L})\]

$\rightarrow$ One can also transform the likelihood to a cost function by simply taking the negative and reducing it via the mean, $C = -\frac{1}{n} \log \mathcal{L}$; hence the name negative-log-likelihood loss.

\[w_{ML} = \underset{w}{\mathrm{argmin}}\ C\]

However, there is no analytical solution for $w$ given the form of the cost function. That’s where gradient descent comes to the rescue. We first randomly initialize $w$ and iteratively update it till a convergence criterion is reached.

\[w_{t+1} = w_t - \eta\ \nabla_w C,\]

where $\eta$ is the learning rate.

The gradient of the cost function wrt $w$ can be obtained via chain rule:

\[\frac{dC}{dw} = -\frac{1}{n} \sum_{i=1}^n \frac{d \log \mathcal{L}_i}{dw} = -\frac{1}{n} \sum_{i=1}^n \left[ \frac{d \log \mathcal{L}_i}{d\hat{y_i}} \frac{d\hat{y_i}}{dz_i} \frac{dz_i}{dw}\right]\]

The individual components in the chain can be written as:

\[\frac{dz_i}{dw} = \frac{d (w^T x_i + b)}{dw} = x_i\] \[\frac{d\hat{y_i}}{dz_i} = \frac{e^{-z_i}}{(1 + e^{-z_i})^2} = \hat{y_i} (1 - \hat{y_i})\] \[\frac{d \log \mathcal{L}_i}{d\hat{y_i}} = \frac{y_i}{\hat{y_i}} - \frac{(1-y_i)}{(1 - \hat{y_i})}\]

Putting it all together, the gradient of the cost function wrt $w$ becomes:

\[\nabla_w\ C = -\frac{1}{n} \sum_{i=1}^n x_i [y_i (1 - \hat{y_i}) - (1 - y_i) \hat{y_i})] = -\frac{1}{n} \sum_i^n y_i x_i \left[\frac{1}{1 + e^{y_i w^T x_i}}\right].\]

Likewise, the gradient of the cost function wrt the bias term $b$ can be written as:

\[\nabla_b\ C = -\frac{1}{n} \sum_i^n y_i \left[\frac{1}{1 + e^{y_i w^T x_i}}\right]\]

More often that not, using the maximum likelihood prescription as above leads to estimates that have high variance. Especially in the case when one has fewer data-points than that is representative of the population, the classifier will be able to clearly separate the two classes with a linear boundary, with no misclassified points. In that case the value of $w$ will blow up. Points on either side of the boundary will be classified as belonging to their respective classes with probability of 1. This however may not generalize to new data-points and lead to a classifier with high variance. You basically read too much into your data, leading to overfitting.

This can be prevented using regularization. To keep the estimated $w$ from becoming too large, one can add a penalty term to the likelihood.

\[w_{reg} = \underset{w}{\mathrm{argmax}} \log\ \mathcal{L} - \lambda ||w||^2 \qquad \mathrm{Ridge}\] \[w_{reg} = \underset{w}{\mathrm{argmax}} \log\ \mathcal{L} - \lambda ||w||_1 \qquad \mathrm{Lasso}\]

Poisson Process

Sat, 28 Sep 2019 17:23:00 +0000

Let’s imagine rain falling. One obvious parameter describing this process is the rate - whether its drizzling or pouring! Let’s now focus on a tiny patch of land and assume that the rate is constant and will term this as $\lambda$. We can describe rain as a Poisson process.

Now what in the world is a Poisson process. If we think $\lambda$ as the number of raindrops falling on the patch per minute, and let’s say we wait for 5 minutes; how many raindrops will we see? Well, on average we would see $5\lambda$ drops. However, we might see more or we might see less. A Poisson process is one in which this count of drops is Poisson distributed.

This discrete probability distribution is given by:

\[P(k\ \mathrm{events\ in\ interval}\ t) = \frac{ (\lambda t)^k e^{-\lambda t}}{k!}\]

An interesting property about Poisson distribution is that its mean and variance are equal and they are simply $\lambda t$.

Now let’s say we were curious about counting the inter-arrival time between raindrops. Well if there are on average $\lambda$ drops falling per minute, the inter-arrival time on average should be $1/\lambda$. But what is the actual shape of the probability distribution of the inter-arrival times? In this case, the pdf is over a continuous domain and happens to be the exponential distribution. It has one free parameter and you guessed it, its the same $\lambda$ parameter that we have been using till now.

\[P(t|\lambda) = \lambda\ e^{-\lambda t}\ \forall\ t \ge 0\]

The distribution has mean $1/\lambda$ as we expected, and a variance of $1/\lambda^2$.

Here we can imaging a scenario. You decide to go inside to get your cup of tea. You now think how long will it take for you to see a raindrop on average once you go back outside. You calculate, on average the inter-arrival time is $1/\lambda = 6$ seconds. Given that you decide to go out randomly, it will take 3 seconds before you see the next raindrop. But is it?

Plotted are the arrival times(in seconds) of the first 20 drops that I simulated using $\lambda=10\ \mathrm{drops/min}$.

When you go out carrying your cup of tea which of the intervals(region between two drops) above are you more likely to hit. Obviously the regions with larger width. In other words, regions with larger width are oversampled. However these larger width regions also occurs less frequently (Refer fig. 2). So the probability distribution of encountering region $R$ of interval size $T$ is:

\[P(\mathrm{Region}\ R\ \mathrm{of\ size}\ T) = \lambda^2 T\ e^{-\lambda T}\]

On expectation, the region that we are most likely to encounter has width given by:

\[E[W] = \int_0^\infty T\ pdf(T)\ dT = \int_0^\infty T\ \lambda^2 T\ e^{-\lambda T} = \frac{2}{\lambda}\]

Since we are walking in randomly, the interval that we would observe is then half of the above value. But guess what this value is $1 / \lambda$, the expected arrival times between two raindrops, which in our case is 6 seconds and not 3 seconds are we thought before.

A take on this interesting phenomenon in case of bus arrival times, known as the waiting time paradox; which states:

When waiting for a bus that comes on average every 10 minutes, your average waiting time will be 10 minutes.

A nice discussion on this paradox can be found in this post by Jake Vanderplas. The waiting time paradox is a specific example of the inspection paradox.

Let’s continue on our journey of exploring distributions via raindrops. You now ask, what is the time interval between $n$ consecutive raindrops? There are $n-1$ intervals between $n$ consecutive raindrops, with each interval having a size $t$ given by the exponential distribution. So what we want is the random variable which is the sum of $k = n-1$ exponentially distributed random variables. The pdf of this random variable is the Gamma distribution:

\[P(x | k, \lambda) = \frac{\lambda^k\ x^{k - 1} e^{-\lambda x}}{\Gamma(k)}\]

Our exploration uncovers a key finding that the various distributions are interconnected. For more such relations refer to this link. What we discussed was Poisson process in the time domain, raindrops or photons hitting a telescope; one can also have Poisson processes in the spatial domain. Its easier to visualize this in 2D, and we can take an example of trees growing in a forest. Analogous to the time domain discussion above, we have a rate parameter $\lambda$ which is number of trees per unit area. If we take a square of area A, move it across the forest, each time counting the number of trees that fall in the square, the count is going to be a random variable which, the same as before, is going to be Poisson distributed:

\[P(k\ \mathrm{trees\ in\ area}\ A) = \frac{ (\lambda A)^k e^{-\lambda A}}{k!}\]

PDF of a dependent variable

Sun, 07 Apr 2019 17:18:23 +0000

``The Calculus required continuity, and continuity was supposed to require the infinitely little; but nobody could discover what the infinitely little might be."
-- Bertrand Russell in Mysticism and Logic and Other Essays, The Floating Press, 1 August 2010, p.100

Many-a-time when working with model fitting to data, one encounters a situation where one need to find the probability distribution of a random variable, which itself is a function of another random variable.

Given that $x$ has a probability distribution $pdf_X(x)$, and that the variable $y$ is related to $x$ as $y = f(x)$, what is the probability distribution function of $y$?

Naively, one might think for a given value $y’$, one can just pick up its probability as $f_X(x’)$, where $y’ = f(x’)$. This is correct when $x$ is a discrete random variable and we are talking about probability mass function. However, when $x$ is a continuous random variable, this approach gives wrong results. For the continuous case, the probability of a single point has no meaning. Rather, it is the probability of finding something in a given interval that is conserved across transformations. Making the interval become infinitesimal, we can approximate the area under the curve by a rectangle as shown in the figure. This area (mass) is the same when we transform to the variable $y$. In other words the following equality holds:

\[\mathrm{pdf}_X(x)\ dx = \mathrm{pdf}_Y(y)\ dy, \qquad ......dy = f'(x)\ dx\]

Let’s look at a few examples:

1. LogNormal Distribution: If $x$ is Gaussian distributed, then $y = e^x$ is said to have lognormal distribution. Let’s find its pdf using the above relation.

\[g_Y(y) = f_X(x) \frac{dx}{dy} = \frac{1}{\sqrt{2 \pi \sigma^2}} e^{-\displaystyle \frac{(x -\mu)^2}{2 \sigma^2}} \left(\frac{1}{y}\right) = \frac{1}{\sqrt{2 \pi \sigma^2}y} e^{-\displaystyle \frac{(\ln y -\mu)^2}{2 \sigma^2}}\]

2. Flux transmission: The flux field we observe from the absorption of Lyman-alpha photons by neutral Hydrogen in the intergalactic medium is related to the density field which is Gaussian in nature through the relation:

\[F(\mathbf{x}) = \exp \left[-A \exp \left(\beta \left[\delta_g(\mathbf{x}) - \frac{\sigma_g^2}{2} \right] \right) \right]\]

The dependence of flux on density is thus highly non-linear. However, at earlier times when the variance on $\delta$ was small, the flux was roughly Gaussian distributed too. To find the exact pdf, we start by noting that:

\[\frac{dF}{d\delta_g} = \beta\ F \ln F\]

Thus we have,

\[p(F) = -\frac{p(\delta_g)}{\beta\ F\ ln F},\]

where $\delta_g$ for a given $F$ can be calculated from the first equation. We need a negative sign cause $F$ is a monotonically decreasing function of $\delta_g$.