Linear Algebra & Gradients

Linear Algebra in AI

Models are just vast collections of weights stored in matrices. To optimize them, we calculate gradients—the slope of the loss function—which tell us how to nudge weights to reduce error.

import numpy as np

# Matrix multiplication for a neural layer
X = np.random.randn(4, 3)  # batch of 4 samples, 3 features
W = np.random.randn(3, 2)  # weight matrix
b = np.zeros(2)            # bias

output = X @ W + b  # forward pass
print(output.shape)  # (4, 2)

NumPy Vectorization

We use NumPy for high-performance array processing. Vectorized operations allow us to avoid slow Python loops, executing math in optimized C-code instead.