Fixed Point vs Floating Point: A Numerical Representation Deep Dive
A ground-up look at how floating-point and fixed-point numbers are represented and how arithmetic works for each
Algorithms
Newton-Gregory Interpolation
Interpolate equally-spaced data efficiently and discover its connection to Taylor series.
BKM
Compute logarithms and exponentials without a floating point unit.
CORDIC
Compute sine, cosine, and exponentials using only addition, subtraction, and bit shifts.
The AAA Algorithm
Fit rational functions to data with poles and discontinuities where polynomials fail.
Tanhsinh Quadrature
Tackle tricky integrals with endpoint singularities using a clever variable transformation.
Lagrange Interpolation
Approximate functions using polynomial interpolation without solving linear systems.
Optimization
Loss Spikes in Gradient Descent
Loss spikes aren’t noise. They’re gradient descent briefly exceeding the edge of stability and snapping back. Here’s why.
Robust Regression Without Gradients
L1 regression is more robust to outliers than least squares, but harder to solve. We walk through four algorithms, each addressing a limitation of the previo...
Golden Section Search for Robust Regression
Golden section search reuses objective evaluations to efficiently minimize 1D functions. Learn how this classical algorithm connects to the golden ratio and ...
Lagrange Duality
Derive and interpret the dual form of an optimization problem.
Quadratic Penalty Algo
Solve constrained optimization problems using your favorite unconstrained solver.
Nonlinear Least Squares
Fit nonlinear models using Gauss-Newton and Levenberg-Marquardt algorithms.
Visualization
Plotting Ellipses
Plot ellipses using conic, quadratic, and parametric representations.
3D Plotting using Non-Rectangular Grids
Create cleaner 3D surface plots using radial and elliptical grids in matplotlib.
Machine Learning
Numpy Regression Trees
Implement a regression tree from scratch using only numpy.
Statistics
Chatterjee’s Xi Coefficient
Detect nonlinear relationships that Pearson and Spearman miss.