Isotonic Regression tries to fit a line/plane/hyperplane to a sequence of observations that lies as "close" as possible to the observations while maintaining monotonicity. This post takes an in-depth look at how the paper Generalized Isotonic Regression solves this problem in an arbitrary number of dimensions while supporting any convex differentiable loss function. First, Linear Programming is introduced, and then it is shown how the partitioning-based algorithm follows from the KKT conditions.
Kernel Density Estimation approximates a probability density function from collected samples. In contrast to a histogram, we do not need first to discretise the data by placing it into bins, instead building a continuous estimate. Applied to height measurements of adults, for example, we obtain a continuous density describing the likelihood of each possible height. This article will first introduce the concept of a Kernel as applied to smoothing data points before implementing it in the Julia programming language. Next, this implementation will be modified to calculate the Kernel Density Estimate of 1-dimensional data before exploring possibilities for determining an optimal bandwidth.