Introduction to Mathematics and Optimization

Niels Lauritzen

Start reading →
  1. The language of mathematics and prompting Studying with chatbots — then logic, sets, numbers, proofs and functions, ending with a first look at neural networks.
  2. Linear equations From one equation to Gauss elimination — then polynomials, interpolation, secret sharing and fitting data.
  3. Matrices Matrices as linear maps: products, inverses, transposes and positive definiteness.
  4. What is optimization? Optimization problems made precise: convexity, linear optimization and separating labeled data.
  5. Euclidean vector spaces The geometry of data: dot products, the perceptron, least squares, cosine similarity and attention — then limits and continuity.
  6. Convex functions Why convexity makes optimization easy: derivatives, Newton's method, Taylor polynomials and the convexity tests.
  7. Several variables Gradients, gradient descent and backpropagation — from logistic regression to a neural network that writes.
  8. The Hessian The second derivative in higher dimensions: Newton's method for critical points, the Hessian test and deciding definiteness.
  9. Convex optimization The synthesis: separating hyperplanes, support vector machines, kernels, interior point methods and the KKT conditions.