Basic Mathematical Aspects for Machine Learning
Fundamentals of Mathematical Analysis
- Functions and their properties.
- Limit of a function (basic concepts).
- Derivative of a function (+ its geometrical and mechanical meaning).
- The derivative of a complex function.
- Extremes of a function. Convexity of a function.
- Partial derivatives and the gradient.
- Gradient in optimization problems.
- The directional derivative.
- The tangent plane and linear approximation.
Basics of linear algebra
- Vector space.
- Linear independence.
- Norm and scalar product of vectors.
- Definition of a matrix. Operations on matrices.
- Rank and determinant of a matrix.
- Systems of linear equations.
- Types of matrices.
- Eigenvectors and eigenvalues.
- Matrix expansions (spectral, singular).
- Approximation with matrices of lesser rank.
- Singular expansion and low rank approximation.
Methods of optimization
- Optimization of nonsmooth functions (+ local minima problem).
- Simulated annealing method.
- Genetic algorithms. Differential evolution algorithm.
- Nelder-Mead method.
Probability Theory and Mathematical Statistics
- Definition of probability. Properties of probability.
- Conditional probabilities. Complete probability formula. Bayes formulas.
- Discrete random variables.
- Continuous random variables.
- Estimating distributions from a sample. Statistics.
- Characteristics of distributions.
- Important statistics (sample mean, median, mode, variance, interquartile range).
- The central limit theorem.
- Confidence intervals.