The manual presents modern optimization methods and many of their applications in machine learning problems.

First, the basic concepts of machine learning are presented: collection of objects, models, loss functions, etc.

Probabilistic and statistical interpretations of machine learning problems are given. The concept of optimal predictors is introduced. Algorithms for their existence and construction are discussed. The conditions for convergence, stability, and optimality of these algorithms are studied. Various estimates of the difference between empirical and real risks, expressed in the fundamental theorem of machine learning, are given.

Gradient descent methods and their modifications are described, especially stochastic and accelerated gradient methods, which are widely used in machine learning algorithms.

The manual is intended for undergraduate and graduate students studying computer science. This may be useful for researchers wishing to deepen their theoretical knowledge in the field of machine learning.

It is assumed that the reader is familiar with the basics of probability theory, linear algebra, mathematical analysis, optimization methods, and the theory of algorithms. 396