The book discusses the theoretical aspects of classification machine learning. At the center of the presentation is learning as the ability of the applied algorithms to provide empirical generalization. Issues of sampling complexity, accuracy and reliability of classifiers are directly related to learning. Much attention is paid to algorithmic methods for analyzing learning processes and synthesizing decisive rules, including the Kolmogorov approach associated with algorithmic information compression. Principles of selection of training models and families of classifying algorithms depending on statements and properties of solved problems are described. The book is intended for specialists in machine learning theory; it will be useful for graduate students, developers of intelligent software and senior students of mathematical specialties specializing in this field.