The tutorials for the course Theoretical Foundations of Neural Networks are held on odd weeks on Tuesdays from 12:00 to 13:40 in room T-124, or as announced in lectures and tutorials.
study materials
The exercise materials are in the form of Jupyter notebooks using Python. For easy installation of Python, Jupyter, and the required packages, we will use uv . The first exercise is intended to familiarize you with this technology. To replicate the exact environment in which the notebooks were created, you can use the files pyproject.toml and uv.lock.| # | name | topics | assignment | solution |
|---|---|---|---|---|
| 1 | introduction to neural networks | basics of machine learning in Python, automatic differentiation, simple neural networks, their architecture, gradient descent, activation functions, hyperparameters and their selection | notebook | |
| 2 | optimization | optimization of neural networks, SGD, Momentum, Nesterov, Adagrad, Adadelta, Adam, AdamW | ||
| 3 | regularization | data splitting into training and test sets, overfitting, bias-variance trade-off, L1, L2 regularization, early stopping, dropout, batching, cross-validation, its variants and usage, double descent | ||
| 4 | recurrent networks and text models | recurrent networks, natural language processing, attention, transformers | ||
| 5 | graph neural networks | learning on graphs, recurrent neural networks on graphs, networks based on random walks, analogies with text processing, convolutions on graphs |