The tutorials for the course Theoretical Foundations of Neural Networks are held on odd weeks on Tuesdays (there are some exceptions, exact dates bellow) from 12:00 to 13:40 in room T-124.
credit
In order to be able to take the exam, you have to attend the tutorials and actively participate in them. There is no strict limit on attendance, contact me if you will have missed multiple tutorials. The tutorials are not graded, but they are a prerequisite for taking the exam.
study materials
The exercise materials are in the form of Jupyter notebooks using Python. For easy installation of Python, Jupyter, and the required packages, we will use uv . The first exercise is intended to familiarize you with this technology. To replicate the exact environment in which the notebooks were created, you can use the files pyproject.toml and uv.lock.
| # | date | name | topics | assignment | solution |
|---|---|---|---|---|---|
| 1 | 2026-10-06 | introduction to neural networks | basics of machine learning in Python, automatic differentiation, simple neural networks, their architecture, gradient descent, activation functions, hyperparameters and their selection | notebook | |
| 2 | 2026-10-20 | optimization | optimization of neural networks, SGD, Momentum, Nesterov, Adagrad, Adadelta, Adam, AdamW | notebook | |
| 3 | 2026-11-03 | regularization | data splitting into training and test sets, overfitting, bias-variance trade-off, L1, L2 regularization, early stopping, dropout, batching, cross-validation, its variants and usage, double descent | ||
| 4 | 2026-12-01 | recurrent networks and text models | recurrent networks, natural language processing, attention, transformers | ||
| 5 | 2026-12-15 | graph neural networks | learning on graphs, recurrent neural networks on graphs, networks based on random walks, analogies with text processing, convolutions on graphs |