Dynamic resource allocation problems in communication networks

During the course of this 5 days in-person summer school, we will study mathematical tools to solve the dynamic resource allocation problems in communication networks. Most of the resource allocation problems are known to be NP-hard to solve. However, it is possible to design efficient heuristics using the theory of Markov decision processes. We will talk about different heuristics, such as LP-based and Whittle index policies, and provide proof of their performances. To complete these first formal approaches, we will also take a different path to solve such problems by using Deep Reinforcement Learning algorithms. All the techniques seen during this class will be illustrated on different network problems such as adaptive routing, TCP control, optimal control of the age of information, and optimal channel selection. This course will also have some lab sessions and we will provide code in python to implement the different algorithms seen during the class.

Syllabus

Part I: Provably efficient heuristics for solving large-scale resource allocation problems.

• Day I: Introduction to resource allocation problem and MDP and Restless Bandit in Finite Horizon. Slides, Lab, Correction

• Day II: Weakly coupled MDP and the resolving heuristic. Slides, Lab, Correction

• Day III: Constrained Finite Horizon Stochastic Optimization Problems. Slides

Part II: Machine Learning for Resource Allocation Problems

• Day IV: Online and Distributed algorithm for Resource Allocation Problem with unknown parameters. Slides

• Day V: Deep-learning applied to Resource Allocation Problems. Slides Lab Learning Distribution Lab RL