Introduction -- PART I: PROBABILISTIC REASONING: Representation. Inference. Parameter Learning. Structure Learning.
Simple Decisions -- PART II: SEQUENTIAL PROBLEMS: Approximate Value Functions. Online Planning. Policy Search.
Policy Gradient Estimation. Policy Gradient Optimization. Actor-Critic Methods. Policy Validation -- PART III: MODEL UNCERTAINTY: Exploration and Exploitation. Model-Based Methods. Model-Free Methods. Imitation Learning --
PART IV: STATE UNCERTAINTY: Beliefs. Exact Belief State Planning. Offline Belief State Planning. Online Belief State Planning. Controller Abstractions -- PART V: MULTIAGENT SYSTEMS: Sequential Problems. State Uncertainty.
Collaborative Agents -- APPENDICES: A: Mathematical Concepts -- B: Probability Distributions -- C: Computational Complexity -- D: Neural Representations -- E: Search Algorithms -- F: Problems -- G: Julia.

The book first addresses the problem of reasoning about uncertainty and objectives in simple decisions at a single point in time, and then turns to sequential decision problems in stochastic environments where the outcomes of our actions are uncertain. It goes on to address model uncertainty, when we do not start with a known model and must learn how to act through interaction with the environment; state uncertainty, in which we do not know the current state of the environment due to imperfect perceptual information; and decision contexts involving multiple agents. The book focuses primarily on planning and reinforcement learning, although some of the techniques presented draw on elements of supervised learning and optimization. Algorithms are implemented in the Julia programming language. Figures, examples, and exercises convey the intuition behind the various approaches presented.