Content: In this lecture we analyse the class of Markov processes in continuous time. We start with a focus on processes processes in discrete state spaces. For this we introduce point processes as a ...

We consider a class of discrete time, dynamic decision-making models which we refer to as Periodically Time-Inhomogeneous Markov Decision Processes (PTMDPs). In these models, the decision-making ...