• An extended lecture/slides summary of the book Reinforcement Learning and Optimal Control: Ten Key Ideas for Reinforcement Learning and Optimal Control
• Videolectures on Reinforcement Learning and Optimal Control: Course at Arizona State University, 13 lectures, January-February 2019.
• Overview lecture on Reinforcement Learning and Optimal Control: Video of book overview lecture at Stanford University, March 2019. Click here for the slides from the lecture.
• Lecture on Feature-Based Aggregation and Deep Reinforcement Learning: Video from a lecture at Arizona State University, on 4/26/18. Video from Youtube, and Lecture Slides. Based on the paper “Feature-Based Aggregation and Deep Reinforcement Learning: A Survey and Some New Implementations”, Lab. for Information and Decision Systems Report, MIT, April 2018; http://arxiv.org/abs/1804.04577.
• Lectures on Exact and Approximate Finite Horizon DP: Videos from a 4-lecture, 4-hour short course at the University of Cyprus on finite horizon DP, Nicosia, 2017. Videos from Youtube. (Lecture Slides: Lecture 1, Lecture 2, Lecture 3, Lecture 4.) Based on Chapters 1 and 6 of the book Dynamic Programming and Optimal Control, Vol. I, 4th Edition, Athena Scientific.
• Lectures on Exact and Approximate Infinite Horizon DP: Videos from a 6-lecture, 12-hour short course at Tsinghua Univ. on approximate DP, Beijing, China, 2014. From the Tsinghua course site, and from Youtube. (Complete Set of Lecture Slides.) Based on the book Dynamic Programming and Optimal Control, Vol. II, 4th Edition: Approximate Dynamic Programming, Athena Scientific.
• Video from a January 2017 slide presentation on the relation of Proximal Algorithms and Temporal Difference Methods, for solving large linear systems of equations. The slides are hard to read at times in the video, so you may wish to download the PDF version of the slides. See also Related Slides from NIPS 2017 Click here for a Related Report. Based on the books Convex Optimization Algorithms, Nonlinear Programming, 3rd Edition, and Dynamic Programming and Optimal Control, Vol. II, 4th Edition: Approximate Dynamic Programming,Athena Scientific.
• Video of a survey lecture on Approximate DP research from ADPRL 2014. (Lecture Slides.)

• Video from a January 2017 slide presentation on the relation of Proximal Algorithms and Temporal Difference Methods, for solving large linear systems of equations. The slides are hard to read at times in the video, so you may wish to download the PDF version of the slides. Click here for a Related Report, and a more recent set of Slides from NIPS 2017. Based on the books Convex Optimization Algorithms, Nonlinear Programming, 3rd Edition, and Dynamic Programming and Optimal Control, Vol. II, 4th Edition: Approximate Dynamic Programming, Athena Scientific.
• Video from a survey lecture (April 2016) on Incremental methods for minimizing a sum of a large number of functions, a common problem in many practical contexts, including machine learning and signal processing. Related Report and (Lecture Slides.) Based on the books Convex Optimization Algorithms, and Nonlinear Programming, 3rd Edition.
• Video from a 2008 lecture on Polyhedral Approximations in Convex Optimization, a unifying inner and outer approximation framework, which includes cutting plane and simplicial decomposition methods. Related Journal Paper and Lecture Slides

Video from a Oct. 2017 Lecture at UConn on Optimal control, abstract, and semicontractive dynamic programming. Related paper, and set of Lecture Slides.

Video from a May 2017 Lecture at MIT on the solutions of Bellman’s equation, Stable optimal control, and semicontractive dynamic programming. Related paper, and set of Lecture Slides.

Videos from a 5-lecture series on Semicontractive Dynamic Programming, a type of methodology, introduced in the research monograph Abstract Dynamic Programming.

The monograph aims at a unified and economical development of the core theory and algorithms of total cost sequential decision problems. Semicontractive DP refers qualitatively to a collection of models where some policies have a regularity/contraction-like property but others do not. They are exemplified by models involving a termination state, such as shortest path-type problems, both deterministic and stochastic.

The lectures focus on research, which is described in recent papers and an on-line 2nd edition of the monograph. The lectures are as follows:

• • Video lecture 1: Introduction and Semicontractive Examples. Set of Lecture Slides.
  • Video lecture 2: Semicontractive Analysis for Stochastic Optimal Control. Set of Lecture Slides.
  • Video lecture 3: Extensions to Abstract DP Models. Set of Lecture Slides.
  • Video lecture 4: Applications to Stochastic Shortest Path and Other Problems. Set of Lecture Slides.
  • Video lecture 5: Algorithms. Set of Lecture Slides.