Title :

Adaptive Learning Structures for Real-Time Optimal Control and Graphical Games

Speaker :

Prof. F.L. Lewis

Fellow IEEE, Fellow IFAC, U.K. Charted Engineer

Moncrief-O’Donnell Endowed Chair,Head, Advanced Controls & Sensors Group,UTA Research Institute (UTARI),The University of Texas at Arlington, USA

Venue :

Room 215, William M. W. Mong Engineering Building, CUHK

Date :

May 22, 2013, Wednesday
4:30 PM - 5:30 PM

Abstract :

This talk will discuss some new adaptive control structures for learning online the solutions to optimal control problems and multi-player differential games. Techniques from reinforcement learning are used to design a new family of adaptive controllers based on actor-critic mechanisms that converge in real time to optimal control and game theoretic solutions. Continuous-time systems are considered. A new sort of distributed game is defined- namely multi-agent graphical games, where the interactions between players are restricted by a distributed communication topology.

Optimal feedback control design has been responsible for much of the successful performance of engineered systems in aerospace, industrial processes, vehicles, ships, robotics, and elsewhere since the 1960s. Optimal control design is performed offline by solving optimal design equations including the algebraic Riccati equation and the Game ARE. It is difficult to perform optimal designs for nonlinear systems since they rely on solutions to complicated Hamilton-Jacobi-Bellman or HJI equations. Finally, optimal design generally requires that the full system dynamics be known.

Optimal Adaptive Control. Adaptive control has provided powerful techniques for online learning of effective controllers for unknown nonlinear systems. In this talk we discuss online adaptive algorithms for learning optimal control solutions for continuous-time linear and nonlinear systems. This is a novel class of adaptive control algorithms that converge to optimal control solutions by online learning in real time. In the linear quadratic (LQ) case, the algorithms learn the solution to the ARE by adaptation along the system motion trajectories. In the case of nonlinear systems with general performance measures, the algorithms learn the (approximate smooth local) solutions of HJ or HJI equations. The algorithms are based on actor-critic reinforcement learning techniques. Multi-Player Differential Games. New algorithms will be presented for solving online non zero-sum multi-player games for continuous-time systems. We use an adaptive control structure motivated by reinforcement learning policy iteration. The result is an adaptive control system with multiple tuned control loops that learns based on the interplay of agents in a game, to deliver true online gaming behavior. Graphical Games. A new formulation for control of multi-agent cooperative systems is given. A novel form of game among agents in a communication graph is formulated where each agent is allowed to interact only with its neighbors. A new notion of Nash equilibrium is defined that is suitable for graphical games.

Biography :

F.L. Lewis, Fellow IEEE, Fellow IFAC, Fellow U.K. Institute of Measurement & Control, PE Texas, U.K. Chartered Engineer, is Distinguished Scholar Professor, Distinguished Teaching Professor, and Moncrief-O`Donnell Chair at The University of Texas at Arlington Research Institute. IEEE Control Systems Society Distinguished Lecturer. He obtained the Bachelor`s Degree in Physics/EE and the MSEE at Rice University, the MS in Aeronautical Engineering from Univ. W. Florida, and the Ph.D. at Ga. Tech. He works in feedback control, reinforcement learning, intelligent systems, and distributed control systems. He is author of 6 U.S. patents, 250 journal papers, 360 conference papers, 15 books, 44 chapters, and 11 journal special issues. He received the Fulbright Research Award, NSF Research Initiation Grant, ASEE Terman Award, Int. Neural Network Soc. Gabor Award 2009, U.K. Inst Measurement & Control Honeywell Field Engineering Medal 2009. Received IEEE Computational Intelligence Society Neural Networks Pioneer Award 2012. Distinguished Foreign Scholar, Nanjing Univ. Science & Technology. Project 111 Professor at Northeastern University, China. Received Outstanding Service Award from Dallas IEEE Section, selected as Engineer of the Year by Ft. Worth IEEE Section. Listed in Ft. Worth Business Press Top 200 Leaders in Manufacturing. Received the 2010 IEEE Region 5 Outstanding Engineering Educator Award and the 2010 UTA Graduate Dean’s Excellence in Doctoral Mentoring Award. Elected to UTA Academy of Distinguished Teachers 2012. He served on the NAE Committee on Space Station in 1995. Founding Member of the Board of Governors of the Mediterranean Control Association. Helped win the IEEE Control Systems Society Best Chapter Award (as Founding Chairman of DFW Chapter), the National Sigma Xi Award for Outstanding Chapter (as President of UTA Chapter), and the US SBA Tibbets Award in 1996 (as Director of ARRI`s SBIR Program).

    **************************************** ALL ARE WELCOME ****************************************

Enquiries: Ms. Winnie Wong / Flora Au-Yeung, Department of Mechanical and Automation Engineering, CUHK at 3943 8337 / 3943 7026. *MAE Series (2012-13) is contained in the World-Wide Web home page at http://www3.mae.cuhk.edu.hk/mae_seminars.php.