IRIM Spring Seminar | Combining Learning and Control in Cyber-Physical Systems

Abstract: Cyber-physical systems (CPS), in most instances, represent systems of subsystems with an informationally decentralized structure. To derive optimal control strategies for such systems, we typically assume an ideal model, e.g., controlled transition kernel. Such model-based control approaches cannot effectively facilitate optimal solutions with performance guarantees due to the discrepancy between the model and the actual CPS. On the other hand, in most CPS there is a large volume of data of a dynamic nature which is added to the system gradually in real time and not altogether in advance. Thus, traditional supervised learning approaches cannot always facilitate robust solutions using data derived offline. By contrast, applying reinforcement learning approaches directly to the actual CPS might impose significant implications on safety and robust operation of the system. In this talk, I will present a theoretical framework founded at the intersection of control theory and learning that circumvents these challenges in deriving optimal strategies for CPS. In this framework, we aim at identifying a sufficient information state for the CPS that takes values in a time-invariant space, and use this information state to derive separated control strategies. Separated control strategies are related to the concept of separation between the estimation of the information state and control of the system. By establishing separated control strategies, we can derive offline the optimal control strategy of the system with respect to the information state, which might not be precisely known due to model uncertainties or complexity of the system, and then use learning methods to learn the information state online while data are added gradually to the system in real time. This approach could effectively facilitate optimal solutions with performance guarantees in a wide range of CPS applications such as emerging mobility systems, networked control systems, smart power grids, cooperative cyber-physical networks, cooperation of robots, and internet of things.

Bio: Dr. Andreas Malikopoulos is the Terri Connor Kelly and John Kelly Career Development Associate Professor in the Department of Mechanical Engineering and the Director of the Sociotechnical Systems Center at the University of Delaware (UD). Prior to these appointments, he was the Deputy Director and the Lead of the Sustainable Mobility Theme of the Urban Dynamics Institute at Oak Ridge National Laboratory, and a Senior Researcher with General Motors Global Research & Development. He received a Diploma from the National Technical University of Athens, Greece, and his M.S. and Ph.D. degrees from the University of Michigan, Ann Arbor, in 2004 and 2008, respectively, all in Mechanical Engineering. His research interests span several fields, including analysis, optimization, and control of cyber-physical systems; decentralized stochastic systems; stochastic scheduling and resource allocation; and learning in complex systems. Dr. Malikopoulos is the recipient of several prizes and awards, including the 2007 Dare to Dream Opportunity Grant from the University of Michigan Ross School of Business, the 2007 University of Michigan Teaching Fellow, the 2010 Alvin M. Weinberg Fellowship, the 2019 IEEE Intelligent Transportation Systems Young Researcher Award, and the 2020 UD’s College of Engineering Outstanding Junior Faculty Award. He has been selected by the National Academy of Engineering to participate at the 2010 German-American Frontiers of Engineering (FOE) Symposium and organize a session in transportation at the 2016 European-American FOE Symposium. He has also been selected as a 2012 Kavli Frontiers of Science Scholar by the National Academy of Sciences. Dr. Malikopoulos has been an Associate Editor of the IEEE Transactions on Intelligent Vehicles and IEEE Transactions on Intelligent Transportation Systems from 2017 through 2020. He is currently an Associate Editor of Automatica and IEEE Transactions on Automatic Control. He is a Senior Member of the IEEE, a Fellow of the ASME, and a member of the Board of Governors in the IEEE Intelligent Transportation Systems Society.