University of Texas at Austin

Past Event: Oden Institute Seminar

An Optimal Control Framework for Efficient Training of Deep Neural Networks

Lars Ruthotto, Department of Mathematics and Computer Science, Emory University

1 – 2PM
Friday Apr 13, 2018

POB 6.304

Abstract

One of the most promising areas in artificial intelligence is deep learning, a form of machine learning that uses neural networks containing many hidden layers. Recent success has led to breakthroughs in applications such as speech and image recognition. However, more theoretical insight is needed to create a rigorous scientific basis for designing and training deep neural networks, increasing their scalability, and providing insight into their reasoning. In this talk, we present a new mathematical framework that simplifies designing, training, and analyzing deep neural networks. It is based on the interpretation of deep learning as a dynamic optimal control problem similar to path-planning problems. We will exemplify how this understanding helps design, analyze, and train deep neural networks. First, we will focus on ways to ensure the stability of the dynamics in both the continuous and discrete setting and on ways to exploit discretization to obtain adaptive neural networks. Second, we will present new multilevel and multiscale approaches, derived from he continuous formulation. Finally, we will discuss adaptive higher-order discretization methods and illustrate their impact on the optimization problem.

Event information

Date
1 – 2PM
Friday Apr 13, 2018
Location POB 6.304
Hosted by