Optimization Based Deep Neural Networks and Beyond

This talk will introduce novel deep neural networks (DNNs) as constrained optimization problems. In particular, the talk will introduce DNNs with memory, which help overcome the vanishing gradient challenge. The talk will also explore reducing the computational complexity of DNNs by introducing a bias ordering. Approximation properties of the DNNs will also be discussed. 
These proposed DNNs will be shown to be excellent surrogates to parameterized (nonlinear) partial differential equations (PDEs), Bayesian inverse problems, data assimilation problems, with multiple advantages over the traditional approaches. The DNNs will also be applied to chemically reacting flows problems. The latter require solving a system of stiff ODEs and fluid flow equations. These are highly challenging problems, for instance, for combustion the number of reactions can be significant (over 100). Due to the large CPU requirements of chemical reactions (over 99% of total CPU time), a large number of flow and combustion problems are presently beyond the capabilities of even the largest supercomputers.