A Batch-Random Algorithm for Learning on Distributed, Heterogeneous Data
Friday, November 9, 2018
9AM – 10AM
Most deep learning models are based off artificial neural networks with multiple layers between the input and output layers. The parameters defining these layers are initialized using random values and are "learned" from data, typically using algorithms that approximate gradient descent. With the increase in the amount of data available for learning, deterministic learning algorithms are often expensive and rarely used in practice. Stochastic gradient descent, and variants using mini-batch gradient descent are the most commonly used algorithms for practical learning problems. The first part of this talk will focus on introducing these concepts, with mentions of potential pitfalls with using them in physics-based problems. The second part of this talk will present a new algorithm that works on distributed, biased data without having to pre-shuffle data.