Authors
Georg Heigold, Erik McDermott, Vincent Vanhoucke, Andrew Senior, Michiel Bacchiani
Publication date
2014/5/4
Conference
2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Pages
5587-5591
Publisher
IEEE
Description
ABSTRACT This paper explores asynchronous stochastic optimization for sequence training
of deep neural networks. Sequence training requires more computation than frame-level
training using pre-computed frame data. This leads to several complications for stochastic
optimization, arising from significant asynchrony in model updates under massive
parallelization, and limited data shuffling due to utterance-chunked processing. We analyze
the impact of these two issues on the efficiency and performance of sequence training. In ...
Total citations
20142015201661410
Scholar articles
G Heigold, E McDermott, V Vanhoucke, A Senior… - 2014 IEEE International Conference on Acoustics, …, 2014