Authors
Georg Heigold, Erik McDermott, Vincent Vanhoucke, Andrew Senior, Michiel Bacchiani
Publication date
2014/5/4
Conference
2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Pages
5587-5591
Publisher
IEEE
Description
ABSTRACT This paper explores asynchronous stochastic optimization for sequence training
of deep neural networks. Sequence training requires more computation than frame-level
training using pre-computed frame data. This leads to several complications for stochastic
optimization, arising from significant asynchrony in model updates under massive
parallelization, and limited data shuffling due to utterance-chunked processing. We analyze
the impact of these two issues on the efficiency and performance of sequence training. In ...
of deep neural networks. Sequence training requires more computation than frame-level
training using pre-computed frame data. This leads to several complications for stochastic
optimization, arising from significant asynchrony in model updates under massive
parallelization, and limited data shuffling due to utterance-chunked processing. We analyze
the impact of these two issues on the efficiency and performance of sequence training. In ...
Total citations
Scholar articles
G Heigold, E McDermott, V Vanhoucke, A Senior… - 2014 IEEE International Conference on Acoustics, …, 2014
Dates and citation counts are estimated and are determined automatically by a computer program.