Authors
Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Andrew Senior, Paul Tucker, Ke Yang, Quoc V Le, Andrew Y Ng
Publication date
2012
Conference
Advances in neural information processing systems
Pages
1223-1231
Description
Abstract Recent work in unsupervised feature learning and deep learning has shown that
being able to train large models can dramatically improve performance. In this paper, we
consider the problem of training a deep network with billions of parameters using tens of
thousands of CPU cores. We have developed a software framework called DistBelief that
can utilize computing clusters with thousands of machines to train large models. Within this
framework, we have developed two algorithms for large-scale distributed training:(i) ...
Total citations
20122013201420152016266154219169
Scholar articles
J Dean, G Corrado, R Monga, K Chen, M Devin, M Mao… - Advances in neural information processing systems, 2012