View article

We present a generally applicable tree-like sparse multilayer architecture that has a balanced connection from all input neurons to all output neurons. If the ratio between input and output neurons is fixed, the parameters required by our architecture scale with O(n ^1.5 ) as compared to O(n ² ) in a fully connected layer, where n is the number of input neurons. Our sparse 2-layer architecture performs similar and/or superior when compared to its fully connected 1-layer and 2-layer counter parts on the IMDB review sentiment classification task, the Reuters news categorization task and the CIFAR-10 image classification task.