Skip to main content

Evolving Deep Networks Using HPC...

Publication Type
Conference Paper
Book Title
Proceedings of the 3rd Workshop on Machine Learning in HPC Environments
Publication Date
Publisher Location
United States of America
Conference Name
3rd Workshop on Machine Learning in High Performance Computing Environments (in conjuction with SC17)
Conference Location
Denver, Colorado, United States of America
Conference Sponsor
Conference Date

While a large number of deep learning networks have been studied
and published that produce outstanding results on natural image
datasets, these datasets only make up a fraction of those to which
deep learning can be applied. These datasets include text data, audio
data, and arrays of sensors that have very different characteristics
than natural images. As these “best” networks for natural images
have been largely discovered through experimentation and cannot
be proven optimal on some theoretical basis, there is no reason
to believe that they are the optimal network for these drastically
different datasets. Hyperparameter search is thus often a very im-
portant process when applying deep learning to a new problem. In
this work we present an evolutionary approach to searching the
possible space of network hyperparameters and construction that
can scale to 18, 000 nodes. This approach is applied to datasets of
varying types and characteristics where we demonstrate the ability
to rapidly find best hyperparameters in order to enable practitioners
to quickly iterate between idea and result.