%0 Conference Proceedings
%T Hyperparameter optimization of deep neural networks: combining Hperband with Bayesian model selection
%+ Image, Modélisation, Analyse, GEométrie, Synthèse (IMAGES)
%+ Département Images, Données, Signal (IDS)
%+ MedisysResearch Lab (Medisys)
%A Bertrand, Hadrien
%A Ardon, Roberto
%A Perrot, Matthieu
%A Bloch, Isabelle
%< avec comité de lecture
%Z HB:CAP-17
%( CAp
%B CAp
%C Grenoble, France
%8 2017
%D 2017
%K Deep Learning
%K Gaussian Process
%K Bayesian Optimization
%K Hyperparameter Optimization
%Z Computer Science [cs]/Machine Learning [cs.LG]
%Z Computer Science [cs]/Medical Imaging
%Z Computer Science [cs]/Image Processing [eess.IV]Conference papers
%X One common problem in building deep learning architectures is the choice of the hyper-parameters. Among the various existing strategies, we propose to combine two complementary ones. On the one hand, the Hyperband method formalizes hyper-parameter optimization as a resource allocation problem, where the resource is the time to be distributed between many configurations to test. On the other hand, Bayesian optimization tries to model the hyper-parameter space as efficiently as possible to select the next model to train. Our approach is to model the space with a Gaussian process and sample the next group of models to evaluate with Hyperband. Preliminary results show a slight improvement over each method individually, suggesting the need and interest for further experiments.
%G English
%L hal-02412262
%U https://telecom-paris.hal.science/hal-02412262
%~ INSTITUT-TELECOM
%~ ENST
%~ TELECOM-PARISTECH
%~ PARISTECH
%~ UNIV-PARIS-SACLAY
%~ TELECOM-PARISTECH-SACLAY
%~ LTCI
%~ IDS
%~ IMAGES
%~ INSTITUTS-TELECOM