Bayesian Optimization for Backpropagation in Monte-Carlo Tree Search

In large domains, Monte-Carlo tree search (MCTS) is required to estimate the values of states as efficiently and accurately possible. However, standard update rule in backpropagation assumes a stationary distribution for returns, particularly min-max trees, convergence true value can be slow because averaging. We present two methods, Softmax MCTS Monotone MCTS, which generalize previous attempts improve upon strategy. demonstrate that both methods reduce finding optimal monotone functions, we do so by performing Bayesian optimization with Gaussian process (GP) prior. conduct experiments on computer Go, where returns are given deep neural network, show our proposed framework outperforms methods.