Automatic Gain Tuning based on Gaussian Process Global Optimization (= Bayesian Optimization)
- 1. Automatic Gain Tuning based on Gaussian Process Global Optimization (= Bayesian Optimization) https://is.tuebingen.mpg.de/publications/marco_icra_2016
- 2. = A simple PD control example Global optimal gains, θ to get a minimum cost J ?
- 3. A simple PD control example Procedure of Bayesian Optimization 1. GP prior before observing any data 2. GP posterior, after five noisy evaluations 3. The next parameters θnext are chosen at the maximum of Acquisition function Repeat until you can find a globally optimal θ
- 5. Information gain, : Mutual information for an observed data Reduction of uncertainty in the location ∗ by selecting points that are expected to cause the largest reduction in entropy of distribution ∗ Acquisition function and Entropy Search
- 6. Control design problem propose the use of Entropy Search, a recent algorithm for global Bayesian optimization, as the minimizer for the LQR tuning problem. ES employs a Gaussian process (GP) as a non-parametric model capturing the knowledge about the unknown cost function. consider a system that follows a discrete-time nonlinear dynamic model system states xk, control input uk, and zero-mean process noise wk at time instant k
- 7. common way to measure the performance of a control system is through a quadratic cost function such as cost captures a trade-off between control performance (keeping xk small) and control effort(keeping uk small) Nonlinear control design problem is intractable in general. linear model is often sufficient for control design Control design problem
- 8. Linear Quadratic Regulator (LQR) static gain matrix F can readily be computed by solving the discrete-time infinite-horizon LQR problem for the nominal model (An, Bn) and the weights (Q, R). LQR tuning problem goal of the automatic LQR tuning is to vary the parameters θ such as to minimize the cost
- 9. Optimization problem Consider the approximate one of quadratic cost function
- 10. uncertainty over the objective function J is represented by a probability measure p(J), typically a Gaussian process (GP) LQR TUNING WITH ENTROPY SEARCH prior knowledge about the cost function J as the GP with mean function and covariance function ∗ squared exponential (SE) covariance function
- 11. noisy evaluations of cost function can be modeled as Conditioning the GP on the data then yields another GP with posterior mean and a posterior variance ∗ . shape of the mean is adjusted to fit the data points, and the uncertainty (standard deviation) is reduced around the evaluations points. LQR TUNING WITH ENTROPY SEARCH
- 12. LQR TUNING WITH ENTROPY SEARCH ES is one out of several popular formulations of Bayesian Optimization Aims to reduce the uncertainty in the location ∗ expected to cause the largest reduction in entropy( uncertainty) of distribution, J(θ)
- 13. Bayesian Optimization Bayesian Optimization and How to Scale It Up Zi Wang CS Colloquium, US
- 20. Information gain, : Mutual information for an observed data Reduction of uncertainty in the location ∗ by selecting points that are expected to cause the largest reduction in entropy of distribution ∗
- 21. Probability of Improvement : Gaussian Cumulative Density Ft’n
- 28. Evolution of an example Gaussian process for three successive function evaluations (orange dots) Approximated probability distribution over the location of the minimum pmin(θ) in green