Optimal control of multiscale systems using reduced-order models

Carsten Hartmann; Juan C. Latorre; Wei Zhang; Grigorios A. Pavliotis

doi:10.3934/jcd.2014.1.279

Article Contents

2014, Volume 1, Issue 2: 279-306. Doi: 10.3934/jcd.2014.1.279

This issue Previous Article Detecting isolated spectrum of transfer and Koopman operators with Fourier analytic tools Next Article Lattice structures for attractors I

Optimal control of multiscale systems using reduced-order models

1.
Institute of Mathematics, Freie Universität Berlin, 14195 Berlin
2.
Department of Mathematics, Imperial College London, London SW7 2AZ

Received: June 2014

Revised: September 2014

Published: June 2014

Abstract / Introduction Related Papers Cited by

Abstract

We study optimal control of diffusions with slow and fast variables and address a question raised by practitioners: is it possible to first eliminate the fast variables before solving the optimal control problem and then use the optimal control computed from the reduced-order model to control the original, high-dimensional system? The strategy ``first reduce, then optimize''---rather than ``first optimize, then reduce''---is motivated by the fact that solving optimal control problems for high-dimensional multiscale systems is numerically challenging and often computationally prohibitive. We state sufficient and necessary conditions, under which the ``first reduce, then control'' strategy can be employed and discuss when it should be avoided. We further give numerical examples that illustrate the ``first reduce, then optmize'' approach and discuss possible pitfalls.

Keywords:

Mathematics Subject Classification: Primary: 93E20, 93C70; Secondary: 49L20.

Citation:

Related Papers

Cited by

References

[1]	O. Alvarez and M. Bardi, Viscosity solutions methods for singular perturbations in deterministic and stochastic control, SIAM J. Control Optim., 40 (2002), 1159-1188.doi: 10.1137/S0363012900366741.
[2]	O. Alvarez, M. Bardi and C. Marchi, Multiscale singular perturbations and homogenization of optimal control problems, in Geometric Control and Nonsmooth Analysis, vol. 76, World Scientific, Singapore, (2008), 1-27.doi: 10.1142/9789812776075_0001.
[3]	O. Alvarez, M. Bardi and C. Marchi, Multiscale problems and homogenization for second-order hamilton-jacobi equations, J. Differential Equations, 243 (2007), 349-387.doi: 10.1016/j.jde.2007.05.027.
[4]	A. Antoulas, Approximation of Large-Scale Dynamical Systems, SIAM, Philadelphia, 2005.doi: 10.1137/1.9780898718713.
[5]	Z. Artstein, On singularly perturbed ordinary differential equations with measure-valued limits, Math. Bohem., 127 (2002), 139-152.
[6]	E. Asplund and T. Klüner, Optimal control of open quantum systems applied to the photochemistry of surfaces, Phys. Rev. Lett., 106 (2011), 140404.doi: 10.1103/PhysRevLett.106.140404.
[7]	A. Bensoussan and G. Blankenship, Singular perturbations in stochastic control, in Singular Perturbations and Asymptotic Analysis in Control Systems (eds. P. V. Kokotovic, A. Bensoussan and G. L. Blankenship), vol. 90 of Lecture Notes in Control and Information Sciences, Springer Berlin Heidelberg, (1987), 171-260.doi: 10.1007/BFb0007178.
[8]	A. Bensoussan, Perturbation Methods in Optimal Control, Gauthiers-Villars, Chichester, 1988.
[9]	A. Bensoussan and H. Nagai, An ergodic control problem arising from the principal eigenvalue of an elliptic operator, J. Math. Soc. Japan, 43 (1991), 49-65.doi: 10.2969/jmsj/04310049.
[10]	J.-M. Bismut, Martingales, the malliavin calculus and hypoellipticity under general hörmander's conditions, Z. Wahrsch. Verw. Gebiete, 56 (1981), 469-505.doi: 10.1007/BF00531428.
[11]	R. Buckdahn and Y. Hu, Probabilistic approach to homogenizations of systems of quasilinear parabolic PDEs with periodic structures, Nonlinear Analysis, 32 (1998), 609-619.doi: 10.1016/S0362-546X(97)00505-1.
[12]	R. Buckdahn, Y. Hu and S. Peng, Probabilistic approach to homogenization of viscosity solutions of parabolic pdes, NoDEA Nonlinear Differential Equations Appl., 6 (1999), 395-411.doi: 10.1007/s000300050010.
[13]	Y. Chahlaoui and P. Van Dooren, Benchmark examples for model reduction of linear time invariant dynamical systems, in Dimension Reduction of Large-Scale Systems, vol. 45 of Lect. Notes Comput. Sci. Eng., (2005), 379-392.doi: 10.1007/3-540-27909-1_24.
[14]	P. Dai Pra, L. Meneghini and W. Runggaldier, Connections between stochastic control and dynamic games, Mathematics of Control, Signals and Systems, 9 (1996), 303-326.doi: 10.1007/BF01211853.
[15]	M. H. Davis and A. R. Norman, Portfolio selection with transaction costs, Math. Oper. Res., 15 (1990), 676-713.doi: 10.1287/moor.15.4.676.
[16]	P. Dupuis, K. Spiliopoulos and H. Wang, Importance sampling for multiscale diffusions, Multiscale Model. Simul., 10 (2012), 1-27.doi: 10.1137/110842545.
[17]	P. Dupuis and H. Wang, Importance sampling, large deviations, and differential games, Stochastics and Stochastic Reports, 76 (2004), 481-508.doi: 10.1080/10451120410001733845.
[18]	L. C. Evans, The perturbed test function method for viscosity solutions of nonlinear PDE, P. Roy. Soc. Edinb. A, 111 (1989), 359-375.doi: 10.1017/S0308210500018631.
[19]	W. H. Fleming and W. M. McEneaney, Risk-sensitive control on an infinite time horizon, SIAM J. Control Optim., 33 (1995), 1881-1915.doi: 10.1137/S0363012993258720.
[20]	W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Springer, 2006.
[21]	V. Gaitsgory, Suboptimization of singularly perturbed control systems, SIAM J .Control Optim., 30 (1992), 1228-1249.doi: 10.1137/0330065.
[22]	Z. Gajic and M.-T. Lim, Optimal Control of Singularly Perturbed Linear Systems and Applications, CRC Press, New York, 2001.doi: 10.1201/9780203907900.
[23]	K. Glover, All optimal Hankel-norm approximations of linear multivariable systems and their $L^{\infty}$-error bounds, Int. J. Control, 39 (1984), 1115-1193.doi: 10.1080/00207178408933239.
[24]	G. Grammel, Averaging of singularly perturbed systems, Nonlinear Analysis, 28 (1997), 1851-1865.doi: 10.1016/S0362-546X(95)00243-O.
[25]	S. Gugercin and A. Antoulas, A survey of model reduction by balanced truncation and some new results, Int. J. Control, 77 (2004), 748-766.doi: 10.1080/00207170410001713448.
[26]	C. Hartmann, Balanced model reduction of partially observed Langevin equations: An averaging principle, Math. Comput. Model. Dyn. Syst., 17 (2011), 463-490.doi: 10.1080/13873954.2011.576517.
[27]	C. Hartmann, B. Schäfer-Bung and A. Zueva, Balanced averaging of bilinear systems with applications to stochastic control, J. Control Optim., 51 (2013), 2356-2378.doi: 10.1137/100796844.
[28]	C. Hartmann and C. Schütte, Efficient rare event simulation by optimal nonequilibrium forcing, J. Stat. Mech. Theor. Exp., 2012 (2012), P11004.doi: 10.1088/1742-5468/2012/11/P11004.
[29]	C. Hartmann, V. Vulcanov and C. Schütte, Balanced truncation of linear second-order systems: A Hamiltonian approach, Multiscale Model. Simul., 8 (2010), 1348-1367.doi: 10.1137/080732717.
[30]	C. J. Holland, A minimum principle for the principal eigenvalue for second-order linear elliptic equations with natural boundary conditions, Comm. Pure Appl. Math., 31 (1978), 509-519.doi: 10.1002/cpa.3160310406.
[31]	N. Ichihara, A stochastic representation for fully nonlinear PDEs and its application to homogenization, J. Math. Sci. Univ. Tokyo, 12 (2005), 467-492.
[32]	P. Imkeller, N. S. Namachchivaya, N. Perkowski and H. C. Yeong, Dimensional reduction in nonlinear filtering: A homogenization approach, Ann. Appl. Probab., 23 (2013), 2290-2326.doi: 10.1214/12-AAP901.
[33]	Y. Kabanov and S. Pergamenshchikov, Two-scale Stochastic Systems: Asymptotic Analysis and Control, Springer, Berlin, Heidelberg, Paris, 2003.doi: 10.1007/978-3-662-13242-5.
[34]	P. V. Kokotovic, Applications of singular perturbation techniques to control problems, SIAM Review, 26 (1984), 501-550.doi: 10.1137/1026104.
[35]	P. Kokotovic, Singular perturbation techniques in control theory, in Singular Perturbations and Asymptotic Analysis in Control Systems (eds. P. V. Kokotovic, A. Bensoussan and G. L. Blankenship), vol. 90 of Lecture Notes in Control and Information Sciences, Springer Berlin Heidelberg, (1987), 1-55.doi: 10.1007/BFb0007176.
[36]	T. Kurtz and R. H. Stockbridge, Stationary solutions and forward equations for controlled and singular martingale problems, Electron. J. Probab, 6 (2001), 52pp.doi: 10.1214/EJP.v6-90.
[37]	H. J. Kushner, Direct averaging and perturbed test function methods for weak convergence, Lect. Notes Contr. Inf., 81 (1986), 412-426.doi: 10.1007/BFb0007118.
[38]	H. J. Kushner, Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems, Birkhäuser, Boston, 1990.doi: 10.1007/978-1-4612-4482-0.
[39]	H. J. Kushner and P. G. Dupuis, Numerical Methods for Stochastic Control Problems in Continuous Time, Springer, New York, 2001.doi: 10.1007/978-1-4613-0007-6.
[40]	J. C. Latorre, P. Metzner, C. Hartmann and C. Schütte, A structure-preserving numerical discretization of reversible diffusions, Commun. Math. Sci., 9 (2011), 1051-1072.doi: 10.4310/CMS.2011.v9.n4.a6.
[41]	P.-L. Lions, G. Papanicolaou and S. R. S. Varadhan, Homogenization of hamilton-jacobi equations, Preprint.
[42]	P.-L. Lions and P. E. Souganidis, Correctors for the homogenization of hamilton-jacobi equations in the stationary ergodic setting, Commun. Pure Appl. Math., 56 (2003), 1501-1524.doi: 10.1002/cpa.10101.
[43]	P. Malliavin, Stochastic calculus of variations and hypoelliptic operators, in Proceedings of the International Conference on Stochastic Differential Equations 1976, Wiley, New York, (1978), 195-263.
[44]	B. Moore, Principal component analysis in linear system: Controllability, observability and model reduction, IEEE Trans. Automat. Control, 26 (1981), 17-32.doi: 10.1109/TAC.1981.1102568.
[45]	E. Nelson, Dynamical Theories of Brownian Motion, Princeton University Press, 1967.
[46]	G. Nguetseng, A general convergence result for a functional related to the theory of homogenization, SIAM J. Math. Anal., 20 (1989), 608-623.doi: 10.1137/0520043.
[47]	B. K. Øksendal, Stochastic Differential Equations: An Introduction With Applications, Springer, 2003.doi: 10.1007/978-3-642-14394-6.
[48]	G. Papanicolaou, A. Bensoussan and J. Lions, Asymptotic Analysis for Periodic Structures, Elsevier, Burlington, MA, 1978.
[49]	A. Papavasiliou, G. A. Pavliotis and A. M. Stuart, Maximum likelihood drift estimation for multiscale diffusions, Stochastic Process. Appl., 119 (2009), 3173-3210.doi: 10.1016/j.spa.2009.05.003.
[50]	J. H. Park, R. B. Sowers and N. S. Namachchivaya, Dimensional reduction in nonlinear filtering, Nonlinearity, 23 (2010), 305-324.doi: 10.1088/0951-7715/23/2/005.
[51]	G. A. Pavliotis and A. M. Stuart, Parameter estimation for multiscale diffusions, J. Stat. Phys., 127 (2007), 741-781.doi: 10.1007/s10955-007-9300-6.
[52]	G. Pavliotis and A. Stuart, Multiscale Methods: Averaging and Homogenization, Springer, 2008.
[53]	H. Pham, Continuous-time Stochastic Control and Optimization with Financial Applications, Stochastic modelling and applied probability, Springer, Berlin, Heidelberg, 2009.doi: 10.1007/978-3-540-89500-8.
[54]	M. Robin, Long-term average cost control problems for continuous time Markov processes: A survey, Acta Appl. Math., 1 (1983), 281-299.doi: 10.1007/BF00046603.
[55]	C. Schütte, S. Winkelmann and C. Hartmann, Optimal control of molecular dynamics using markov state models, Math. Program. Ser. B, 134 (2012), 259-282.doi: 10.1007/s10107-012-0547-6.
[56]	H. Stapelfeldt, Laser aligned molecules: Applications in physics and chemistry, Physica Scripta, 2004 (2004), 132-136.doi: 10.1238/Physica.Topical.110a00132.
[57]	A. Steinbrecher, Optimal control of robot guided laser material treatment, in Progress in Industrial Mathematics at ECMI 2008 (eds. A. D. Fitt, J. Norbury, H. Ockendon and E. Wilson), Springer Berlin Heidelberg, (2010), 505-511.doi: 10.1007/978-3-642-12110-4_79.
[58]	A. Vigodner, Limits of singularly perturbed control problems with statistical dynamics of fast motions, SIAM J. Control Optim., 35 (1997), 1-28.doi: 10.1137/S0363012994264207.
[59]	F. Watbled, On singular perturbations for differential inclusions on the infinite interval, J. Math. Anal. Appl., 310 (2005), 362-378.doi: 10.1016/j.jmaa.2005.01.067.
[60]	J. Zabczyk, Exit problem and control theory, Syst. Control Lett., 6 (1985), 165-172.doi: 10.1016/0167-6911(85)90036-2.