THE STOCHASTIC LINEAR QUADRATIC OPTIMAL CONTROL PROBLEM IN HILBERT SPACES: A POLYNOMIAL CHAOS APPROACH

. We consider the stochastic linear quadratic optimal control problem for state equations of the Itˆo-Skorokhod type, where the dynamics are driven by strongly continuous semigroup. We provide a numerical framework for solving the control problem using a polynomial chaos expansion approach in white noise setting. After applying polynomial chaos expansion to the state equation, we obtain a system of inﬁnitely many deterministic partial diﬀeren- tial equations in terms of the coeﬃcients of the state and the control variables. We set up a control problem for each equation, which results in a set of deter- ministic linear quadratic regulator problems. Solving these control problems, we ﬁnd optimal coeﬃcients for the state and the control. We prove the opti- mality of the solution expressed in terms of the expansion of these coeﬃcients compared to a direct approach. Moreover, we apply our result to a fully sto- chastic problem, in which the state, control and observation operators can be random, and we also consider an extension to state equations with memory noise.


(Communicated by Irena Lasiecka)
Abstract. We consider the stochastic linear quadratic optimal control problem for state equations of the Itô-Skorokhod type, where the dynamics are driven by strongly continuous semigroup. We provide a numerical framework for solving the control problem using a polynomial chaos expansion approach in white noise setting. After applying polynomial chaos expansion to the state equation, we obtain a system of infinitely many deterministic partial differential equations in terms of the coefficients of the state and the control variables. We set up a control problem for each equation, which results in a set of deterministic linear quadratic regulator problems. Solving these control problems, we find optimal coefficients for the state and the control. We prove the optimality of the solution expressed in terms of the expansion of these coefficients compared to a direct approach. Moreover, we apply our result to a fully stochastic problem, in which the state, control and observation operators can be random, and we also consider an extension to state equations with memory noise.
1. Introduction. Stochastic optimization of infinite dimensional systems arise in many applications, and has become a very active research field in recent years. For finite dimensional systems, extensive results in the field can be found for instance in [15,63]. In particular, the linear quadratic regulator problem (LQR) has been well studied in deterministic setting. The stochastic analogue in finite dimensions was first solved by Wonham and Kushner in the 1960's [32,60,61]. In the infinite dimensional setting, the stochastic linear quadratic regulator (SLQR) problem was first treated by Ichikawa for systems driven by strongly continuous semigroups and bounded control and noise operators [27], where a full Riccati synthesis of the problem analogous to that obtained in finite dimesnions was developed. In later works, Flandoli and Da Prato considered the problem in the analytic semigroups framework for Neumann or Dirichlet type control operators, which represent parabolic PDE with boundary controls [12,14]. For systems with singular estimates which model a certain class of coupled parabolic/ hyperbolic PDEs, the stochastic linear quadratic problem has been studied by Hafizoglu [22]. In [23], the results were extended to the case including the disturbance in the control and nonzero G (Bolza problem). An approximation scheme for solving the control problem and the associated Riccati equation was also introduced in [39]. Other results have been proposed for systems with stochastic coefficients in [20,21].
In this work, we consider a polynomial chaos approach for solving the infinite dimensional SLQR problem. The aim is to provide a numerical framework that can be used to obtain efficient numerical solutions to the stochastic linear quadratic problem (or a generalized version of it) which consists of the state equation dy(t) = (Ay(t) + Bu(t)) dt + Cy(t) dW (t), over all possible controls u and subject to the condition that y satisfies the state equation (1). Operators R and G are bounded observation operators taking values in H, E denotes the expectation and y T = y(T ). A control process u * is called optimal if it minimizes the cost (2) over all control processes, i.e. for which it holds min u J(u) = J(u * ).
The corresponding trajectory is denoted by y * . Thus, the pair (y * , u * ) is the optimal solution of the problem (1)- (2) and is called the optimal pair. First of all, note that state equation (1) can be written in an equivalent abstract form asẏ (t) = Ay(t) + Bu(t) + Cy(t) ♦Ẇ (t), where ♦ denotes the Wick product andẆ (t) an H-valued white noise process. In order to preserve mean dynamics in (1), we represent the random perturbation as a stochastic convolution and obtain the Wick-version of the state equation. Using the Wick product instead of the usual pointwise multiplication we are able to establish a new approach for solving optimal control problems based on the application of the chaos expansion method. Since each square integrable stochastic process v on Gaussian white noise probability space has a unique chaos expansion representation in a Fourier-Hermite orthogonal polynomial basis, v = α∈I v α H α with deterministic coefficients v α , we are able to split the deterministic effects from the randomness and to reduce the original stochastic problem to a family of deterministic ones. The Wick product of two processes v and h is a process given in the Moreover, the relation E (v♦h) = E (v) · E (h), holds regardless of whether v and h are independent or not. The expectation of the Wick version of the state equation satisfies the corresponding deterministic optimal control problem. Recall, if at least one of the processes v, h is deterministic, then their Wick product and ordinary product coincide, i.e. v♦h = v · h. Historically, the Wick product first arose in quantum physics, as a renormalization operation, and later played an important role in many problems involving stochastic partial differential equations, in the theory of stochastic integration [19,25]. By introducing the Wick product ♦ in the considered stochastic problem, one uses an Itô-Skorokhod interpretation of the SPDE. The study of Wick versions of stochastic equations, both linear and nonlinear, together with the study of probabilistic properties of obtained solutions and the comparison with the properties of solutions of corresponding initial equations, can be found in [8,25,45,48,58].
In this work we combine known results of control theory for the SLQR problem with white noise analysis methods. Particularly, in order to characterize the optimal solution in terms of the polynomial chaos, we apply the chaos expansion method to (1)- (2). Since the control operator B is bounded, we apply the results from [27]. Then, we state the sufficient and necessary condition for the existence of the optimal solution of the considered SLQR problem in terms of the coefficients of the chaos and the solution of the Riccati equation. Theorem 3.1 and Theorem 3.2 are the main contribution of the paper.
Our approach can be generalized to different types of state equations. Always assuming that we are working with linear equations we can consider that operators in the equation are random, see Section 4.2. Another generalization is to consider a different type of noise. In particular, we will discuss in detail how the proposed approach can be extended if we are dealing with noise with memory, which is a special type of noise that is represented in terms of a stochastic integral [11], i.e. we consider the state equation of the forṁ with y(0) = y 0 , where δ represents the Itô-Skorokhod integral. Moreover, we analyze a problem with an even more general type of noise with memory, which is given bẏ with y(0) = y 0 , where δ t (Cy) is the integral process. Optimal control problems involving equations of type (3) and (4) have applications in economics and finance and have been recently studied in [11] using the stochastic maximum principle. Note that, since the argument of the stochastic integral is given as an action of C on y, the evolution equation (3) and (4), each respectively contains a memory property. The disturbance in (3) is a zero mean random variable for all t ∈ [0, T ], while in (4) the perturbation is given via a zero mean stochastic process. We point out that up to our knowledge there is no numerical algorithm for solving these problems. The method proposed in this paper is pioneer in this aspect too. Polynomial chaos which was first introduced by Wiener in 1938 [59], has recently been used in engineering applications to quantify evolving uncertainty in systems, see e.g. [18]. Using polynomial chaos, a stochastic system can be represented as a deterministic system with higher dimensionality, but the computational cost is reduced since extensive sampling is no longer required to capture the uncertainty. Only recently, there have been few works on the application of polynomial chaos in stochastic control of engineering systems (finite dimensional) [26,49,55]. Some very recent works in particular have been concerned with a polynomial chaos approach to linear control systems modeled by the stochastic LQR, see [16,17,57] White noise analysis was introduced by Hida and furter developed by many authors [25,45]. In order to build spaces of stochastic test and generalized functions, one has to use series decompositions via orthogonal functions as a basis, with certain weight sequences. Depending on the stochastic measure, this basis can be represented as a family of orthogonal polynomials. The classical Hida approach suggests to start with a nuclear space E and its dual E , such that and then take the basic probability space to be Ω = E endowed with the Borel sigma algebra of the weak topology and an appropriate probability measure P [24,25]. In this work we deal with a Gaussian white noise space. Thus, the underlying measure is the Gaussian measure. The corresponding orthogonal polynomial basis is constructed using the Hermite polynomials and any orthogonal basis of L 2 (R). In this case E and E are the Schwartz spaces of rapidly decreasing test functions S(R) and tempered distributions S (R) respectively.
The spaces of generalized random variables are stochastic analogues of deterministic generalized functions. They have no point value for ω ∈ Ω, only an average value with respect to a test random variable. Following the idea of the construction of S (R) as an inductive limit space over L 2 (R) with appropriate weights, one can define stochastic generalized random variable spaces over L 2 (Ω) by adding certain weights in the convergence condition of the series expansion. Several spaces of this type, weighted by a sequence q = (q α ) α∈I , denoted by (Q) −ρ , for ρ ∈ [0, 1] were described in [41]. Thus a Gel' fand triplet is obtained, where the inclusions are continuous. The most common weights and spaces appearing in applications are q α = (2N) α which correspond to the Kondratiev spaces of stochastic test functions (S) ρ and stochastic generalized functions (S) −ρ , and exponential weights q α = e (2N) α linked with the exponential growth spaces of stochastic test functions exp(S) ρ and stochastic generalized functions exp(S) −ρ . Note that, following ideas from financial mathematics, fractional white noise spaces could be constructed by replacing Brownian motion with fractional Brownian motion [25,41], or more general with Lévy processes .
The problem of pointwise multiplication of generalized stochastic functions in white noise analysis is overcome by introducing the Wick product. The most important property of the Wick multiplication is its relation to the Ito-Skorokhod integration [25]. In Section 3 we express the diffusion component of (1) in terms of the Wick product as well as in terms of the Itô-Skorokhod integral.
In white noise setting, the Skorokhod integral δ represents an extension of the Itô integral from a set of adapted processes to a set of non-adaptive processes. They coincide on the set of adapted processes. It is an adjoint operator of the Malliavin derivative D. Their composition is known as the Ornstein-Uhlenbeck operator R and is a self-adjoint operator on L 2 (Ω) that has the elements of the orthogonal basis as its eigenvalues. These operators are the main operators of an infinite dimensional stochastic calculus of variations called the Malliavin calculus [50]. Classes of elliptic and evolution stochastic differential equations (SDEs) that involve operators of the Malliavin calculus within white noise framework were recently studed in [46,40,44,38,48,58]. In [42,43] it was proved that the Malliavin derivative indicates the rate of change in time between the ordinary product and the Wick product. In this paper, we consider stochastic optimal control problems with stochastic perturbations given in an integral form. Moreover, we interpret multiplication as a Wick-type multiplication. By use of the Wiener-Itô chaos expansion representations of integrals we are able to achieve new results.
The chaos expansion methodology is a very useful technique for solving many types of SDEs [40,44,45]. The main statistical properties of the solution, its mean, variance, higher moments, can be calculated from the formulas involving only the coefficients of the chaos expansion representation [46,58]. Moreover, numerical methods for SDEs and uncertainty quantification based on the polynomial chaos approach have become very popular in recent years. They are highly efficient in practical computations providing fast convergence and high accuracy. For instance, in order to apply the stochastic Galerkin method, the derivation of explicit equations for the polynomial chaos coefficients is required. This is, as in the general chaos expansion, highly nontrivial and sometimes impossible. On the other hand, an analytical representation of the solution allows for all statistical information to be retrieved directly, e.g. mean, covariance function and even sensitivity coefficients, see [47,62] and references therein for a detailed explanation.
In order to illustrate our approach, we consider the stochastic linear quadratic problem (1)- (2). In [23,39], the disturbance in the control and the state is given by a convolution operator. In [44], the authors solve evolution equations involving stochastic convolution operators by combining the chaos expansion approach and the deterministic theory of semigroups in white noise framework. In this paper we will follow the ideas provided in [44] and apply the polynomial chaos expansion to the state equation, and obtain a system of infinitely many deterministic partial differential equations in terms of the coefficients of the state and the control. For each equation we set up a control problem which then gives rise to a system of infinitely many deterministic LQR problems. Solving each control problem, we find optimal coefficients for the state y and the control u. Summing up all obtained optimal coefficients in the chaos expansion representations of the state and the control we obtain the pairỹ andũ. We investigate the optimality of the solutionsỹ andũ and then formulate a necessary and sufficient condition for the existence of the optimal solution of the initial SQLR problem in terms of coefficients, Theorem 3.1 and Theorem 3.2.
In the first part of the paper, we deal with simple coordinatewise operators (deterministic operators) while in the second part of the paper we extend our ideas to the fully stochastic problem, i.e. we allow the operators in the state equation and the cost function to be random. Our approach "chaos expansion+optimization" can be applied to open loop control systems and in general to optimization problems in the same setting.
The paper is organized as follows: In Section 2, we briefly introduce basic concepts, results and notations on the infinite dimensional deterministic and stochastic LQR problems, solutions, white noise analysis and chaos expansions. Then, in Section 3 we apply polynomial chaos methodology to the state equation and set up linear quadratic control problems in terms of the coefficients and discuss the optimality of the solutions expressed in terms of the expansion of these coefficients. We prove the existence of the optimal control in the feedback form and give the optimality condition. Applications are included in Section 4, an example of a stochastic optimal control involving state equation with memory. We also discuss our approach for a general LQR with random coefficients and provide some application of an infinite dimensional control system from strucuture-acoustics. Finally, in Section 5 we discuss the numerical implementation of the proposed approach.
2. Basic concepts and notations. Let U and H be separable Hilbert spaces of controls and states respectively with norms · U and · H , generated by the corresponding scalar products. Let (Ω, F, P) be a complete probability space, (w t ) t≥0 be a real valued one-dimensional Brownian motion defined on (Ω, F, P). Let (F t ) t≥0 be the complete right continuous σ-algebra generated by (w t ) t≥0 . We assume that all function spaces are adapted to the filtration (F t ) t≥0 , i.e. we consider only F t -predictable processes. Let L 2 (Ω, P) = L 2 (Ω, F, P) be a Hilbert space of square integrable real valued random variables endowed with the norm F 2 L 2 (Ω,P) = E P (F 2 ) = E (F 2 ), for F ∈ L 2 (Ω, P), induced by the scalar product (F, G) L 2 (Ω,P) = E P (F G), for F, G ∈ L 2 (Ω, P), and E P denotes the expectation with respect to the measure P. From here onwards, we will omit the measure and write in short L 2 (Ω, P) = L 2 (P) and E for the expectation.
We denote by L 2 (Ω, U) a Hilbert space of U-valued square integrable random variables and by L 2 ([0, T ] × Ω, U) we denote a Hilbert space of square integrable F T -predictable U-valued stochastic processes u endowed with the norm Since U is a separable Hilbert space, the spaces L 2 ([0, T ]×Ω, U) and L 2 ([0, T ], L 2 (Ω, U)) are isomorphic [43]. Moreover, an H-valued Brownian motion is denoted by ) be a Hilbert space of F Tpredictable continuous H-valued stochastic processes y endowed with the norm 2.1. The SLQR problem: Existence of solution. The infinite dimensional SLQR optimal control problem on Hilbert spaces is given by the state equation (1), subject to the quadratic cost functional (2). The dynamics of the problem, the operator A, is deterministic and represents an infinitesimal generator of a strongly continuous semigroup (e At ) t≥0 on the state space H. Operators A and C are operators on H, while operator B is the operator acting from the control space U to the state space H. We take operator C to be linear and bounded. We assume operators R and G to be linear and bounded operators on the space W and Z respectively. We denote by D(S) the domain of a certain operator S, and by S * the adjoint operator of S.
The aim of the stochastic linear quadratic problem is to minimize the cost functional J(u) over a set of square integrable controls u ∈ L 2 ([0, T ] × Ω, U), which are adapted to the filtration (F t ) t≥0 .
The following theorem gives the conditions for the existence of the optimal control in the feedback form using the associated Riccati equation. For more details on existence of mild solutions to the SDE (1) we refer to [13] and for the optimal control and Riccati feedback synthesis we refer the reader to [27]. Then the optimal control u * of the linear quadratic problem (1)-(2) satisfies the feedback characterization in terms of the optimal state y * where P(t) is a positive self-adjoint operator solving the Riccati equatioṅ 2.1.1. Inhomogeneous deterministic LQR problem. Here we invoke the solution to the inhomogeneous deterministic control problem of minimizing the performance index subject to the inhomogeneous differential equation under the same assumptions on A and B. For the homogeneous problem, case f = 0, we refer to [34], and we refer to [36] where the inhomogeneous optimal control problem for singular estimate type systems was considered. It is enough to assume that f ∈ L 2 ((0, T ), H), to obtain the solution for the optimal state and control (x * , u * ). The feedback form of the optimal control for the inhomogeneous problem (6)- (7) is given by where P d (t) solves the Riccati equation for all v, w in D(A), while k(t) is a solution to the auxiliary differential equation with the boundary conditions P d (T ) = G G and k(T ) = 0. Cy(s) 2 H ds < ∞) = 1; (3) for arbitrary t ∈ [0, T ] and P-almost surely, it satisfies the integral equation An H-valued adapted process y(t) is a mild solution of the state equation and P-almost surely, it satisfies the integral equation Mild solutions are the limits of strong solutions. In the case of a deterministic state equation, i.e. for C = 0, a mild solution y ∈ L 2 ([0, T ]; H) can be written in the form Note that, under the assumptions of the Theorem 2.1, and given a control u ∈ L 2 ([0, T ]; L 2 (Ω, U)), i.e. g(t) = Bu(t), and the deterministic initial data y 0 ∈ H, there exits a unique mild solution y ∈ L 2 ([0, T ]; L 2 (Ω, H)) to the controlled state equation (1), cf. [13].

2.2.
White noise analysis and chaos expansions. In this section we recall briefly some basic facts from white noise analysis that are needed in our analysis.
, the family of Hermite polynomials and the family of Hermite functions. The family of Hermite functions forms a complete orthonormal system in L 2 (R) with respect to the Lebesgue measure. These functions are the eigenfunctions for the harmonic oscillator in quantum mechanics. The Hermite functions satisfy the recurrent formula and h n (x) = nh n−1 (x), for n ∈ N and h 0 (x) = 1, while for the Hermite functions the identity formula for derivatives holds. Moreover, for constants c and γ independent of n. Clearly, ξ n , n ∈ N belong to the Schwartz space of rapidly decreasing functions S(R), i.e. they decay faster than polynomial of any power. The Schwartz spaces can be characterized in terms of the Hermite basis in the following manner: The space of rapidly decreasing functions as a projective limit space with continuous inclusions.
From the Wiener-Itô chaos expansion theorem it follows that each random variable F ∈ L 2 (Ω) has a unique representation of the form The space spanned by {H α : |α| = k} is called the Wiener chaos of order k and is denoted by H k , k ∈ N 0 . Thus, H 0 is the set of constant random variables, i.e. for α = 0 we obtain the expectation of a certain random variable. The space H 1 consists of linear combinations of elements ω, · (for example Brownian motion and singular white noise are elements of the Wiener chaos of the first order chaos) and the space k j=0 H j is the set of random variables of the form p( ω, · ), where p is a polynomial of degree n ≤ k with real coefficients. This implies that each H k is a finite-dimensional subspace of L 2 (Ω). Moreover, where the sum is an orthogonal sum [25].
Remark 1. In this paper, the considered white noise space (Ω, F, P) is the Gaussian white noise space, where the measure P = µ is the Gaussian measure. The Fourier-Hermite polynomials (9) form an orthogonal basis of the Hilbert space L 2 (Ω, P) = L 2 (Ω, µ). The further analysis will also hold for other types of white noise spaces or fractional white noise spaces, for which the corresponding Hilbert space L 2 (Ω, P) has an orthogonal polynomial basis. For example, for Poisson measure P = ν, the Charlier polynomials form an orthogonal polynomial basis of the space L 2 (Ω, ν). Note here that there exists a unitary mapping between L 2 (µ) and L 2 (ν) [41]. In general, one can work with the Askey-scheme of hypergeometric orthogonal polynomials and the Sheffer system [56]. Therefore the presented analysis can be provided in the same manner in all these cases.
Let H be a real separable Hilbert space with the scalar product < ·, · > H , and let {e k } k∈N be one orthonormal basis in H. The space of H-valued square integrable random variables can be represented as One of the typical complications that arise in solving SDEs is the blowup of L 2 norms of processes, i.e. their infinite variance. Therefore, the weighted spaces in which the considered equation has a solution have to be introduced. For example, such spaces are the Kondratiev spaces (S) −ρ , ρ ∈ [0, 1] of generalized random variables, which represent the stochastic analogue of Schwartz spaces as generalized function spaces. The largest space of Kondratiev stochastic distributions is (S) −1 , obtained for ρ = 1. Now we introduce the Wick product ♦ of random variables. For F = α∈I f α H α and G = β∈I g β H β the element F ♦G is called the Wick product of F and G and is given in the form It is well known that the Kondratiev spaces (S) 1 and (S) −1 are closed under the Wick multiplication. The Wick product is a commutative, associative operation, distributive with respect to addition. In particular, for the orthogonal polynomial basis of L 2 (Ω) we have H α ♦H β = H α+β , for α, β ∈ I. Whenever F , G and F ♦G are integrable it holds E(F ♦G) = E(F ) · E(G), without independence requirement [25,43]. The ordinary product F · G of random variables F, G ∈ L 2 (Ω) is defined by using the multiplication formula

2.2.2.
Stochastic processes. Since a square integrable stochastic process is defined as a measurable mapping [0, T ] → L 2 (Ω), then by a generalized stochastic process we consider a measurable mapping from [0, T ] into a Kondratiev space (S) −1 . The chaos expansion representation of generalized stochastic process F follows from the Wiener-Itô chaos expansion theorem. A process F can be represented in the form where f α , α ∈ I are measurable real functions and there exists p ∈ N 0 such that for all t ∈ [0, T ] If H is a real separable Hilbert space, then the expansion (11) holds also for Hvalued stochastic processes, for f α ∈ H. Particularly, for F ∈ L 2 ([0, T ], H) ⊗ (S) −1 the condition (12) transforms to the following for p > 1 it follows that singular white noise is an element of the space (S) −1 , for all t ≥ 0, see [25]. It is integrable and the relation d dt w t =ẇ t holds in the distributional sense. Clearly, both Brownian motion and singular white noise are Gaussian processes.
Let {e k } k∈N be an orthonormal basis of H. Then H-valued white noise process is given in the formẆ In general, a chaos expansion representation of an H-valued Gaussian process, that belongs to the Wiener chaos space of order one is given in the form with real coefficients g ki . If the condition is fulfilled, then process G given in the form (14), belongs to the space L 2 ([0, T ] × Ω, H). If the sum (15) is infinite then the representation (14) is formal, and if additionally holds for some p ∈ N 0 , the process G, for each t, belongs to the Kondratiev space of stochastic distributions, see [41,45,54]. Throughout the paper, we work with Hilbert space valued stochastic processes. Thus, an H-valued stochastic process v, standard or generalized, has chaos expansion representation of the form where the coefficients v α satisfy a certain convergence condition of the form α∈I v α 2 H r α < ∞ for an appropriate family of weights {r α } α∈I . Note that the deterministic part of v in (16) is the coefficient v 0 (t), which is the (generalized) expectation of a process v.
The Wick product of two stochastic processes is defined in an analogous way as it was defined for random variables and generalized random variables (10), for more details see [40].

Operators.
Following [44], we now introduce two classes of operators that we are dealing with, namely coordinatewise and simple coordinatewise operators. An operator O is called a coordinatewise operator if it is composed of a family of operators {O α } α∈I , such that for a process v =

Stochastic integration and Wick multiplication.
For a square integrable process v that is adapted in the filtration (F t ) t≥0 generated by an H-valued Brownian motion (W t ) t≥0 , the corresponding stochastic integral T 0 v t dW t is considered to be the Itô integral I(v). When v is not adapted to the filtration, then the stochastic integral is interpreted as the Itô-Skorokhod integral. From the fundamental theorem of stochastic calculus it follows that the Itô-Skorokhod integral of a H-valued stochastic process v = v t (ω) can be represented as a Riemann integral of the Wick product of v t with a singular white noise where the derivativeẆ t = d dt W t is taken in sense of distributions [25]. Thus, for an H-valued adapted processes v the Itô integral and the Skorokhod integral coincide, i.e. I(v) = δ(v). Note that the Itô integral is an H-valued random variable, i.e. I : M 2 → L 2 (Ω). From the Wiener-Itô chaos expansion theorem it follows that there exists a unique family a α , α ∈ I such that the Itô integral can be represented in the chaos expansion form On the other hand, applying the property (10) to (17) we obtain a chaos expansion representation of the Skorokhod integral. Clearly, Thus, where v α (t) = k∈N v α,k e k (t) is the chaos expansion representation of v α in the orthonormal basis with coefficients v α,k =< v α , e k > H ∈ R and ω ∈ Ω. Combining (20) and (18) we obtain the coefficients a α , for all α ∈ I and α > 0 in the form As mentioned in Section 2. Hence we obtained the chaos expansion representation form of the Itô-Skorokhod integral. Therefore, we are able to represent the stochastic perturbation appearing in equation (1) explicitly. Note also that δ(v) belongs to the Wiener chaos space of higher order than v, see also [25,42].
Therefore, we say that a square integrable H-valued stochastic process v given holds. Then the Itô-Skorokhod integral of v is of the form (20) and we write v ∈ Dom(δ). Proof. Let v satisfies the condition (22). Then we have where we used (α + ε (k) )! = (α k + 1) α! and the estimate α k + 1 ≤ 2|α| for all for α ∈ I, k ∈ N.
Detailed analysis of domain and range of operators of the Malliavin calculus in spaces of stochastic distributions can be found in [43].
3. Chaos expansions approach. In this section we study the optimal control problem subject to the state equation and provide the main results of the paper. We assume that all the operators are simple coordinatewise operators and: Thus, the actions of the operators are given by Ay(t, ω) = α∈I Ay α (t) H α (ω), Bu(t) = α∈I Bu α (t) H α (ω) and Cy(t, ω) = α∈I Cy α (t) H α (ω), where such that for all α ∈ I the coefficients y α ∈ L 2 ([0, T ], H) and u α ∈ L 2 ([0, T ], U).
Since the operator C is a bounded linear operator on H while B is bounded from U to H, then C is a bounded operator on  H < ∞. Then, the optimal control problem (1)-(2) has a unique optimal control u * given in the chaos expansion form where P d (t) solves the Riccati equation (8), i.e.
and k(t) is a solution to the auxiliary differential equation with the terminal condition k α (T ) = 0 and y * = α∈I y * α H α is the optimal state. Proof. We divide the proof in several steps. First, we analyze the state equation and apply the chaos expansion method to its equivalent Wick version.
Due to the fundamental theorem of stochastic calculus, an integral of Itô type of an integrable H-valued stochastic process is equal to the Riemann integral of the Wick product of a process and H-valued singular white noise (13) where W (t) is a H-valued Brownian motion [25]. Therefore, the state equation can be written in standard differential form, on a class of admissible square integrable processes, aṡ y(t) = Ay(t) + Bu(t) + Cy(t)♦Ẇ (t), By applying the chaos expansion method to (25), we obtain a system of deterministic equations. Setting up a control problem for each equation we seek for the optimal control u and the corresponding optimal state y in the form (23). Thus, the goal is to obtain the unknown coefficients u α and y α for all α ∈ I.
We apply the chaos expansion method to transform the initial condition y(0) = y 0 , for a given H-valued random variable y 0 . Hence we obtain Since the chaos expansion in orthogonal polynomial basis {H α } α∈I is unique, we obtain a family of initial conditions for the coefficients of the state y α (0) = y 0 α , for all α ∈ I, where y 0 α ∈ H, α ∈ I. Note that, in case that the initial condition is deterministic y 0 ∈ H, then its chaos expansion representation have only one non-zero element, i.e. y 0 0 in the zeroth level. Next, we apply the chaos expansion method to the state equation (25). The process y is considered to be differentiable if and only of its coordinates are differentiable deterministic functions anḋ we refer to [44]. From the assumption (A2) and the property (19) for each y ∈ D(C) it follows where {e i } i∈N denote the orthonormal basis of functions in H. Then, by (A1) and (A3), the equation (25) can be written as Due to the uniqueness of the chaos expansion representations in orthogonal polynomial basis (9), the previous equation reduces to the system of infinitely many deterministic initial value problems: 2 • for |α| > 0: The system of equations (26), (27) is deterministic, and the unknowns correspond to the coefficients of the control and the state variables. It describes how the stochastic state equation propagates chaos through different levels. Note that for α = 0, the equation (26) corresponds to the deterministic version of the problem and the state y 0 is the expected value of y. The terms y α−ε (i) (t) are obtained recursively with respect to the length of α. The sum in (27) goes through all possible decompositions of α, i.e., for all j for which α − ε (j) is defined. Therefore, the sum has as many terms as multi-index α has non-zero components. Existence and uniqueness of solutions for the systems (26), (27) follows from the assumptions (A1), (A2) and (A3) made on the operators A, B and C. Now we set up optimal control problems for each α-level. Considering the deterministic version of the cost function, the problems are defined as: subject to y 0 (t) = Ay 0 (t) + Bu 0 (t), y 0 (0) = y 0 0 , and 2 • for |α| > 0: the control problem subject to which can be solved by induction on the length of multi-index α ∈ I.
In the next step of the proof we solve the family of deterministic control problems, i.e. we discuss the solution of the deterministic system of control problems (28) and (29). 1 • For α = 0 the state equation (26) is homogeneous, thus the optimal control for (26), (28) is given in the feedback form where P d (t) solves the Riccati equation (8). 2 • For each |α| > 0 the state equation (27) is inhomogeneous and the optimal control for (29) is given by where P d (t) solves the Riccati equation (8), while k(t) is a solution to the auxiliary differential equation (24) with the terminal condition k α (T ) = 0, as discussed in Section 2.1.1.
Summing up all the coefficients we obtain the optimal solution (u * , y * ) represented in terms of chaos expansions. Thus, the optimal state is given in the form and the corresponding optimal control where P d (t) is a simple coordinatewise operator corresponding to the deterministic operator P d and K is a stochastic process with coefficients k α (t), i.e. of the form K = α∈I k α (t) H α , with k 0 = 0.
In the next step we prove the optimality of the obtained solution. Under the assumptions of Theorem 2.1, the optimal control problem (1)-(2) is given in feedback form by with a positive self-adjoint operator P(t) solving the stochastic Riccati equation (5). Since the state equations (1) and (25) are equivalent, we are going to interpret the optimal solution (33), involving the Riccati operator P(t) in terms of chaos expansions. Thus, J(u * ) = min u J(u), holds for u * of the form (33). On the other hand, the stochastic cost function J is related with the deterministic cost function J by, We used the fact that {H α } α∈I is an orthogonal basis of the Hilbert space of square integrable random variables, i.e. E(H α H β ) = α! δ α,β , where δ α,β is the Kronecker delta symbol and also the fact that the norms and can be represented in terms of the coefficients of processes y and u. Thus i.e. the optimal control obtained via direct Riccati approach u * coincides with the optimal control obtained via chaos expansion approach α∈I u * α (t)H α (ω). Moreover, the optimal states are the same and thus the well-posedness of the solution of the optimal state equation obtained via chaos expansion approach follows.
As a final step in the proof, we provide the convergence of the chaos expansions in the optimal state. After applying the chaos expansions to the original state equation we obtained the system of deterministic problems (26) and (27). For each state equation in this system we formulated an optimal control problem for which the solution has the feedback form (30) and (31). The set of optimal controls for the resulting system were then used to determine the set of optimal states via the system of equations with the initial conditions y α (0) = y 0 α , for all α ∈ I. We assumed in (A1) that the operator A is an infinitesimal generator of a strongly continuous semigroup {S t } t≥0 = (e At ) t≥0 such that e At L(H) ≤ M e θt holds for some positive constants M and θ. Since the operators B, B * and P d are deterministic and bounded, the operator BB * P d is also bounded and thus A + BB * P d is an infinitesimal generator of a strongly evolution (T t ) t≥0 such that for all t ≥ 0.
The importance of the convergence result can be seen in its applications for the error analysis that arises in the actual truncation when implementing the algorithm numerically.
3.1. Characterization of optimality. The optimality of our approach (34) can be characterized in terms of the solution of the stochastic Riccati equation (5). The following theorem summarizes our result.
Theorem 3.2. Let conditions (A1)-(A4) hold. Assume that y 0 is either deterministic or a square integrable H-valued random variable, i.e. it holds E y 0 2 H < ∞ and assume P is a simple coordinatewise operator that corresponds to operator P .
Then, the solution of the optimal control of the problem (1)-(2) obtained via chaos expansion (32) is equal to the one obtained via Riccati approach (33) if and only if hold for all t ∈ [0, T ].
Proof. Let us assume first that (32) is equal to (33), then The difference between P(t) and P d (t) is expressed through the stochastic process K, which comes from the influence of inhomogeneities. Assuming that P is a simple coordinatewise operator that corresponds to operator P , we will be able to see the action of stochastic operator P on the deterministic level, i.e. level of coefficients. Thus, for y given in the chaos expansion form (23) and Since k 0 (t) = 0 it follows P (t) = P d (t), for t ∈ [0, T ] and for |α| > 0 (P (t) − P d (t)) y * α (t) = k α (t), such that (24) with the condition k α (T ) = 0 holds. We differentiate (37) and substitute (24), together with (5), (8) and (27). Thus, after all calculations we obtain for |α| = 0 (P (t) − P d (t)) y * 0 (t) = 0 and for |α| > 0 Note that assuming (36) and P is a simple coordinatewise operator that corresponds to operator P , we can go backwards in the analysis and prove that the optimal controls (33) and (32) are the same.
The condition (36) for |α| = 1, i.e. α = ε (j) , j ∈ N reduces to the condition C P (t) C y * ε (j) (t) = P (t)(Cy 0 (t) · e j (t)), while for |α| = 2 it reduces to one of the following situations: for α = 2ε (j) , j ∈ N it becomes C P (t) C y * 2ε (j) (t) = P (t)(Cy * ε (j) (t) · e j (t)), and for α = ε (j) + ε (k) , j, k ∈ N, j = k it becomes C P (t) C y * 2ε (j) (t) = P (t)(Cy * ε (j) (t) · ξ k (t) + Cy * ε (k) (t) · e j (t)) and so on. The recurrence involved in (36), represents a memory property in the noise. This concept has been recently studied in [11]. In the next section we study a control problem with a state equation involving noise with memory. Remark 2. The assumptions (A1)-(A4) from Theorem 3.2 hold for many applications and they are standard in optimal control [34,35]. On the other hand, due to the fact that the Riccati equation (5) is deterministic, its solution is naturally related to P, where P is a simple coordinatewise operator (Section 2.2.3). The latter is not necessary true for stochastic Riccati equations (49) (Section 4.2), there P might not necessary be a simple coordinstewise operator.
where D is a simple coordinatewise operator related to a bounded operator D. Similar to (25), equation (38) can be written aṡ Therefore, by applying the chaos expansion method, one obtains the following deterministic system of equations: a) for |α| = 0: y 0 (t) = Ay 0 (t) + Bu 0 (t), y 0 (0) = y 0 0 , b) for |α| > 0: Then, the optimal states have the form: 1 • for |α| = 0: y 0 (t) = (A − BB P )y 0 (t), y 0 (0) = y 0 0 , 2 • for |α| > 0: Note that, our approach is optimal in this case as well. On the other hand, a direct Riccati approach will lead to an optimal state given by where P (t) is the solution of (Ṗ + P A + A P + C P C + R R − (B P + D P C) (I + D P D) −1 (B P + D P C)v, w = 0 for all v, w in D(A).
From the computational point of view, our approach has a lot of potential as it avoids solving (39), and will be explored in future work. Finally, we point out that a convergence framework for the stochastic linear problem in the general framework of singular estimates has been developed recently in [39]. 4. Applications. In this section we extend the results of Section 3 to optimal control problems with state equations involving memory noise. We also consider the state equations with random coefficients following the framework of [20,21] and give an example of a control system from structure acoustics. 4.1. State equation with memory noise. We apply the introduced method to optimal control problems involving noise with memory. Particularly, we study the SLQR problem with the state equation of the forṁ subject to the cost functional J(u) given by (2). Here δ denotes the Itô-Skorokhod integral. In the same setting, we can also consider the state equation in more general form y (t) = Ay(t) + Bu(t) + δ t (Cy(t)), where T ] is the integral Itô-Skorokhod process. For t = T , δ = δ T . Note that solving the problem for δ, the problem for δ t is straight As discussed before, the fact that y appears in the stochastic integral implies that the noise contains a memory property [11]. The disturbance δ is a zero mean random variable for all t ∈ [0, T ], while δ t is a zero mean stochastic process.
There exists an operatorC such that there is a one to one correspondence betweeñ C ♦ and δ • C, i.e.C ♦ y = δ(Cy) . Therefore, (40) can be written aṡ i.e. there is a correspondence between the Wick form perturbation and the Skorokhod integral representation [44].
In the following, we apply the chaos expansion approach for solving the SLQR problem related to (40) and compare the solution to the actual solution obtained by a direct Riccati approach applied to equation (42). Since there is no explicit form ofC, the suggested polynomial chaos approach for solving the problem is quite promising.
Similarly as in the previous section, we apply the chaos expansion method to (40) and thus transform the equation to a corresponding infinite family of deterministic equations. We look for the optimal coefficients u α and y α , α ∈ I. Then, we obtain the system of deterministic optimal control problems 1 • for α = 0: the control problem subject to y 0 (t) = Ay 0 (t) + Bu 0 (t), y 0 (0) = y 0 0 , 2 • for |α| > 0: the control problem where (Cy α−ε (i) (t)) i denotes the ith component of Cy α−ε (i) , i.e. a real number, obtained in the previous inductive step. The sum is finite with as many summands as multi-index α has non-zero components.
For |α| = 0 the state equation in (43) is homogeneous and the optimal control for the state equation is given in the feedback form (30), with positive self adjoint operator P d that satisfies the Riccati equation (8). On the other hand, for each |α| > 0 the state equation in (44) is inhomogeneous with the inhomogeneity term i∈N (Cy α−ε (i) ) i . Thus, the optimal control is given by (31), where k α are the solutions to the auxiliary differential equations for |α| > 0, with the final condition k α (T ) = 0. Summing up all the coefficients, obtained as optimal on each level α, the optimal state is then given in the form The optimal state in each level is given by: 1 • for |α| = 0, i.e. α = (0, 0, ...) = 0: y 0 (t) = (A − BB P d (t)) y 0 (t), y 0 (0) = y 0 0 , 2 • for |α| ≥ 0: where k α are solutions of (24). Thus, the optimal state computed by chaos expansion corresponds tȯ where BB P d is a simple coordinatewise operator given through the deterministic operator (BB P d ), where P d is the solution of (8) and K is a stochastic function given by the expansion where k 0 = 0 and k α are given by (45) respectively. Equation (46) represents the optimal state when we control each level of the chaos expansion. On the other hand, a direct Riccati approach for the SLQR problem related to (42) or (41), up to our knowledge has not been studied in the literature.
Finally, we point out that the convergence of the chaos expansions can be established using a similar argument to the one described in the proof of Theorem 3.1.

4.2.
Random coefficients. Let us consider a stochastic linear quadratic control problem of the form subject to the performance index whereĀ is independent of ω and is the infinitesimal generator of a C 0 -semigroup, A , B, C, R and G are allowed to be random. The optimal control is given in feedback form in terms of an operator P(t) solving the backward stochastic Riccati equation with P 0 (T ) = G G. The two operators P and Q are unknown, and Q is sometimes referred to as a martingale term, see [20,21] and references therein.
If the operators involved have chaos expansion representations, the same ideas can be applied to fully stochastic problem. Let us consider the operatorĀ to be a coordinatewise operator, i.e. an operator composed of a family of operators {Ā α } α∈I , whereĀ α are infinitesimal generators of C 0 -semigroups defined on a common domain that is dense in H and For the case whenĀ is independent on randomness, only nonzero operator in the family {Ā α } α∈I is obtained for |α| = 0, i.e.Ā 0 =Ā andĀ α = 0 for all |α| > 0.
Applying the polynomial chaos method to (47), we obtain: a) for |α| = 0, i.e. α = (0, 0, ...) = 0: b) for |α| > 0: Setting up control problems at each level for (50) and (51), as explained in Section 4.1, in analogy to (46) the optimal state is given by whereP is a coordinatewise operator composed by the family {P α } α∈I . The operators P α correspond to the solution of the Riccati equation for the coefficientsĀ α , A α , B α , C α , R α and G α , i.e. it holdṡ for each α ∈ I. Note that (52) is a deterministic Riccati equation for each α. Alsō K is a H-valued stochastic process given bȳ where k 0 = 0 and k α , for |α| ≥ 1 are given by Equations (53) have a final condition equal to zero. Therefore, in order to control the system (47)-(48) we control each level through the chaos expansion. This implies solving a deterministic control problem at each level. Although theoretically we have to solve all these problems, numerically we can solve (m+p)! m!p! problems in order to achieve convergence. The value of p is in general equal to the number of uncorrelated random variables in the system and m is typically chosen by some heuristic method [46,58,62].

4.3.
A specific example from SPDE control. The approach outlined in this paper can be applied to a large class of systems in engineering which are mathematically modeled by partial differential equations. Control problems with stochastic coefficients also arise naturally in mathematical finance. In particular, the linear quadratic optimal control problem with stochastic coefficients and the corresponding backward stochastic Riccati equations (BSREs) have been extensively studied in the finite-horizon and finite-dimensional case [9,10,28,29,30,31,52,53]. Note that our approach is also valid for finite-dimensional systems since the polynomial chaos method can be applied to systems governed by random matrices.
As an example, we include a control system from structure acoustics which has been well studied in the deterministic setting [2,3,4,37]. The system consists of an acoustic chamber with piezoelectric control mechanism applied to the flexible wall of the chamber. Mathematically, the system is modelled by an open region Ω ⊂ R 3 with boundary ∂Ω = Γ 0 Γ 1 representing a rigid wall and a flexible wall respectively. The acoustics in the chamber are modelled by a wave equation in the variable z which denotes acoustic pressure dz t = c 2 ∆z dt + (∇z + z t + w + w t ) dW t on Ω × [0, T ], where c is the speed of sound and W t is a one dimensional Wiener process on a complete probability space. On the other hand, the dynamics of the elastic wall Γ 1 , are modelled by a damped second order equation in the displacement variable w dw t + ∆ 2 w dt + ρ∆w t dt = ρ 1 z t dt + J a j u j δ ξj dt + (∇w + w t + z + z t ) dW t in Γ 1 × [0, T ], where ρ, ρ 1 > 0. The piezoelectric control mechanism is mathematically represented by the derivatives of Dirac delta functions supported at curves ξ j with the controls u ∈ R J while a j (x) are smooth functions on Γ 1 . The acoustic pressure satisfies the boundary conditions We consider the system subject to the initial conditions z 0 ∈ H 1 (Ω), z 1 ∈ L 2 (Ω) and w 0 ∈ H 2 (Γ 1 ) ∩ H 1 0 (Γ 1 ) and w 1 ∈ L 2 (Ω). The multiplicative noise in the system is captured by a bounded operator C on the finite energy space. The control objective is to minimize the functional J(z, z 1 , w, w 1 , u) = e At Bu H ≤ c|u| t 3/8+ , for all u ∈ R J [2]. There has been many works in the literature addressing Riccati feedback synthesis of such control systems known as singular estimate control systems in the deterministic case [36] and references therein, and more recently in the stochastic case [22,23]. The possible extension and application of the polynomial chaos approach to this class of control systems which typically involve boundary or point control of systems of coupled hyperbolic-parabolic partial differential equations with noise, would be numerically very promising. 5. Numerical approximation. Numerical methods for stochastic differential equations and uncertainty quantification based on the polynomial chaos approach have become popular in recent years. They are known as stochastic Galerkin methods and they are highly efficient in practical computations providing fast convergence and high accuracy [62]. In the following, we summarize the numerical framework proposed in this paper for solving the SLQR problem using polynomial chaos expansion.
First of all, we use a finite dimensional approximation of the Fourier-Hermite orthogonal polynomials {H α } α∈I [62]. This is standard in the so-called stochastic Galerkin methods. Then, we set up deterministic control problems for each level (28) and (29). We solve the control problem via Riccati approach and compute the optimal state for each level. We then compute the approximate optimal state and optimal control for the original problem. The main steps are sketched in the following Algorithm: Main steps of the stochastic Galerkin method for SLQR problems 1: Choose finite set of polynomials Hα and truncate the random series to a finite random sum. 2: Set up deterministic control problems for each level of the chaos expansion (26) and (27). 3: Compute the optimal control via Riccati approach for each level. 4: Compute the optimal state for each level. 5: Compute the approximate statistics of the solutions from obtained coefficients. 6: Generate Hα and compute the approximate optimal state and optimal control.
We denote by I m,p the set of α = (α 1 , ..., α m , 0, 0, ...) ∈ I with m = max{i ∈ N : α i = 0} such that |α| ≤ p. As a first step, we represent y in its truncated polynomial chaos expansion form y, i.e. we approximate the solution with the chaos expansion in ⊕ p k=0 H k with m random variablesỹ(t, ω) = α∈Im,p y α (t) H α (ω); the previous sum has P = (m+p)! m!p! terms. Once the coefficients of the expansionỹ are obtained, we are able to compute all the moments of the random field, e.g. the expectation Ey = y 0 and the variance of the solution V ar(ỹ) = α∈Im,p α!ỹ 2 α . We would like to underline that the polynomial chaos expansion converges quite fast, i.e even small values of p may lead to very accurate approximation. The error generated by the truncation of the chaos expansion, in L 2 (Ω, H) is for x ∈ D. Note that if instead of a Gaussian random variable, a stochastic generalized function is considered, i.e. when the coefficients are singular, the error E 2 → 0 converges in a certain space of weighted generalized stochastic functions. Finally, we would like to point out that efficient solvers for differential Riccati equations have been proposed in recent years [1,5,6,7,33]. The potential of this approach is notable. An efficient numerical implementation is work in progress and will be reported somewhere else.