AN INFINITE TIME HORIZON PORTFOLIO OPTIMIZATION MODEL WITH DELAYS

. In this paper we consider a portfolio optimization problem of the Merton’s type over an inﬁnite time horizon. Unlike the classical Markov model, we consider a system with delays. The problem is formulated as a stochastic control problem on an inﬁnite time horizon and the state evolves according to a process governed by a stochastic process with delay. The goal is to choose investment and consumption controls such that the total expected discounted utility is maximized. Under certain conditions, we derive the explicit solutions for the associated Hamilton-Jacobi-Bellman (HJB) equations in a ﬁnite dimensional space for logarithmic and power utility functions. For those utility functions, veriﬁcation results are established to ensure that the solutions are equal to the value functions, and the optimal controls are derived, too.

1. Introduction. We consider a stochastic portfolio management problem on an infinite time horizon taking into account the history of the portfolio performance. Investor's portfolio consists of a risky and a riskless asset. In a classical Merton's type model a Markovian stochastic process such as geometric Brownian process is used to describe the price of the risky asset. In such a model past information is irrelevant and decisions are made only on the basis of current information. A substantial amount of work has been done on models with such settings. In Bielecki-Pliska [1], Fleming-Sheu [14], continuous time portfolio optimization models of Merton's type were considered, where the mean returns of individual asset categories are explicitly affected by underlying economic factors such as dividends and interest rates. The case of a Merton-type model with consumption and the interest rate, which varies in a random, Markovian way, was considered in the first author's other papers (see Fleming-Pang [11] and [12], Pang [27] and [28]). In some other extensions of the model, stochastic volatility is taken into consideration (see Fleming-Hernandez-Hernandez [10], Fouque-Papanicolaou-Sircar [15]).
However, in real world, the historic performance of the risky asset influences the decision of investors. As we know, investors tend to look at the moving average or exponential moving average for a stock before they make the investment decision. A stock with current price lower than its exponential moving average may signal the downward trend for the stock price, therefore it will scare away some investors. Due to the weaker demand, the price may go down further. On the other hand, if the current price is higher than its exponential moving average, it may signal a upward trend, so it will attract more investment. Due to the higher demand, the price may go up further. Therefore, it is motivated to consider a stochastic portfolio optimization model with historic information, or delays.
In particular, we want to consider two delay variables, Y (t) and Z(t): where X(t) is the total wealth. More details about the model will be given in Section 2. As we can see, Y (t) is the exponential moving average of the total wealth, and Z(t) is the historical wealth. A special case would be λ = 0, and now Y (t) is just the moving average. Typically, higher historical wealth (in terms of Y (t), Z(t)) usually implies the investor has invested heavily on stocks rather than the risk-free asset, therefore, due to the good performance, the investor will tend to allocate more on stocks, and that will increase the stock demand. This type of models with delay can be used to describe a stochastic system with delayed information. Such problems with delay arise in many situations. In Federico [9], an optimal control problem arising in the management of a pension fund with dynamics governed by a stochastic differential equation with delay is considered. In Federico-Goldys-Gozzi [8], the authors consider a class of optimal control problems with state constraints, where the state equation is a differential equation with delays. Koivo [18] studied optimal control of linear stochastic systems that have feedback loop. In Elsanousi-Øksendal-Sulem [7], a stochastic optimal harvest problem has been considered. Problems with delay also arise in modeling optimal advertising under uncertainty (see  and Gozzi-Marinelli-Savin [17]). Some early works on optimal control for systems with delays can be found in Kolmanovskii-Maizenberg [19], Kolmanovskii-Shaikhet [20], Lindquist [23,24] and the references therein.
In this paper, we consider a portfolio optimization problem for stochastic systems with delay variables given by (1) on an infinite time horizon. The problem is formulated as a stochastic control problem. The equation of the state variable X(t) is given by (10) and (11) and the objective function is given by (24). The goal is to choose the optimal investment control k(t) and the consumption control c(t) to maximize the objective function to obtain the value function defined by (25). In this paper, we use two methods, including a newly developed functional Ito's formula, to derive the associated Hamilton-Jacobi-Bellman (HJB) equation. The functional Ito's formula was first initiated by Dupire [6] and was later studied in Cont-Fournié [5]. Details to be given in Section 3.
Some related problems have been studied in Chang-Pang-Yang [4] and Pang-Hussain [29]. In [4], the authors have investigated a model with delay using utility functions of HARA(Hyperbolic Absolute Risk Aversion) type on a finite time horizon. In [29], the authors consider a model with delay using utility functions of exponential type and logarithmic type on a finite time horizon. Here in this paper, we consider the portfolio optimization problem with utility function of HARA type and logarithmic type on an infinite time horizon. Unlike in the finite time horizon cases considered in [4], [29], now the HJB equation is independent of time variable s, and there are no terminal conditions or boundary conditions. So the solutions are totally different. Further, to establish the verification results, we will need to estimate the growth rate of some expectations (see the proofs of Theorem 4.1, and Theorem 5.1), which give us more challenges compared to the finite time horizon case.
On the other hand, some researchers have considered stochastic systems with delays given by the following system: , and it is the segment of the path from t − h to t, ϕ ∈ C[−h, 0] is the initial path and u is the control in some admissible control space Π. The goal is to choose a control u to minimize the cost function: The value function is given by For this type of control problems, the idea for the derivation of associated Hamilton-Jacobi-Bellman (HJB) equation was presented in Mohammed [25] and [26]. Some related works can be found in Chang-Pang-Pemy [2], [3] and the references therein. The solution of a stochastic control problem with delay is assumed to depend on the initial condition ϕ, which is in an infinite dimensional space C[−h, 0], the space of all continuous functions on [−h, 0]. However, if the system only depends on the delay through process Y (t) and Z(t) defined by (1), it is possible to obtain a solution in a finite dimensional space.
The paper is organized as follows. In Section 2, the problem is formulated as a stochastic control problem. Some preliminary results about the properties of the state variable X(t) and the delay variables Y (t), Z(t) are given. In Section 3, using the dynamic programming principle, we heuristically derive the Hamilton-Jacobi-Bellman equation for the value function in terms of the initial variables x = X(0), y = Y (0). In Sections 4-5, we derive the explicit solutions for log utility and power utility (HARA utility with γ = 0), respectively. The optimal control polices and the verification results are established correspondingly. We conclude our paper in Section 6.
2. Problem formulation and some preliminaries. Consider an investor's portfolio consisting of a risky asset and a riskless asset. The riskless asset earns the investor a fixed interest rate r > 0. For example, we can treat the money in a bank as the investment on the riskless asset. We assume that the investor can freely move her money between two assets at any time and her consumption comes from the riskless asset.
Let K(t) be the amount invested in the risky asset and L(t) be the amount invested on the riskless asset. The net wealth is given by X(t) = K(t) + L(t). We consider a situation where the performance of the risky asset has memory. We assume that the performance of the risky asset depends on the following delay variables Y (t) and Z(t): where λ > 0 is a constant and h is the delay parameter. Let {B(t), t ≥ 0} be a standard one-dimensional standard Brownian motion defined on a complete filtered probability space (Ω, F, P ; F), where F = {F t , t ≥ 0} is the P -augmented natural filtration generated by the Brownian motion {B(t), t ≥ 0}. We assume that K(t) and L(t) follow the stochastic differential equations: where µ 1 , µ 2 , µ 3 and σ are positive constants, I(t) is the investment rate on the risky asset at t, and C(t) is the consumption rate. The equation for X(t) follows by using X(t) = K(t) + L(t): The initial condition is the information about X(t) for t ∈ [−h, 0]: |ϕ(θ)|.
Let L 2 (Ω, J) be a Banach space for all (F, B(J))-measurable maps Ω → J that are in L 2 in the Bochner sense, where B(J) is the Borel σ− field on J. For any φ ∈ L 2 (Ω, J), the Banach norm is given by where the norm || · || is given by (7). To describe the allocation between the risky asset and the riskless asset, we now treat K(t) and C(t) as our control variables. The state variables are X(t) and Y (t). As we can see in equation (5), the change of the wealth process X(t) depends on the delay variable Y (t). For technical reasons, we modify the model described in (5) and consider the following model Remark 1. If we assume that K(t) > 0 almost surely, we can use the following delay variables Y (t) and Z(t): instead of (2), so we can reach (9).
Instead of using K(t) and C(t), we use c(t) = C(t) X(t) and k(t) = K(t) X(t) as our consumption and investment controls, respectively. Using Initial condition is given by where ϕ ∈ J and ϕ(θ) > 0, ∀θ ∈ [−h, 0]. For functional differential equations we use the following conventional notation: Then, for any Since ψ t ∈ J, its norm is given by the sup norm: Using the above notation, initial condition (11) can be written as Definition 2.1 (Admissible Control Space). Let Π denote the admissible control space. A control policy (k(t), c(t)) is said to be in the admissible control space Π if it satisfies the following conditions: where Λ > 0 is a constant.
Here we assume that the investment control k(t) is square integrable (equation (14)), to ensure that the Ito's integral T 0 k(t)dB(t) is well defined. On the other hand, the consumption control c(t) only appears in the drift part, not the diffusion part, so we do not need a similar condition for c(t).
We have the following result: For any control (k(t), c(t)) ∈ Π, the equation (10) with initial condition (11) has a unique strong solution X : [−h, ∞) × Ω → R. Furthermore, for t ∈ [0, ∞), X t ∈ J and for any positive integer n, we have where C n > 0 is a constant.

TAO PANG AND AZMAT HUSSAIN
Proof. Let ϕ 2 be the L 2 norm of ϕ as given in (8). Then, by virtue of this lemma is a corollary of Theorem I.2 of Mohammed [26].
We have the following result: The solution X(t) of the system (10) -(11) satisfies Proof. By the definition of Y (t), Z(t), it is sufficient to show that For a given initial condition Define a stopping time τ as the first time X(t) hits zero: To show that X(t) > 0, a.s., it is sufficient to show that for the same ϕ > 0 as in (11). The solution is given as Apparently, we haveX Moreover, using Ito's formula, we get Consider a new stochastic process R(t) defined by Then, we can get By the definition of Y (t), Z(t) and τ , it is easy to see that Since By the definition of τ , if τ < ∞, we must have X(τ ) = 0, which is a contradiction with the above inequality. So we must have that This completes the proof. The utility function U (C) is defined based on the consumption rate. The problem under consideration is a portfolio optimization problem on an infinite time horizon with the objective function given by Then the value function is given by Note that V (ϕ) is a functional defined on an infinite dimensional space C[−h, 0]. We turn V into a function defined on a finite dimensional space as follows Further, we assume that the value function V only depends on (x, y) i.e.
The reason of this assumption will be given in Lemma 3.3.

3.
Hamilton-Jacobi-Bellman equation. We will need some kind of Ito's formula with respect to the function of the delay variable Y (t). There are two methods for this purpose. One is the traditional method and the other one is to use the functional Ito's formula. Although we can reach the same result via both methods, by using the functional Ito's formula, we can also establish a necessary condition that the HJB equation can be derived in a finite dimensional space (see Lemma 3.3). First we use the traditional method to derive the Ito's formula for functions of (X(t), Y (t)). Let f ∈ C 2,1 (R 2 ) and define Lemma 3.1 (Ito's Formula). Consider the system given by (10)-(11). We have where where x, y, z, k and c are evaluated as Proof. The idea of the proof is very similar to that of Lemma 2.1 in Elsanousi-Øksendal-Sulem [7]. We repeat it here to make the paper self-contained.
Applying classical Ito's formula to G = f (X(t), y(X t )), the result follows.
On the other hand, we can use the method of functional Ito's formula. Recall that we use X(t) to denote the current value and we use X t : [−h, 0] → R to denote the path of X(t) from t − h to t. For a functional f (X t ) of X t , we have the following functional Ito's formula: where The above derivatives and the functional Ito's formula was initiated by Dupire [6] and was later studied in Cont and Fournié [5]. Now let us consider the following functionals: where φ(θ) is a smooth function with a continuous first order derivative φ (θ).
Proof. This lemma has been proved in Pang-Hussain [29]. We put the proof here so that this paper is self-contained. From the definitions, it is easy to see that ∂ x y(X t ) = 0, and ∂ xx y(X t ) = 0. On the other hand,

TAO PANG AND AZMAT HUSSAIN
Thus, it is easy to verify that By virtue of the functional Ito's formula (31), we can get (38). Take φ(θ) = e λθ , then we can get the initial delay variables defined by (26). It is easy to check that Therefore, we can get As we can see, we have derived the same result (30) as in Lemma 3.1.
In Chang-Pang-Yang [4] and other papers, such as Elsanousi-Øksendal-Sulem [7] and Larssen-Risebro [22], it is simply assumed that V does not depend on z but the reason was not given. The following lemma explains why this is a necessary condition. Proof. It is easy to check that As we can see, ∂ t z(X t ) usually does not exist because the path of Brownian motion is not differentiable. Therefore, if the value function V depends on z, it is impossible to derive the HJB equation in a finite dimensional space. From the above lemma, we can see that to derive the HJB equation for the value function V in a finite dimensional space, it is necessary that V does not depend on z.
Now we assume that the value function V only depends on (x, y) as given by (27). We will derive the HJB equation satisfied by the value function V (x, y) heuristically. To derive the HJB equation, we need the following dynamic programming principle.
Proof. The proof is similar to the proof of Theorem 4.2 of Larssen [21] and we omit it here. Now we can use the dynamic programming principle given by Lemma 3.4 to heuristically derive the HJB equation. In particular, if we assume that V (x, y) is smooth enough, then we can use the Ito's formula to get where G k,c is given by (29). Integrate it over [0, T ], and we can get Assume that On the other hand, from the equation (39), we can get Together with equation (41), we can get the following HJB equation: Since G k,c is given by (29), we can write the HJB equation as the following: So we have heuristically derived the HJB equation for our stochastic control problem. For details and general theory about dynamic programming principle and HJB equations, please see Fleming-Rishel [13] or Yong-Zhou [30].
In this paper, we will consider utility functions of logarithmic type and HARA type. For each utility function, we will find an explicit solution of the corresponding HJB equation and we will verify that the solution is equivalent to the value function by establishing the verification theorem. The optimal control polices will be derived, too. The optimal control problem is then solved with the explicit value function and the optimal controls.
We want to point out that our goal is to solve the stochastic control problem to get the value function and the optimal controls while the HJB equation we derived heuristically just serves as an intermediate vehicle to find the explicit form of the value function and the optimal control. Therefore, we do not need to formally show that the value function is a classical or viscosity solution of the above HJB equation. Further, we do not need to show that the HJB equation has a unique solution, either.
4. Logarithmic utility function. We consider a logarithmic utility function given by U (cX) = log cX .
Now the HJB equation (43) becomes The candidates for optimal controls are Substituting k * and c * in (45), we obtain To cancel the items involving z, we assume that and we look for the solution of the form where η 1 , η 2 are two constants to be determined. It is easy to see that By plugging (49) and (50) into (47) we get Assume that and µ 3 e λh (r + µ 3 e λh ) = µ 2 − µ 3 λe λh .
Then we can plug (52) into (51) and use (53) to cancel u. The explicit formula for η 2 is Therefore (45) has a solution where η 2 is given by (54). The optimal investment and consumption control policies are Now we verify that V (x, y) is the maximum expected discounted utility and k * , c * are the optimal policies. Theorem 4.1 (Verification Theorem). Assume that condition (53) holds and let V (x, y) be given by (55). Then V (x, y) ∈ C 2,1 (R × R) and it is a solution of (45) such that (57) Moreover, we have (a) V (x, y) ≥ J(x, y; k, c) for any admissible control process (k(t), c(t)) ∈ Π.
Proof. By the construction of the function V (x, y), it is easy to check that V (x, y) ∈ C 2,1 (R × R) and it is a solution of (45). Next we verify the condition (57). Using (55), we have Using the definition of admissible control space Π, we have Therefore, we have Then we have, Thus condition (57) is verified.
Let G k,c be the generator of the process (X(t), Y (t)) for any control process (k(t), c(t)) ∈ Π. Then by using Ito's rule Integrating above on [0, T ] and also noting that V (x, y) is classical solution of (45), we have By virtue of (57), Using (55) Let Then using (53), we have Using Ito's rule, we have Since (k(t), c(t)) ∈ Π, it is easy to verify that T 0 σk(t) X(t) S(t) dB(t) is a martingale. Therefore, we can get For any (k(t), c(t)) ∈ Π, by virtue of (15) and (16), we have Then from (62), we can get lim T →∞ e −βT E[log(S(T ))] = 0.
Thus, we have Combined with (58), this implies (a). Now assume that k * , c * are given by (56). We can easily verify that they are F t -measurable. In addition, from Lemma 2.3, we know that X(t) > 0, a.s., so we can get that k * (t), c * (t) are well defined and c * (t) ≥ 0. In addition, it is not hard to check that (15) and (16) are true for k * (t) and c * (t). Finally, since X(t) > 0 and k * (t) is well defined, we can get that P r(k * (t) < ∞) = 1, ∀t ≥ 0. Therefore, we can get P r T 0 (k * (t)) 2 dt < ∞ = 1, ∀T > 0.
So we can get that (k * (t), c * (t)) ∈ Π. Moreover, the equation for X(t) now is Let Using assumption (53), we have The solution is . (66) Using (k * (t), c * (t)) as controls, similar to (58), now we can get By virtue of (55) and (66), we can get Then it is easy to show that Hence we have V (x, y) = J(x, y; k * , c * ). Therefore (b) is proved. This completes the proof. 5. HARA utility function. In this section we consider HARA utility function The HJB equation (43) can now be written as The candidates for optimal controls are Substituting k * and c * in (71), we obtain Suppose solution is of the form where u ≡ x + µ 3 e λh y. Now we have By substituting (74) and (75) in (73) we get Assume that µ 3 e λh (r + µ 3 e λh ) = µ 2 − µ 3 λe λh .
(77) Then the explicit formula for η is Assume that Then it is easy to check that η > 0. Therefore the solution of the HJB equation (71) is given by The optimal investment and consumption control policies are where η is given by (78). It remains to verify that V (x, y) is equal to the value function and k * , c * are the optimal policies. We give the verification in the following theorem.
Therefore, we have where Λ 1 η > 0 is a constant. Now using the definition of Y (t), Therefore, noting that |X(t)| ≤ X t , we have where Λ 2 > 0 is a constant independent of t. Therefore, by (17) and noting that 2γ < 2, we can get This verifies condition (82). Let G k,c be the generator of the process (X(t), Y (t)) for any control process (k(t), c(t)) ∈ Π. Then by using Ito's rule Integrating above on [0, T ] and also noting that V (x, y) is classical solution of (71), we have where, by virtue of (82), T 0 e −βt σk(t)X(t)V x (t, X(t), Y (t))dB(t) is local martingale under P . Then we have Since V (x, y) ≥ 0, we have lim sup Then, taking T → ∞ in (84), we can get Hence V (x, y) ≥ J(x, y; k, c) for all admissible (k(t), c(t)). This proves (a). Now assume that k * , c * are given by (81). We note that k * (t) and c * (t) are F t -measurable. In addition, from Lemma 2.3, we know that X(t) > 0, a.s., so we can get that k * (t), c * (t) are well defined and c * (t) ≥ 0. In addition, it is not hard to check that (15) and (16) are true for k * (t) and c * (t). Finally, since X(t) > 0 and k * (t) is well defined, we get P r(k * (t) < ∞) = 1, ∀t ≥ 0. Therefore, we can get P r T 0 (k * (t)) 2 dt < ∞ = 1, ∀T > 0.
By virtue of (79), we can get that η > 0 and Thus (90) is established. So we can get (87). Then, by virtue of (86), we can get that V (x, y) = J(x, y; k * , c * ). This completes the proof of (b).
6. Conclusion. In this paper, we study a stochastic portfolio optimization model with delays over an infinite time horizon. We consider utility functions that have been widely used, such as log utility and HARA utility. Due to the delay variables Y (t) and Z(t), the system is no longer a Markovian system. Under certain conditions, we have derived the explicit formulas for the value functions as well as the optimal investment and consumption controls. We want to point out that a crucial condition for both log utility case and the non-log HARA utility case is (see (53) and (77)) µ 3 e λh (r + µ 3 e λh ) = µ 2 − µ 3 λe λh .
This condition is necessary to ensure that the HJB equations (45) and (71) have solutions that are independent of z. As we have showed in Lemma 3.3, the independence of z is a necessary condition that we can solve the delay problem in a finite dimensional space. Actually, this is the main reason that stochastic control problems with delays are very challenging (see Larssen-Risebro [22]). More discussions about the condition (92) can be found in Chang-Pang-Yang [4] (Section 5 and 6). Finally, in this paper, we only consider the delay variables that depend on the total wealth process X(t). We can also model the stock price process directly with a stochastic process with delays. In addition, we can consider some other extensions, such as stochastic volatility models, transaction costs, etc. Those will be our future research topics.