AN OPTIMAL MEAN-REVERSION TRADING RULE UNDER A MARKOV CHAIN MODEL

This paper is concerned with a mean-reversion trading rule. In contrast to most market models treated in the literature, the underlying market is solely determined by a two-state Markov chain. The major advantage of such Markov chain model is its striking simplicity and yet its capability of capturing various market movements. The purpose of this paper is to study an optimal trading rule under such a model. The objective of the problem under consideration is to find a sequence stopping (buying and selling) times so as to maximize an expected return. Under some suitable conditions, explicit solutions to the associated HJ equations (variational inequalities) are obtained. The optimal stopping times are given in terms of a set of threshold levels. A verification theorem is provided to justify their optimality. Finally, a numerical example is provided to illustrate the results.


1.
Introduction. Major market models treated in mathematical finance in recent years can be classified into two broad categories: Cox-Ross-Rubinstein's binomial tree model (BTM) and Brownian motion based models. See related books by Hull [15], Elliott and Kopp [8], Fouque et al. [10], Karatzas and Shreve [17], and Musiela and Rutkowski [20] among others.
The BTM is widely used in option pricing due to its simplicity and yet clear advantage when pricing American type options. However, a main drawback of the BTM is its non-Markovian nature. The lack of Markovian property makes it difficult to work with mathematically, not to mention closed-form solutions. To preserve the BTM's simplicity and enhance its mathematical tractability, a two-state Markov chain model was considered in Zhang [29]. The Markov chain model in [29] appears to be natural for not frequently traded securities such as illiquid stocks. Its capability of capturing this type of markets makes it preferable in related applications. In addition, it can be seen from Example 1 to follow that the Markov chain model is closely related to the traditional mean-reversion diffusion when the jump rates of the Markov chain are large. Related Markov chain based models can be found in Van der Hoek and Elliott [23] and Norberg [21]. Van der Hoek and Elliott introduced a stock price model based on stock dividend rates and a Markov chain noise. Norberg used a Markov chain to represent interest rate and considered a market model driven are involved extensively. We provide a set of sufficient conditions that guarantee the optimality of the trading rules. Finally, to illustrate the results, we present a market backtesting example with the daily closing prices of Dow Jones Industrial Average from 1961 to 1980.
The main contributions of this paper include: (1) Introduction of a new meanreversion model driven by a two-state Markov chain. Such model is very simple in structure and yet powerful enough to capture many important market features.
(2) Development of mean-reversion trading strategies that are simple to implement. The corresponding optimal stopping problem is solved using a smooth-fit technique and hypergeometric analysis is involved in a substantial way.
This paper is organized as follows. In §2, we formulate the mean-reversion trading problem under consideration. In §3, we study preliminary properties of the value functions. In §4, we consider the associate HJ equations and their solutions. Then in §5, we discuss additional inequalities including the variational inequalities arising from the HJ equations and related convexity conditions. In §6, we give a verification theorem under suitable conditions. A numerical example is considered in §7. Several technical lemmas are postponed and provided in Appendix. Let S t denote the stock price (or a function of the price) at time t given by the equation where a > 0 is the rate of reversion and b the equilibrium. Here f (1) = f 1 > 0 represents the uptick return rate and f (2) = f 2 < 0 the downtick return rate.
Example 2.1. (Convergence to a Mean-Reversion Diffusion). In this example, we present a simple Markov chain model and demonstrate how it approaches the corresponding mean-reversion diffusion. Given ε > 0, we consider f 1 = σ/ √ ε, f 2 = −σ/ √ ε, λ 1 = λ 2 = 1/ε. Using the asymptotic normality given in Yin and Zhang [26,Theorem 5.9], we can show that the solution to (1) S t = S ε t converges weakly to the solution of dS t = a(b−S t )dt+σdW t , where W t is a standard Brownian motion.
In particular, taking a = 1, b = 2, σ = 0.5, and ε = 0.1, we generate the Monte Carlo sample paths of (S ε t , α ε t ). One sample is given in (a) of Figure 1. Similarly, we vary ε and plot the corresponding sample paths in (b) and (c) of Figure 1. It is clear from these pictures that as ε gets smaller and smaller, the fluctuation of α t is more and more rapidly and the corresponding S ε t approaches to a mean-reversion diffusion.
Remark 2.1. The main advantage of our model is its simplicity and flexibility in capturing various market conditions. For example, in a typical sideways (trendless) market, the price range can be precisely defined. In our model, such 'price band' arises naturally. A candidate for the band can be given by the interval We consider the state process  with initial vector (X 0 , α 0 ) = (x, α).
Let F t = {X r : r ≤ t} denote the filtration generated by X t . Note that α t is observable and F t = {α r : r ≤ t}. Let denote a sequence of stopping times with respect to {F t }. A buying decision is made at τ b n and a selling decision at τ s n , n = 1, 2, . . .. We consider the case that the net position at any time can be either long with one share of the stock or flat with no stock position. Let i = 0, 1 denote the initial net position. If the initial net position is long (i = 1), then one should sell the stock before acquiring any future shares. The corresponding sequence of stopping times is denoted by . Likewise, if the initial net position is flat (i = 0), then one should start to buy a share of the stock. The corresponding sequence of stopping times is denoted by Λ 0 = (τ b 1 , τ s 1 , τ b 2 , τ s 2 , . . .). Let K > 0 denote the fixed transaction cost (e.g., slippage and/or commission). Given the initial state vector (X 0 , α 0 ) = (x, α), the initial net position i = 0, 1, and the decision sequences (Λ 0 and Λ 1 ), we consider the corresponding reward functions where ρ > 0 is a given discount factor.
In this paper, given random variables x n , the term E{ For i = 0, 1, let V i (x, α) denote the value functions with the initial vector (X 0 , α 0 ) = (x, α) and initial net positions i = 0, 1. That is, It follows from (2) that X t = xe −at + e −at t 0 e ar f (α r )dr. Therefore, J i (x, α, Λ i ) is linear in x and V i (x, α) is convex in x for fixed i = 0, 1 and α = 1, 2.
In practice, the transaction cost K > 0 is typically small when compared to all other parameters. Here we assume it is smaller than both (f 1 /a) and (−f 2 /a). In addition, both λ 1 and λ 2 are large numbers relative to a. So we also assume λ 1 > a and λ 2 > a.
We summarize conditions to be imposed in this paper: (A1) f 1 > 0 and f 2 < 0; (A2) K < min{f 1 /a, |f 2 |/a}. (A3) λ 1 > a and λ 2 > a. Note that in this paper the stock price X t is differentiable and the value of α t can be given in terms of the derivative of X t with respect to t.

Properties of the value functions.
Lemma 3.1. The following inequalities hold.
Given Λ 0 , integrate both sides over [τ b n , τ s n ] to obtain It follows that, for (x, α) ∈ I 0 × M, This implies (a).
we note that, for a given Λ 1 with τ s 1 = 0, Since the rest stopping times in Λ 1 are arbitrary, it follows that V 1 (x, α) ≥ V 0 (x, α)+ x − K. Similarly, we can show the second inequality in (b).
4. HJ equations. Let A denote the generator of (X t , α t ), i.e., for any differentiable functions h(x, i), i = 1, 2, where h denotes the derivative of h with respect to x. The associated HJ equations should have the form: for α = 1, 2.
In this paper, our goal is to find the value functions by solving these HJ equations with all value function properties satisfied. Then we proceed to show that such functions v i (x, α) are equal to the value functions.
To solve these HJ equations, for each fixed i = 0, 1, we consider equations when ρv i (x, α) − Av i (x, α) = 0, α = 1, 2. Using the generator A, we can write For each fixed i = 0, 1 and α = 1, 2, let y α (x) = v i (x, α). Then, using the first equation, we can write y 2 in terms of y 1 : Its derivative with respect to x is given by Substitute these into the second equation in (7) to obtain To simplify this equation, let Moreover, applying the chain rule to obtain This reduces the equation for y 1 (x) into an equation for u(ξ) as Next, we introduce the additional parameters Then, we can write the above differential equation in terms of (γ 1 , γ 2 , γ 3 ) as This is exactly the classical hypergeometric equation. Related properties of hypergeometric functions can be found in [19] and [25]. For any real numbers η i , i = 1, 2, 3, let for any η = η i . Then, for 0 < ξ < 1, two independent solutions of (8) can be given by We consider these solutions over a neighborhood in I 0 containing the lower end f 2 /a. Then any solution over this interval can be written as a linear combination of F 1 and F 1 , i.e., for some constants In particular, we need this solution to be bounded near f 2 /a. This requires Therefore, on the neighborhood of I 0 containing f 2 /a we can assume that v 1 (x, 1) Alternatively, we can use the second equation in (7) and write y 1 in terms of y 2 : and its derivative in terms of y 2 and y 2 Substituting these into the first equation in (7) to obtain Then, we can write the above differential equation in terms of w(ξ): with γ 3 = (ρ + λ 1 )/a. Its two independent solutions can be given by Similarly, we consider these solutions over a neighborhood in I 0 containing the upper end f 1 /a. Again, any solution can be written as a linear combination of F 2 and F 2 , i.e., for some constants C 2 and C 2 , v i (x, 2) = C 2 F 2 (x) + C 2 F 2 (x), i = 0, 1. We now need this solution to be bounded near f 1 /a, which implies C 2 = 0 because Therefore, on the neighborhood of where F 1 , F 2 , G 1 , and G 2 are defined in (9), (11), (10), and (12), respectively.

4.2.
Order of threshold levels. If α t = 1, let b 1 denote the buying threshold (to buy when X t < b 1 ) and s 1 the selling threshold (to sell when X t > s 1 ), respectively. Likewise, if α t = 2, let b 2 and s 2 denote the corresponding buying and selling thresholds, respectively.
In the purely selling case, it is shown in [29] that s 1 > s 2 . Intuitively, b 1 should be greater than b 2 as well.
In addition, in this paper, we consider the case when b 1 < s 2 .
The opposite inequality b 1 ≥ s 2 means to buy when α t = 1 and X t < b 1 and sell as soon as α t jumps to 2 and X t > s 2 . This case rarely arises in practice because the jump rates λ 1 and λ 2 are typically large so that α t switches between 1 and 2 frequently.
In view of these, we consider the threshold levels having the following order The corresponding equalities in the HJ equations in (6) are marked in Figure 2.
Using convexity of the value functions and variational inequalities in (6), it is shown in Lemma 8.2 (Appendix) that It is easy to check, under Assumption (A2), that b 2 and s 1 given above indeed satisfy the inequalities b 2 > f 2 /a and s 1 < f 1 /a.
Using the power series form of Similarly, we have g 1 (x)) .

4.4.
Smooth-fit conditions. Note that the convexity of v i (x, α) in x on I 0 implies that they are continuous in the interior of I 0 . The continuity of these functions at the threshold levels leads to the following conditions: Recall that b 2 = (f 2 − ρK)/(ρ + a) and s 1 = (f 1 + ρK)/(ρ + a). Using the first and last equations in (13), we can write Then, we can write the second and the third equations in terms of C 1 and C 2 : .
Similarly, we can write the fourth and the fifth equations as follows: If the above 2 × 2 matrices are invertible, then we can eliminate C 1 and C 2 and obtain The solutions to the HJ equations (6) should have the form: In Figure 3, these functions are so labelled in their corresponding intervals in Figure 2.

Variational inequalities and convexity.
In this section, we consider the variational inequalities in (6) and related convexity inequalities.

Variational inequalities.
Note that with the functions given in (15), all variational inequalities in (6) have to be satisfied, i.e., for (x, α) ∈ I 0 × M, In what follows, we consider these inequalities on each intervals: We need to show It is easy to see that the last two inequalities follow directly from the first two equalities in (17). To see the first inequality ρv 0 (x, 1) ≥ Av 0 (x, 1), note that it can be written in terms of v 1 (x, 1)and v 1 (x, 2): which is equivalent to because ρv 1 (x, 1) = Av 1 (x, 1). The above inequality is clearly satisfied because b 2 = (f 2 − ρK)/(ρ + a) and f 2 < f 1 . Similarly, on [s 1 , f 1 /a], we can use s 1 = (f 1 + ρK)/(ρ + a) and show that all VIs in (16) hold.

Next, we consider the VIs on
We need the following inequalities to hold: The third inequality is automatically satisfied. The second and forth ones can be written as: Finally, the first inequality in (20) can be given as We can work out all needed inequalities this way on all other intervals and summarize these inequalities as follows: where

Convexity.
Recall that the value functions are convex on I 0 . We expect the solutions to the HJ equations in (6) to be also convex.
Recall also that for i = 1, 2.
Proof. Clearly, both F 1 and F 2 are convex because their second derivatives are positive.
To show the convexities of G 1 and G 2 , we first consider with u(ξ) = F (γ 1 , γ 2 ; γ 3 , ξ). Multiply its both sides by (−(1 − ξ) γ1+γ2−γ3−1 ) to obtain The right hand side of the above equation is equal to We next apply the formula with n = 1 and get Hence we conclude that For u(ξ) = F ρ a , ρ+λ1+λ2 a ; ρ+λ2 a , ξ , the corresponding g(ξ) is given by Then the convexity of F ρ a , ρ+λ1+λ2 a ; ρ+λ2 a + 1, ξ implies that g(ξ) is convex. It is So G 1 is convex. Similarly, we can show the convexity of G 2 .
Proof. Here, we only show the convexity of h 1 (x). The convexity for k 1 (x) can be given in a similar way. Recall that Let It follows that Apply the above formula, we obtain Recall the derivative formula for hypergeometric function: It follows that In view of this and the convexity of the hypergeometric functions with positive parameters, the convexity of w 1 (ξ) follows on [0, 1]. So is the convexity of h 1 (x).
Lemma 5.3. g 2 B + h 1 C 1 + h 2 and g 1 A + k 1 C 2 + k 2 are convex provided that where Proof. It is easy to see that both g 1 (x) and g 2 (x) are convex. In addition, note that g 1 A + k 2 can be written as g 1 (A + A 0 )+ a linear function. So A + A 0 > 0 implies that g 1 A + k 2 is convex. Then in view of the convexity of k 1 (x) in Lemma 5.2, it follows that g 1 A + k 1 C 2 + k 2 is convex. Similarly, we can show the convexity of To guarantee the convexity of the functions v i (x, α), it is sufficient to require the convexity of v 0 (x, 1) at x = b 1 and v 1 (x, 2) at x = s 2 .
Proof. We only sketch the proof because it is similar to that of Zhang and Zhang [27,Theorem 5]. First, for any stopping times 0 ≤ θ 1 ≤ θ 2 , we can show as in [27] that α θ2 ), for i = 0, 1. For any given Λ 0 , it follows from this inequality (with θ 1 = 0 and θ 2 = τ b 1 ) and the VIs in (6) that

Continue this way and recall
To establish the equalities with Λ i = Λ * i , first note that, in view of Lemma 8.3 (Appendix), τ b * n and τ s * n are finite a.s. for each n. Using this condition, the inequalities in (25) become equalities when τ s i is replaced by τ s * i and τ b i is replaced by τ b * i , respectively. Finally, the boundedness of v 0 and τ s * n → ∞ imply, for i = 0, Similarly, we can show the case when i = 1.
Remark 6.1. We point out that a sufficient condition for τ s * n → ∞ can be given by s 2 − b 1 > 2K following the proof in [27]. (This inequality is satisfied in all cases in our numerical example in the next section.) 7. A numerical example.
Then, it follows that Therefore, the jump rates are given by At the end of 1971, we update the system parameter values using the most recent data (1971) and dropping the 1961 prices. We keep rolling forward this way. The parameter values are given in Table 1 and the corresponding threshold levels are provided in Table 2. In addition, the conditions required in Theorem 5.1 are satisfied in all these cases.   In each trading year, a buying signal is triggered when (X t ,α t ) exits D b and a selling signal is generated when (X t ,α t ) exits D s . In Figure 4, we define b[n] as a piecewise constant function determined by the b-column in Table 1.  Table 2. Also, in Figure 4, X nδ = log(c[n]) − b[n] is plotted.
Initially, we allocate the capital of $100K. When a trading signal is triggered, buy (and sell) the index with the entire account balance less the transaction fees.
The corresponding equity curve is also provided in Figure 4. There are five trades between 1971-1980. The ending balance is $142818.61 which amounts 42.28% gain.
8. Conclusion. This paper considers a mean-reversion trading rule under a Markov chain model. It is shown that the optimal trading rule can be given in terms of four threshold levels that can be obtained by solving algebraic like equations. Such feature is attractive from practical viewpoints because they are easy to implement.
It would be interesting to study related problems with regime switching. For example, the rate of mean reversion and equilibrium levels are regime dependent. In this connection, one may introduce a two-time-scale Markov chain so that the 'fast' states correspond to short term events and the 'slow' states correspond to market trends. In addition, related state estimation may arise due to incomplete observation of the 'slow' components.