A CONTROLLED TREATMENT STRATEGY APPLIED TO HIV IMMUNOLOGY MODEL

. Optimal control can be helpful to test and compare diﬀerent vaccination strategies of a certain disease. This study investigates a mathematical model of HIV infections in terms of a system of nonlinear ordinary diﬀerential equations (ODEs) which describes the interactions between the human immune systems and the HIV virus. We introduce chemotherapy in an early treatment setting through a dynamic treatment and then solve for an optimal chemotherapy strategy. The aim is to obtain a new optimal chemotherapeutic strategy where an isoperimetric constraint on the chemotherapy supply plays a crucial role. We outline the steps in formulating an optimal control problem, derive optimality conditions and demonstrate numerical results of an optimal control for the model. Numerical results illustrate how such a constraint alters the optimal vaccination schedule and its eﬀect on cell-virus interactions.


1.
Introduction. At the beginning of the 1980s, the syndrome of the Acquired Immune Deficiency Syndrome (AIDS) was for the first time described in the United States. It was observed in a group of homosexual men in California and New York who had opportunistic infections and specific tumors. In many countries in sub-Saharan Africa, the pandemic has cut life expectancy. For example, in Botswana life expectancy decreased from 65 years in [1985][1986][1987][1988][1989][1990] to 40 years in 2000-2005. An estimated 34-40 million people are currently living with HIV/AIDS. During the last 25 years, more than 25 million people have died from AIDS-related causes [4]. Clearly, HIV infection is still one of the major health concerns of our time and will remain a problem until a readily available vaccine is developed.
Once HIV enters the body, the human immune system tries to get free of it. The invasion is reported to CD4 + T cells (CD4 positive T lymphocytes, a type of white blood cells). The CD4 cell is a protein marker in the surface of the T cell, where letter T refers to thymus, the organ responsible for maturing these cells after they migrate from the bone marrow (where they are manufactured). The surface of CD4 + T cell has a protein that can bind to foreign substances such as HIV. The HIV needs a host in order to reproduce and the above-mentioned protein provides shelter. The HIV virus is a retrovirus, the RNA of virus is converted into DNA inside the CD4 + T cell. Thus, when infected the CD4 + T cells begin to multiply to fight this pathogen, they produce more virus. Due to the main importance of these cells, their depletion inflicts widespread negative effects on the functioning of the immune system. These cells are the main target of the virus. That's why the decline in the number of CD4 + T cells in peripheral blood is used as an indicator of the disease stage. An infected CD4 + T cell can produce around 500 new viruses before its death and thus becomes a much more important target to destroy than the virus itself [14]. When a free HIV virus enters the body and attacks the uninfected CD4 + T cells, the cells become actively infected for a certain period. Since the HIV life cycle is quite different from any other cell in the human body, many antiretroviral treatments are available which take advantage of its uniqueness. Different drug therapies are administered for different stages of the HIV infected patients. There are more than twenty Food and Drug Administration (FDA) recommended anti-HIV drugs available. Most of these drugs are divided in two categories: Reverse Transcriptase Inhibitors (RTI: AZT, ddI, ddC, D4T, 3TC, delavirdine, nevirapine, abacavir, succinate, and efavirenz) and Protease Inhibitors (PI: ritonavir, saquinavir, indinavir, and nelfinavir) [1]. The RTIs are used to prevent HIV RNA from being converted into DNA, thus blocking integration of the viral code into the target cells [15]. Reverse transcriptase inhibitors prevent HIV from infecting cells or put a stop to infection of new cells. The PIs efficiently reduce the number of infectious virus particles released by an infected cell [15]. Protease inhibitors prevent the production of new infectious virions by infected cells causing the virus to be unable to infect T cells.  Particles of HIV (green spheres), the virus that causes AIDS, bud from an infected white blood cell before moving on to infect other cells. [27] In the recent past, many mathematical models for the treatment of HIV infection have been developed. Such the ones presented in [17,18], can be used to evaluate the early events in the infection, when there are few infected cells and a small number of viruses. The models developed in [25,26], emphasize on the effect of variability among viral strains and try to explain better the rate of HIV evolution and disease progression. Some deterministic models may be found in [10,11,22]. These types of models examine the changes in mean cell numbers and are more applicable to later stages of the process in which population sizes are large. Usually, the dynamics of the CD4 + T cell and virus populations are taken account. In this paper, the immune system is modelled in terms of the population of CD4 + T cells. The dynamics of state variables for HIV immunology model are governed by the following equations [15]: Here T (t) is the concentration of uninfected CD4 + T cells, T i (t) represents the concentration of infected CD4 + T cells and V (t) corresponds to the free infectious virus particles, at moment t in days. Definition and numerical information for the parameters can be found in Table 1. In this model, we assumed that some secondary infection sites play a small role in the infection process that they may be neglected. However, as we shall show, our model can account for many of the characteristics of HIV infection seen clinically: the early infection, the long latency period, low level of virus in the body, and the depletion of CD4 + T cells. This paper is organized as follows: Section 2 describes mathematical models of HIV chemotherapy with a control term, the first part presents the basic mathematical model of HIV treatment using the chemotherapy and the second part focuses on the introduction of an isoperimetric constraint to the given model. The analysis of the optimization problem is presented in the section 3. In section 4, the iterative method is introduced and the numerical simulations are discussed. Finally, in section 5 overall results of our chemotherapeutic approach is discussed. 2. Optimal Control of HIV Immunology Model. Optimal control theory as a tool has been widely used in biomedical problems [15]. One of the key applications of control theory is in the determination of the optimal therapeutic protocols subject to various biomedical constraints and considerations. These applications are wide ranging and include cancer, epidemics, diabetes, leukemia, virological diseases like HIV, HCV, HBV, to name a few. Optimal control is used to find an optimal chemotherapy strategy in the treatment of HIV/AIDS. A great deal of research has been conducted on the effect of chemotherapy on the virus. Fister et al. [6] used an optimal control which represents the percentage effect the chemotherapy has on the fundamental interaction of the CD4 + T cells with the virus. Krischner et al. [6] using an objective functional based on maximizing the T -cell count and minimizing the systemic cost of therapy, determined the optimal therapy (like protease inhibitor) for HIV infected patients. Butler et al. [6] used a single control representing the percentage effect the chemotherapy on viral infectivity (this would simulate a drug such as AZT). Joshi [9] considered a similar problem for HIV patients but with combination therapy of immune boosting and viral suppressing drugs. Adams et al. [2] studied the modeling, data analysis and optimal therapy for HIV patients. Stengel [26] considered a drug resistant strain of HIV which resulted in rapid viral replication and determined that a continued optimal treatment is the key to sustained virological response in this case. Generally, in viral infection, the drug strategy affects either the virus infectivity or reduces the virion production. Presently, the most widely used medications for chemotherapy of HIV infection are developed to reduce the virus infectivity. That is why we focused on this type of strategy, where chemotherapy control effect the infectivity of the virus.

Mathematical Formulation with Chemotherapy.
We introduce a control u(t) which enters as percentage of the chemotherapy effects in the system of ordinary differential equations and affects the dynamic of the state system. The main purpose of this study is to develop a mathematical framework that can be used to understand the drug therapies in optimum controlled way for which it should maximize the healthy T cell count of an HIV infected patient, and minimize any side effects caused by the chemotherapy control. Our HIV model is given by the following nonlinear system of differential equations: With initial conditions A schematic diagram of the above HIV model is shown below: The logistic term Tmax in the first equation shows that production rate of T cell never grow larger than T max (i.e T (t) < T max ). Since existing drugs reduce the viral infectivity, that is why we focused to this type of strategy, and multiplying the kV (t)T (t) term in all three equations (1-3) by a chemotherapy function u(t), to achieve this affect mathematically. Due to both the adverse effects induced by the medications and the resistance developed by the virus at the prescribed drugs, we choose our control class as Lebesgue measurable functions defined on a finite period of time [0, t f ], with the restriction 0 ≤ u(t) ≤ 1. The case when u(t) = 0 corresponds to maximal use of chemotherapy and the treatment is absent for u(t) = 1. The description of the different parameters for above model is listed in the following table: Mathematical Formulation with an isoperimetric constraint. We extend our model by considering the situation in which it is supposed a limited supply of chemotherapy that could be administered to the patient during the treatment period is known. If B denotes the total amount of chemotherapy which will be given over t f days, then we have an integral constraint defined as follows: This type of constraint is known as an isoperimetric constraint [15] and can be handled by creating another state variable I(t), such that with conditions 3. Characterization of the Optimal Control. The problem is to maximize the objective functional where the weight parameter a ≥ 0 which relates the number of uninfected T cells concentration in a meaningful way to the percentage of chemotherapy. Our target is to maximizing the benefit based on the CD4 + T cell count, and minimizing the cost of chemotherapy to the body based on the percentage effect of the chemotherapy given by 1 − u(t) ; hence, we are maximizing the difference. If the control corresponds to maximal use of chemotherapy, then the maximal cost is represented as 1 − u(t) 2 . We assume that the relationship between the benefit to cost functional is not linear, and hence, we choose a simple non-linear control cost term [3]. The goal, therefore is to characterize the optimal control u * satisfying where U is the control set defined by

3.1.
Existence of the State system. First, the existence of state solutions of problems (1-6) given an optimal control in the admissible set U is shown.
Proof of Theorem 3.1. The state variables we consider here represent supersolutions for given problems (1)(2)(3)(4)(5)(6). From the given equations we have Also we have Lastly Since u(t) ∈ U , then, along with T (t), T i (t), V (t) and I(t) are bounded above. Via a maximum principle [24] and standard existence theory for first-order nonlinear differential equations, we obtain the existence of solutions to the problems (1-6).

3.2.
Existence of an Optimal Control. Now, the existence of an optimal control for the state system is analyzed. Using the fact that the solution to each state equation is bounded, the existence of an optimal control for each problem can be determined using the theory developed by Fleming and Rishel [7](Theorem 4.1).
i.e 0 ≤ θu 1 + (1 − θ)u 2 ≤ 1, for all u 1 , u 2 ∈ U and θ ∈ [0, 1]. Therefore, U is convex and condition (ii) is satisfied. For the third condition, we first reconsider the right-hand sides of (1-6) below: We see by the representations of F 1 , F 2 , F 3 and F 4 that they are continuous in t, T, T i , V, I and u since all are positive. Also, the system is bilinear in the control and can be rewritten as Where X = (T (t), T i (t), V (t), I(t)), and g(t, X) is a vector valued function of X with Also, the bound of the right-hand side of the state system is obtained as follows: where C incorporates the upper bound of the given constant matrix and the bound on kT max . Hence, the right-hand side of each state equation is bounded above by a sum of the control and the state. Lastly, the integrand of the objective functional is concave on U . One can consider the second partial of the integrand of the objective functional with respect to the control and find that it is negative. To obtain the necessary lower bound for the integrand, we see that Here b 2 = aT max and b 1 = 1 2 with α = 2. Therefore, condition (iv) is complete and so is the proof. [23] provides necessary conditions for an optimal control problem. This principle converted the problem of finding a control which maximizes the objective function J subject to the state system (1-6) to the problem of maximizing the Hamiltonian H, pointwisely with respect to u. So it is sufficient to derive the Hamiltonian H instead of deriving the objective function J defined in (7) in order to characterize the optimal control u * . The Hamiltonian is defined from the formulation of the objective function as follows:

The Optimality Conditions. The Pontryagin's Maximum Principle
where F i is the right hand side of the differential equation of i-th state variable. By applying Pontryagin's Maximum Principle [23] we obtain the following theorem.
Theorem 3.3. There exists an optimal control u * and corresponding solution T (t), T i (t), V (t)and I(t), that maximizes J(u) over U. Furthermore, there exists adjoint functions λ 1 (t), λ 2 (t), λ 3 (t) and λ 4 (t) satisfying the equations with transversality conditions Moreover, the optimal control is given by Proof of Theorem 3.3. The adjoint equations and transversality conditions can be obtained by using Pontryagin's Maximum Principle such that Since T (t), T i (t) and V (t) do not have fixed values at the final time (4), the values of the associated adjoints λ 1 (t), λ 2 (t) and λ 3 (t) at the final time are zero and I(t) has initial and terminal condition (6), that's why λ 4 (t) has no transversality condition.
The optimal control u * on the interior of the control set can be solved from the optimality conditions, By using the bounds on the controls, we get .
In addition, the second derivative of the Hamiltonian H with respect to u(t) is negative, indicating a maximum at u * . That is 4. Numerical Results. There are many methods and techniques of programming that can be used to solve numerically the optimal control problem (1)(2)(3)(4)(5)(6)(8)(9)(10)(11)(12) and namely, to find that maximizes the objective function (7). Generally to solve the optimality system of (1-4), an iterative method with a Runge-Kutta fourth order scheme is used and it is known under the name of Forward-Backward Sweep Method (FBSM) [15]. The principle of this method is that from an initial guess, the state system is solved forward in time and then the adjoint system is solved backward in time. All information about the convergence of this method is given in [8,19].
In this paper, after defining and introducing an isoperimetric constraint to the basic model, note that all states of the system (1-3) have a free end conditions (4) except the last state I(t) that both their initial and final conditions (6) are known. Our iterative method (FBSM) is limited and cannot deal with this type of problems directly. Therefore, another technique must be developed to be able to solve it numerically. Note that the adjoint system (8-10) satisfies the following transversality conditions: where λ 4 (t f ) is unknown.
Firstly, suppose that λ 4 (t f ) = θ. Then, the optimality system is solved using the FBSM iterative method with the given initial conditions for state sytems at t 0 = 0, x(0) = (T 0 , T i0 , V 0 , 0) and terminal conditions for adjoint systems at t f , λ(t f ) = (0, 0, 0, θ). The iterations continue until convergence is achieved, a value of I(t) at the final time is finally obtained and it is denoted by i f which is a function that depends on θ and unlikely our guess yielded the correct value; in particular, i f = B. However, we consider a mapping θ → i f and our main idea of this numerical solution technique is to define a new function and to seek for the zeros of this function using a root-finding algorithm commonly known in numerical analysis as Secant method [5,15]. Finally, the stored values for the state variables T (t), T i (t), V (t), I(t) and u are outputted during the last iteration, when f (θ) was nearly 0. Therefore, these are taken to be the solutions of the optimal control problem. However, the methods which were used to attain the optimal chemotherapy schedule has been verified previously [15], and research suggests that these optimal treatment schedules are the true solution for this problem.
To understand the impact of the added constraint, we first look at the optimal chemotherapy schedule with the given objective functional and no constraint placed on the chemotherapy. With the absence of the isoperimetric constraint, we remove the state variable I(t) from the optimal control problem and create a MATLAB program to solve the new optimality system. Running the new program with parameters values in Table 1 and initial values (T 0 = 800mm −3 , T i0 = 0.04mm −3 , V 0 = 1.5mm −3 taken from [15]) to produces the optimal chemotherapy schedule and corresponding population dynamics. Here we solve the optimality system of the problem considering without constraint, meaning that we can give chemotherapy as we can. These results show that without any preventive control (green line) the uninfected T cells continue to decrease, the number of infected T i cells and virus V cells increases, at the end of the time interval both cells achieving a maximum value at t = 30. By using chemotherapy we can maximize the number of uninfected T cells and minimize the number of infected T i cells and virus V cells. So we can say that using optimal control we have established control over the infected T i cells and free virus V cells as well as the side effects of diseases. Now we numerically analyze the virus dynamics for the HIV model, as well as the consequence of optimal treatment strategy. Since when u(t) = 0 corresponds to maximal use of chemotherapy for our model and the treatment is absent for u(t) = 1. Figure 6 shows that less percentage of chemotherapy has better administration of drug therapy. On the other hand figure 7 shows that, the solution of the maximum of J can be pursued with an adjustable value for a. By using this study practicing physicians should weigh the apparent side effects of each drug administered to the patient and proceed according to the optimal treatment schedule. From optimal control studies, several interesting results have been obtained for different values of k (rate of CD4 + T cell become infected by free virus). The model is disrupted by a significant increase in the values of k (figure 8), total immune system is collapsed after several days, although chemotherapy is still effective. This type of scenarios can be found during the final stage of HIV called AIDS.   Compare the two optimal vaccination schedules. The schedule with and without constraints requires maximum chemotherapy initially, then a gradual decrease over the next 30 days. However, the chemotherapy rate decreases and at the end of the 30 days chemotherapy programs no more than 10 in case of controlled chemotherapy (figure 10). When a controlled chemotherapeutic optimal dose is administered to the patient it swiftly increased the number of uninfected T cells by 22% during the last 10 days of the treatment period side by side reduced the infected cells T i and the viral load V by 98%. So it is clear that the introduction of an isoperimetric  constraint has allowed controlling and optimizing the drug dose amount, that can be administered to the patient during the treatment period to delay the onset of the many symptoms of AIDS and prolong the life of many HIV infected patients.   Since, our chemotherapy amount is limited, that is So, if the total chemotherapy amount(B) is less than the final time (30 days) then we get more effective chemotherapy. Note that the efficacy of chemotherapy will not greater than 1. From figure 11 and 12 it can be seen that if the value of B (chemotherapy amount) increases the system is gone to out of control. That is, if the value of I(t f )(total percentage of chemotherapy) become closer to final time then the system become unstable. Using small chemotherapy amount (green line) which is also less than the final time we get successful short-term diseases control.

5.
Conclusion. In this paper, we discuss an efficient numerical method based on optimal control to identify the best controlled treatment strategy of Human Immunodeficiency Virus (HIV). Firstly, a control characterizing as a rate of chemotherapy is introduced to the basic model. Then, it is supposed that the exact chemotherapy amount that can be administered to the patient is precisely known. Thus, an isoperimetric constraint is set and the basic model is modified by adding a new state characterizing the evolution of the treatment dose, while choosing an appropriate and homogeneous problem of optimal control to use properly the Pontryagins maximum principle to finally formulate the optimal control. However, it was essential to adapt the iterative FSBM method to the new problem for simulating numerically the optimality system. This unique numerical algorithm developed to solve this problem will serve as a template for future applications in the medicinal treatment of HIV infection.
Numerical results indicate that the optimal chemotherapy schedule with and without the constraint is a dynamic one, in which treatment is adjusted over the short course of administration where treatment is strong at the outset and then gradually lessens in strength over time. Further studies need to be done to incorporate a more accurate model of the immune system and other things as multiple drug treatments together with the resistance effects.