The linear convergence of a derivative-free descent method for nonlinear complementarity problems

Recently, Hu, Huang and Chen [Properties of a family of generalized NCP-functions and a derivative free algorithm for complementarity problems, J. Comput. Appl. Math. 230 (2009): 69-82] introduced a family of generalized NCP-functions, which include many existing NCP-functions as special cases. They obtained several favorite properties of the functions; and by which, they showed that a derivative-free descent method is globally convergent under suitable assumptions. However, no result on convergent rate of the method was reported. In this paper, we further investigate some properties of this family of generalized NCP-functions. In particular, we show that, under suitable assumptions, the iterative sequence generated by the descent method discussed in their paper converges globally at a linear rate to a solution of the nonlinear complementarity problem. Some preliminary numerical results are reported, which verify the theoretical results obtained.

1. Introduction. The nonlinear complementarity problem (NCP for short) is to find a vector x ∈ n such that x ≥ 0, F (x) ≥ 0, and x T F (x) = 0, where F : n → n is a given function. The NCP has been studied extensively due to its various applications in many fields. Such as mathematical programming, economics, engineering and mechanics (see, for example, [7,9,11]). We refer the interested readers to see the excellent monograph by Facchinei and Pang [7]. Various methods for solving the NCP have been proposed in the literature (see, for example, [2,3,4,5,6,10,11,12,13,14,15,16,17,18,19,20,23,24]). Among which, one of the most popular and powerful approaches is to reformulate the NCP as an unconstrained minimization problem [2,4,5,7,8,9,18,19,20,24]. This kind of methods is called the merit function method, where the merit function is generally constructed by some NCP-function.
where φ θp : 2 → is defined by with p > 1 and θ ∈ (0, 1]. It is easy to show that φ θp (·, ·) is an NCP-function. Thus, finding a solution of the NCP is equivalent to finding a global minimum of the unconstrained minimization min x∈ n Ψ θp (x) with the objective function value being zero. It is easy to see that if θ = 0 and p = 2, the function Ψ θp (·) defined by (1) reduces to the natural residual merit function Ψ N R : n → given by where φ N R : 2 → is an NCP-function given by φ N R (a, b) = −2 min{a, b}. If θ = 1, the function Ψ θp (·) defined by (1) reduces to the merit function Ψ p : n → given by where φ p : 2 → is an NCP-function given by φ p (a, b) = (a, b) p − (a + b). The NCP-function φ p was introduced by Luo and Tseng [19], and further studied by Chen [2] and Chen and Pan [4,5]. Obviously, when p = 2, the NCP-function φ p reduces the Fischer-Burmeister NCP-function φ F B (a, b) = √ a 2 + b 2 − (a + b). Recently, the so-called derivative-free methods have attracted much attention, which do not require computation of derivatives of F [12]. Derivative-free methods are particularly suitable for problems where the derivatives of F are not available or are extremely expensive to compute. For the NCP, the authors in [12] investigated a derivative-free descent method based on the merit function Ψ θp and the method is showed to be globally convergent under the assumption that F is strongly monotone. In this paper, our object is to discuss the rate of convergence for the derivative-free descent method discussed by [12] based on the merit function Ψ θp (x) defined by (1).
The paper is organized as follows. In Section 2, we review some definitions and preliminary results that will be used in the sequent analysis. In Section 3, we first discuss some properties of the merit function Ψ θp , including the growth behavior of the merit function Ψ θp , the estimation on the upper bound of Ψ θp , and the Lipschitz continuity of the gradient function ∇Ψ θp ; and then, we establish the linear rate of convergence of the derivative-free descent method discussed in [12]. Some final remarks are given in Section 4.
Throughout this paper, n denotes the space of n-dimensional real column vectors and A T denotes the transpose of the real-valued matrix A. For any differentiable function F : n → , F (x) denotes the gradient of F at x. For any differen- denotes the Jacobian of F at x. The norm · denotes the Euclidean norm. z denotes the smallest integer no less than z, for any z ∈ . The level set of a function Ψ : n → is denoted by L(Ψ, γ) = {x ∈ n | Ψ(x) ≤ γ}. We denote the set of positive integers by N + .

2.
Preliminaries. In this section, we recall some definitions and preliminary results, which will be used in the sequent analysis. The derivative-free descent method discussed in [12] is also given in this section.
Definition 2.1. Given the continuously differentiable function F : n → n , we say that (ii) F is a strongly monotone if, for all x, y ∈ n , F satisfies that x − y, F (x) − F (y) ≥ λ x − y 2 , or, equivalently, ∇F (x)y, y ≥ λ y 2 , for some λ > 0; (iii) F is a uniform P -function with modulus κ > 0 if max 1≤i≤n It is well-known that every monotone function is a P 0 -function and every strongly monotone function is a uniform P -function. For a continuously differentiable function F , if its (transpose) Jacobian ∇F (x) is a P -matrix then it is a P -function (the converse may not be true); and the (transpose) Jacobian ∇F (x) is a P 0 -matrix if and only if F is a P 0 -function. For more properties of various monotone and P (P 0 )-functions, please refer to [7].
The following error bound result from [21] will be used in our analysis later.
Theorem 2.4. Let F be strongly monotone with modulus with λ and Lipschitz continuous with L > 0 on n . Then holds for all x ∈ n , where x * is the unique solution of the NCP.
3. The rate of convergence. In this section, we first investigate several properties of the merit function Ψ θp ; and then, we discuss the linear convergence of Algorithm 2.1. Firstly, we investigate the growth behavior of the two merit functions: Ψ θp and Ψ N R .
be defined in (2). Then, for any p > 1 and θ ∈ (0, 1], Proof. Without loss of generality, we suppose a ≥ b in the following proof. If ab = 0, it is trivial. We will prove the desired results by considering the following two cases: ab > 0; and ab < 0. Case (i). Suppose that ab > 0. In this case, we have the following two subcases: (a) Suppose that a > 0 and b > 0. Then, Let then, Hence, we obtain that Let then, Hence, we obtain that Let then, Hence, we obtain that Combining the results of Cases (i)-(ii), we complete the proof.
From the definitions of Ψ θp ,Ψ N R and Lemma 3.1, we immediately get the following lemma.
Lemma 3.2. Let Ψ θp and Ψ N R be defined by (1) and (3), respectively, with p > 1 and 1 ≥ θ > 0. Then, for any x ∈ n , Secondly, we give an estimation on the upper bound of Ψ θp . Lemma 3.3. For all (a, b) = (0, 0) and p > 1, θ ∈ (0, 1], we have the following inequality: Proof. Without loss of generality, we assume a ≥ b. If ab = 0, it is trivial. We will prove the desired result by considering the following three cases: a > 0 and b > 0; a < 0 and b < 0; and ab < 0. Case (a). Suppose that a > 0 and b > 0. Then, by direct calculation, we get Thus, for all t ∈ (0, 1], we have L (t) ≥ 0. Furthermore, Squaring both sides leads to the desired inequality.
Case (b). Suppose that a < 0 and b < 0. Then, −a > 0 and −b > 0. From Case (a), we have Then, Case (c). Suppose that ab < 0. In this case, we have the following two subcases: (ii) Suppose that |a| < |b|, we have Combining cases (a)-(c), we complete the proof.

From Lemma 3.3, we get
and hence, we complete the proof.
Thirdly, we investigate the Lipschitz continuity of the gradient function ∇Ψ θp . To this end, we need the result on the boundedness of the level set for the function Ψ θp . Since ∇Ψ θp and F are continuous on n , then  Under the assumption that F and ∇F are Lipschitz continuous with some constant L > 0 on n , and from Lemma 3.5, we can further get the next lemma. Proof. Because ∇ b Ψ θp , F and ∇F are continuous on the bounded and closed set B(x 0 ), then there exist some constants C, ρ > 0 such that For all x, y ∈ B(x 0 ), from Lemma 3.5, we have By the same way, we obtain Then, we have where the second inequality is from the consistency of the matrix norm and vector norm. LetL Finally, by using the properties of the function Ψ θp , we show the linear convergence of Algorithm 2.1, for which the following lemma is helpful.
AsL ≥ 1 2 from Lemma 3.6, then we have We show that K and satisfy the inequalities (6) and (7). First, we prove (6) holds. We have that (9)); (8)); By a direct calculation and noting that = η K 4 , it follows that (10) implies (7). Theorem 3.8. Let Ψ θp be defined by (1) with p ≥ 2 and 1 > θ ≥ 0. Suppose F is continuously differentiable and strongly monotone with modulus λ > 0. Let x 0 ∈ n be any given starting point, and suppose that F and ∇F are Lipschitz continuous with some constant L > 0 on B(x 0 ). Then, for the sequence {x k } generated by Algorithm 2.1, it holds that the sequence {Ψ θp (x k )} converges to zero Q-linearly, and {x k } converges R-linearly to the solution of NCP.
Proof. For the sequence {x k } generated by Algorithm 2.1, the sequence {Ψ θp (x k )} is nonincreasing from (5). Hence {x k } is contained in L(Ψ θp , Ψ θp (x 0 )). For γ K ∈ (0, 1), where K is defined in Lemma 3.7, we have that x k , x k + γ K d k (η K ) ∈ B(x 0 ), and Hence, we have In the following, we give estimations of two items in the right-hand side of (11).
Firstly, we estimate the item −γ K ∇Ψ θp (x k ), d k in the right-hand side of the inequality (11). It follows that where the first inequality follows from ∇ a Ψ θp (x k , F (x k )), ∇ b Ψ θp (x k , F (x k )) ≥ 0 by Lemma 2.3(ii), and the second inequality follows from the strong monotonicity of F by Definition 2.1 (ii) and Cauchy-Schwarz inequality. Below we show that the following inequality holds, where > 0 set in Lemma 3.7. By the Cauchy-Schwarz inequality, it is sufficient to show that This above inequality holds if and only if From Lemma 3.7, we have (14) holds, then the inequality (13) holds. Combining (12) and (13), we obtain From Theorem 2.4 and Lemma 3.2, we can further get Since the sequence {Ψ θp (x k )} converges Q-linearly to zero, the sequence {x k } converges R-linearly to the solution x * of the NCP.
4. Numerical results. Some numerical results of Algorithm 2.1 for complementarity problems from MCPLIB [1] are reported in [12]. From the results, we see that Algorithm 2.1 works well for the tested problem in MCPLIB [1]. In this section, we implement Algorithm 2.1 in MATLAB 7.11 for some complementarity problems from MCPLIB [1] to observe the convergence of Algorithm 2.1. All numerical experiments are done at a PC with CPU of 2.4 GHz and RAM of 256 MB. The algorithm was terminated whenever one of the following conditions was satisfied: (1) Ψ θp (x k ) ≤ 10 −9 and d ≤ 10 −3 ; (2) the steplength γ t k is less than 10 −9 ; (3) the number of iterations is more than 100000.
We tests some problems for two purse: one is to investigate the convergence of Algorithm 2.1 with different p; the other is to investigate the convergence of Algorithm 2.1 with different θ.
Firstly, we take "gafni(1)" for example with different p. The parameters are chosen as follows: η = 0.8, σ = 0.5, γ = 0.6 and θ = 0.5.  • For the tested problems, the sequence {Ψ θp (x k )}, got from Algorithm 2.1, converges Q-linearly to zero. This phenomenon indeed verified the conclusions of Theorem 3.8. We also test other problems form MCPLIB [1], the convergent behaviors of Algorithm 2.1 are almost the same. • From Figure 1-3 we may see that the merit function Ψ θp in case when p = 100 has a faster decrease than the case when p = 10; Ψ θp in case when p = 10 has a faster decrease than the case when p = 1.1. We may get a conclusion that the convergence rate of Algorithm 2.1 becomes better when p increases. • From Figure 4-6 we may see that the merit function Ψ θp in case when θ = 0 has a faster decrease than the case when θ = 0.5; Ψ θp in case when θ = 0.5 has a faster decrease than the case when θ = 1. We may get a conclusion that the convergence rate of Algorithm 2.1 becomes worse when θ increases. 5. Some final remarks. In this paper, we obtained several favorite properties of the merit function Ψ θp , including the growth behavior of the merit function Ψ θp , the estimation on the upper bound of Ψ θp , and the Lipschitz continuity of the gradient function ∇Ψ θp . In particular, we showed that the iterative point sequence {x k } generated by Algorithm 2.1 is globally R-linearly convergent and the corresponding  It is interesting whether the NCP-function φ θp and the merit function Ψ θp can be extended to the case of the symmetric cone or not. If yes, whether the properties of these functions and the convergence results of the derivative-free descent method,   Figure 6. Convergence behavior of "josephy(5)" with θ = 1. obtained in [12] and this paper, are still satisfied. These are our subjects of future research.