A new semidefinite relaxation for $\ell_{1}$-constrained quadratic optimization and extensions

In this paper, by improving the variable-splitting approach, we propose a new semidefinite programming (SDP) relaxation for the nonconvex quadratic optimization problem over the $\ell_1$ unit ball (QPL1). It dominates the state-of-the-art SDP-based bound for (QPL1). As extensions, we apply the new approach to the relaxation problem of the sparse principal component analysis and the nonconvex quadratic optimization problem over the $\ell_p$ ($1<p<2$) unit ball and then show the dominance of the new relaxation.

which is known as an ℓ 1 -norm trust-region subproblem in nonlinear programming [3] and ℓ 1 Grothendieck problem in combinatorial optimization [7,8]. Applications of (QPL1(Q)) can be also found in compressed sensing where x 1 is introduced to approximate x 0 , the number of nonzero elements of x.
If Q is negative or positive semidefinite, (QPL1(Q)) is trivial to solve, see [13]. Generally, (QPL1(Q)) is NP-hard, even when the off-diagonal elements of Q are all nonnegative, see [6]. In the same paper, Hsia showed that (QPL1(Q)) admits an exact nonconvex semidefinite programming (SDP) relaxation, which was firstly proposed as an open problem by Pinar and Teboulle [13].
Very recently, different SDP relaxations for (QPL1(Q)) have been studied in [15]. The tightest one is the following doubly nonnegative (DNN) relaxation due to Bomze et al. [2]: where e is the vector with all elements equal to 1, S 2n is the set of 2n × 2n symmetric matrices, Y ≥ 0 means that Y is componentwise nonnegative, Y 0 stands for that Y is positive semidefinite, A•B = trace(AB T ) = n i,j=1 a ij b ij is the standard inner product of A and B, and Notice that the set of extreme points of {x : x 1 ≤ 1} is {e 1 , −e 1 , · · · , e n , −e n }, where e i is the i-th column of the identity matrix I. Define A = [e 1 , · · · , e n , −e 1 , · · · , −e n ] = [I − I] ∈ ℜ n×2n .
Consequently, (QPL1(Q)) can be equivalently transformed to the following standard quadratic program (QPS) [1]: Now we can see that (DNN L1 (Q)) exactly corresponds to the well-known doubly nonnegative relaxation of (QPS) [2]. Moreover, as mentioned in [15], (DNN L1 (Q)) can be also derived by applying the lifting procedure [9] to the following homogeneous reformulation of (QPS): A natural extension of (QPL1(Q)) is It is a relaxation of the sparse principal component analysis (SPCA) problem [10] obtained by replacing the original constraint x 0 ≤ k with (2) due to the following fact: A well-known SDP relaxation for (QPL2L1(Q)) is due to d'Aspremont et al. [4]: Recently, Xia [15] extended the doubly nonnegative relaxation approach from (QPL1(Q)) to (QPL2L1(Q)) and obtained the following SDP relaxation: It was proved in [15] that v(DNN L2L1 ( Q)) = v(SDP X ), where v(·) denote the optimal value of problem (·). Unfortunately, this equivalence result is incorrect though it is true that v(DNN L2L1 ( Q)) ≤ v(SDP X ). A first counterexample will be given in this paper (see Example 2 below) to show it is possible v(DNN L2L1 ( Q)) < v(SDP X ). The other extension of (QPL1(Q)) is 1 p and 1 < p < 2. (QPLp) is known as a special case of the ℓ p Grothendieck problem if the diagonal entries of Q vanish. According to the survey [7], there is no approximation and hardness results for the ℓ p Grothendieck problem with 1 < p < 2. Though (QPLp(Q)) has an exact nonconvex SDP relaxation similar to that of (QPL1(Q)), the computational complexity of (QPLp(Q)) is still unknown [6].
Since the ℓ p unit balls (1 < p < 2) are included in the ℓ 2 unit ball, a trivial bound for (QPLp(Q)) is where λ max (Q) is the largest eigenvalue of Q.
As mentioned by Nesterov in the SDP Handbook [12], no practical SDP bounds of (QPLp(Q)) are in sight for 1 < p < 2. Recently, Bomze [2] used the Hölder inequality to propose the following SDP bound In general, B 1 (Q) dominates B 2 (Q) when p close to 1, though lacking a proof.
In this paper, based on a new variable-splitting reformulation for the ℓ 1constrained set, we establish a new SDP relaxation for (QPL1(Q)), which is proved to dominate (DNN L1 (Q)). We use a small example to show the improvement could be strict. Then we extend the new approach to (QPL2L1(Q)) and obtain two new SDP relaxations. We cannot prove the first new SDP bound dominates (DNN L2L1 (Q)), though it was demonstrated by examples. However, under a mild assumption, the second new SDP bound dominates (DNN L2L1 (Q)). Finally, motivated by the model (QPL2L1(Q)), we establish a new SDP bound for (QPLp(Q)) and show it is in general tighter than The paper is organized as follows. In Section 1, we propose a new variablesplitting reformulation for the ℓ 1 -constrained set and then a new SDP relaxation for (QPL1(Q)). We show it improves the state-of-the-art SDP-based bound. In Section 2, we extend the new SDP approach to (QPL2L1(Q)) and study the obtained two new SDP relaxations. In Section 3, we establish a new SDP relaxation for (QPLp(Q)), which improves the existing upper bounds. Conclusions are made in Section 4.

A New SDP Relaxation for (QPL1(Q))
In this section,we establish a new SDP relaxation for (QPL1(Q)) based on a new variable-splitting reformulation for the ℓ 1 -constrained set.
For any x ∈ ℜ n , let Then we have Now we obtain a new variable-splitting reformulation of the ℓ 1 -constrained set: {x : Applying the lifting procedure [9], we obtain the following new doubly nonnegative relaxation of (QPL1(Q)) We first compare the qualities of v(DNN L1 ) and v(DNN new L1 ).
Proof. According to the definitions, If this is not true, then 0 < e T Y * e < 1. Define It is trivial to see that Y is also feasible to (DNN new L1 (Q)). Moreover, we havẽ which contradicts the fact that Y * is a maximizer of (DNN new L1 (Q)). According to the equality (10), Y * is also a feasible solution of (DNN L1 (Q)). Consequently, v(DNN L1 (Q)) ≥ v(DNN new L1 (Q)). The proof is complete. The following small example illustrates that v(DNN new L1 (Q)) could strictly improve v(DNN L1 (Q)).
0186. Finally, we show that there are some cases for which (DNN new L1 (Q)) has no improvement. This "negative" result is also interesting in the sense that in case we solve (DNN L1 (Q)), we can fix Y i,n+i (i = 1, . . . , n) at zeros in advance.
Proof. Let Y * be an optimal solution of (DNN L1 (Q)). Suppose there is an index k ∈ {1, . . . , n} such that Y * k,n+k > 0. Let δ k = Y * k,n+k and define a symmetric matrix Z ∈ S 2n where Z kk = Z n+k,n+k = δ k , Z k,n+k = Z n+k,k = −δ k and all other elements are zeros. Then Z 0,Q • Z = 2(Q kk + Q n+k,n+k )δ k ≥ 0.

It follows that
Then, Y * + Z is also an optimal solution of (DNN L1 (Q)). Repeat the above procedure until we obtain an optimal solution of (DNN L1 (Q)), denoted by Y * , satisfying Y * i,n+i = 0 for i = 1, . . . , n. Notice that Y * is a feasible solution of (DNN new L1 ). Therefore, we have v(DNN L1 (Q)) ≤ v(DNN new L1 (Q)). Combining this inequality with Theorem 1, we can complete the proof.

New SDP Relaxations for (QPL2L1(Q))
In this section, we extend the above new reformulation approach to (QPL2L1(Q)) and obtain two new semidefinite programming relaxations.
Introducing Y = yy T 0, we obtain the following new SDP relaxation for (QPL2L1(Q)): According to the definition, we trivially have: .
Proof. Both (DNN L2L1 (Q)) and (DNN new≤ L2L1 (Q)) share the same relaxation: Therefore, (R Y ) can be further relaxed to Let Q = U ΣU T be the eigenvalue decomposition of Q, where Σ = Diag(σ 1 , . . . , σ n ) and U are column-orthogonal. Since we can further relax (R X ) to the following linear programming problem: Now it is trivial to verify that v(LP) = max{σ 1 , . . . , σ n } = λ max (Q).
The proof is complete.
However, v(DNN new= L2L1 (Q)) may be not an upper bound of (QPL2L1(Q)), which is indicated by the following example.
So, we have to identify when v(DNN new= L2L1 (Q)) is an upper bound of (QPL2L1(Q)).
Proof. We first notice that the maximum eigenvalue problem is a homogeneous trust-region subproblem and hence has no local-non-global maximizer [11]. Therefore, suppose there is an optimal solution of (QPL2L1(Q)), denoted by x * , satisfying x 2 1 < k, then x * also globally solves (E), i.e., v(QPL2L1(Q)) = x * T Qx * = λ max (Q).

Remark 1
The assumption (18) is generally not easy to verify. However, when Q has a unique maximum eigenvalue, (18) holds if and only if v 1 > √ k, where v is the ℓ 2 -normalized eigenvector corresponding to the maximum eigenvalue of Q. Moreover, according to Corollary 1 and Proposition 2, the assumption (18) can be replaced by the following easy-to-check sufficient condition v(DNN L2L1 (Q)) < λ max (Q).
Motivated by the Hölder inequality (4) and the model (QPL2L1(Q)), we obtain the following new relaxation for (QPLp(Q)): Taking the transformation (11)- (14) and then applying the lifting approach [9], we obtain the following SDP relaxation for (QPL2L1 ≤ (Q)), which is very similar to (DNN new≤ L2L1 (Q)): Theorem 4 Proof. According to the definitions, the second inequality is trivial. It is sufficient to prove the first inequality. We first show B 2 (Q) ≥ v(DNN Lp ( Q)).
where the last inequality follows from Theorem 1. The proof is complete.
We randomly generated a symmetric matrix Q of order n = 10 using the following Matlab scripts: rand('state',0); Q = rand(n,n); Q = (Q+Q')/2; and then compared the qualities of the three upper bounds, v(DNN Lp ( Q)), B 1 (Q) and B 2 (Q). The results were plotted in Figure 1, where the lower bound of QPLp(Q) is computed as follows. Solve (DNN Lp ( Q)) and obtain the optimal solution Y * . Let y, z be the unit eigenvectors corresponding to the maximum eigenvalues of AY * A T and Q, respectively. Then 1 y p y and 1 z p z are two feasible solutions of (QPLp(Q)) and max y T Qy y 2 gives a lower bound of v(QPLp(Q)). From Figure 1, we can see that for 1 < p < 2, though B 2 (Q) and B 1 (Q) cannot dominate each other, both are strictly improved by v(DNN Lp ( Q)).

Conclusion
The SDP relaxation has been known to generate high quality bounds for nonconvex quadratic optimization problems. In this paper, based on a new variable-splitting characterization of the ℓ 1 unit ball, we establish a new semidefinite programming (SDP) relaxation for the quadratic optimization problem over the ℓ 1 unit ball (QPL1). We show the new developed SDP bound dominates the state-of-the-art SDP-based upper bound for (QPL1). There is an example to show the improvement could be strict. Then we extend the new reformulation approach to the relaxation problem of the sparse principal component analysis (QPL2L1) and obtain two SDP formulations. Examples demonstrate that the first SDP bound is in general tighter than the DNN relaxation for (QPL2L1). But we are unable to prove it. Under a mild assumption, the second SDP bound dominates the DNN relaxation. Finally, we extend our approach to the nonconvex quadratic optimization problem over the ℓ p (1 < p < 2) unit ball (QPLp) and show the new SDP bound dominates two upper bounds in recent literature.