An optimized direction statistics for detecting and removing random-valued impulse noise

In this paper, we propose a robust local image statistic based on optimized direction, by which we can distinguish image details and edges from impulse noise effectively. Therefore it can identify noisy pixels more accurately. Meanwhile, we combine it with the edge-preserving regularization to remove random-valued impulse noise in the cause of precise estimated value. Simulation results show that our method can preserve edges and details efficiently even at high noise levels.

1. Introduction. Digital images are frequently corrupted by impulse noise due to transmission errors, malfunction pixel elements in camera sensors, faulty memory locations and timing errors in analog-to-digital conversion [18]. Impulse noise could damage the information of the original image, resulting in corrupted images. They could hamper the subsequent image processing operations, such as segmentation and object recognition. Therefore, de-noising is fundamental and important to image processing.
In order to suppress impulse noise, a variety of techniques have been proposed. The median filter (MF) [26] is one of the most common methods due to its simplicity and high computational efficiency. One limitation of MF is its poor detail preserving capacity since every pixel in the image is replaced by the median value of its neighbors. Therefore, weighted-based median filters are proposed. For example, the weighted median (WM) filter [7], the center weighted median filter (CWM) [21] and the recursive weighted median filter (RWMF) [3]. Although they can preserve more details by giving more weight to desirable pixels in the sliding windows, they are still implemented uniformly across the image without considering whether the processing pixel is noise-free or not. These filters are effective when the impulse noise levels are low.
To enhance the performance of impulse noise filter, a new strategy of detecting noise before filtering has been presented. The noise detector is used to identify the noisy pixels. Then the detected noisy pixels are filtered while the clean ones keep unchanged. Such kind of techniques are usually median or mean type filters based, including the switching median (SM) filter [28], signal dependent rank order mean (SD-ROM) filer [1], the adaptive center-weighted median (ACWM) filter [13], the luo-iterative method [24], the directional weighted median (DWM) filter [15], the adaptive switching median (ASWM) filter [2], the contrast enhancement-based filter (CEF) [17], the optimal direction-based median filter (ODM) [4], the robust outlyingness ratio based non-local mean (ROR-NLM) filter [32] and the two-phase detector based weighted mean filter (TPD-WMF) [23]. However, these filters just use median values or their variances to restore the impulse noise, and they may blur image details even when the images are mildly corrupted.
In the last few years, some techniques using the fuzzy rule for noise removal have been developed [30,33]. On one hand, fuzzy rule is suitable for impulse noise due to its uncertainty feature. On the other hand, discovering the rule-based structure will become quite difficult if the images are highly corrupted. Many researchers proposed improved methods to solve this problem, such as those developed based on neuro-fuzzy [27,34,35], and those based on cluster [29]. The main drawback of them is that they require appropriate and adequate training which involve complex calculations.
In order to overcome the burring detail problem of median type filter, recently an edge-preserving regularization (EPR) method [25] has been proposed to remove impulse noise while allowing edges and noise-free pixels to be preserved. It uses non-smooth data-fitting term along with edge-preserving regularization. Then a two-phase scheme combining this variational method with noise detector is proposed in [11,9]. Their capability is mainly limited by the accuracy of the noise detector in the first phase. In [16], a rank-ordered absolute differences (ROAD) statistic is introduced to identify the impulse noisy pixels. A rank-ordered logarithmic difference (ROLD) [14] based on ROAD is proposed, where it is combined with the edge-preserving regularization method (ROLD-EPR). In [36], ROAD-BEPR filter is proposed, where ROAD statistic is combined with bilateral edge-preserving regularized method. ROAD and ROLD perform well for detecting most impulse noise, but they are not accurate at edges. On the other hand, the EPR method dose offer good filtering performance, but its computational complexity is higher than most of the previously mentioned filters.
Inspired and motivated by the above research works, under the framework of two-phase, we propose a new local image statistic based on optimized direction and ROAD as a new detector and a fast filter based on EPR. In the first phase, our idea is to amplify the weight of absolute difference between centering pixel and surrounding pixels belonged to optimized direction so that the noise detection can be more accurate at edges. In the second phase, we choose a simpler regularization term without data-fitting term to reduce the complexity and the speed of the computation. Combining the two phases, our new method is called WROD-EPR. It outperforms the others in both image restoration and noise detection.
The outline of this paper is as follows. Section 2 shows some related works. In Section 3, we define the robust local image statistic, called the WROD statistic. Section 4 describes our WROD-EPR filter in detail. Experimental results are given to demonstrate the performance of our method in Section 5. Finally, a brief conclusion is presented in Section 6.
2. Related works. In this section, we review some studies related to our work that focus on the noise detectors and filters design for impulse noise removal. We first introduce the impulse noise models and then briefly review ROAD detector and EPR filter.
2.1. Impulse noise models. Unlike Gaussian noise, an important characteristic of impulse noise is that a portion of an image pixel values are replaced with random values while leaving the rest of the image pixels unchanged. The most noticeable and least acceptable pixels in the noisy image are those whose intensities are much different from their neighbors. We suppose that x is the original digital image and y is the degraded image by impulse noise. Let x and y denote two matrices in R m×n . Their relationship is modeled as: where N r : R m×n −→ R m×n represents impulse noise which take values in the dynamic range [d min , d max ] of x with probability of r.
To be more precise, let x i,j and y i,j be the pixel values at location (i, j) in the original image X and the impulse noisy image Y , respectively. When the noise ratio is r, Eq.(1) can be described as: where n i,j is the intensity value of the impulse noise at the location (i, j). There are two common types of impulse noise: fixed-valued (salt-and-pepper); and randomvalued. When n i,j only takes either d min or d max with equal probability, the noise is called salt-and-pepper noise, and when n i,j takes random numbers in [d min , d max ] with a uniform distribution, the noise is called random-valued impulse noise. Between them, random-valued impulse noise is more difficult to be removed. So we only focus on the detection and de-noising of the random valued impulse noise.

2.2.
ROAD. ROAD is a statistic proposed in [16]. Let (i, j) be denoted by p and let Ω p (V ) be the set of neighbor points of the center pixel p with window size (2V + 1) × (2V + 1), i.e., For each pixel q ∈ Ω 0 p , define d p,q as the absolute difference in intensity values of the pixels between p and q, i.e., Sort the values of d p,q in increasing order, and let r k (p) denote the k th smallest one in the ordered queue. Then define where 2 ≤ m ≤ (2V + 1) 2 − 2.
As an example, if we set V = 1 and m = 4, the ROAD provides a measure of how close the current pixel value is to its four closest neighbors in a 3 × 3 windows.
We note that noisy pixels should have intensities vary widely from those of its neighbors, i.e. their ROAD values will be large, whereas for noise-free pixels, their ROAD values should have similar intensity to at least half of their neighbors, i.e., their ROAD values will be small, even for pixels on the edges, see [16]. Thus, we can use ROAD to detect impulse noise. If the ROAD value of a pixel is greater than a certain fixed threshold, we consider it as a noisy pixel; otherwise the pixel is considered noise-free.
2.3. EPR filter for impulse noise removal. After the noise detection, the next problem is how to choose an appropriate filter to remove these detected noisy pixels. Most nonlinear methods replace the noisy pixels by some median values in their vicinities without considering local image features such as possible presence of edges. Therefore, the restoring is not satisfactory for details and edges, especially when the noise level is high. Variational methods based on the minimization of edgepreserving regularization functions have been used in [22,12,31] to preserve the edges and the details in images corrupted by Gaussian noise. In [25], a technique based on a variational framework is proposed to deal with impulse noise.
Let N be the noise candidate set, and let Ω 0 p be the neighborhood of p excluding p andx as the restored image. If a pixel p / ∈ N , it is detected as uncorrupted, we naturally keep its original value, i.e.,x p = y p . On the other hand, if p is a noise candidate, namely p ∈ N , then y p must be restored. In addition, the rule is also applied to its neighbors. In [25], noise candidates are restored by minimizing an objective functional Θ with an l 1 data-fitting term and a regularization term involving an edge-preserving potential function ϕ α (t). See below: where β is a regularization parameter and ϕ α [22,12,5,19,9] is an even edgepreserving potential function having the parameter α.
3. WROD random-valued impulse detector. In this section, we present an effective noise detector for detecting noisy pixels. The proposed detector is based on the ROAD scheme.
3.1. Motivation of WROD. From Eq. (6), we can see that each ROAD value is associated with a pixel. The basic assumption is that larger ROAD values indicate noisy pixels, while smaller ones suggest clean pixels. The noisy pixels are detected by comparing their ROAD values with a pre-defined threshold. However, it may fail in some image patterns (e.g., edges, textures or smooth areas) because it dose not utilize the local information which could provide prior image local features in the process, such as the local mean and the local standard deviation. The ROAD values can easily be affected by different patterns in images. Further discussions are given as follows.
When a neighbor of a processing pixel p contains both flat-region and edge, according to Eq.(5), nearly half of its absolute differences are large no matter whether or not x 0 belongs to flat area or edge. Fig. 1   In Fig. 1(a), we easily observe that there are only two smaller differences whose pixels are co-linear with p along the edge direction. If we select four of the smallest differences in accordance with Eq.(5 and 6) following the suggestion given in [16], we obtain {1, 2, 50, 55}. The value of its ROAD is 108 which exceeds the threshold provided in [16]. Thus, it leads to failed detection by labeling a clean pixel as noisy pixel. More interestingly, if the processing pixel p was corrupted by impulse noise, its intensity value may either close to the left column or to the right column depending on different situations. In this case, the actual noisy pixel should be regarded as noise-free because it is easier to get a small ROAD value. This analysis is also applicable to window of different sizes. So WROD is used to address such concern.
Based on the above discussion, the edges contained in a neighborhood have great influence on ROAD, while their local image features are ignored by ROAD. Hence, the ROAD (including ROLD) statistics are not accurate at image edges detection, although they perform well for detecting most impulse noise.
3.2. WROD statistic. First, we expand the sliding window size, since a smaller window, like 3 × 3, is easy to lead to into false judgment of edges as the percentage of noise-free pixels is too low. With a larger window, the clean pixels on the edge have a relatively large number of copies of similar neighborhoods located along the structure of interest. It can lead us to find the real edges even the noise level is very high. Normally, the window is taken as 5 × 5.
Let Ω p (V ) denote a (2V + 1) × (2V + 1) window with the center pixel located at p, l denote a set of directions from 1 to 4, and h ∈ [1, V ] denote the hop count between p and its surrounding pixels in Ω p (V ), shown in Fig. 2.
Let S (l) h denote a set of coordinates aligned with the l th direction and their hop count is h. We take a 5 × 5 window, namely V = 2, as an example. These sets are demonstrated in Table 1.
For each direction l, the standard deviation with different h is defined by h }. Considering that for a pixel whose spatial hop count is small, its intensity value should be close to the central pixel. We will weight the standard deviation with a larger value when the hop count is small. Thus we define a weighted direction index D (l) based on standard deviation with the weight in the l th direction, i.e., where w(h) is the weight reciprocal of the hop count h, and V h=1 w(h) is the normalization factor. We further denote w(h) as: where α is the control parameter. In order to ensure that the influence of deviation on weighted direction index declines with increasing hop count, the α is restricted to [−1, 0). To further determine the α, we implement the proposed Algorithm (see Section 4) to process the noisy images, and the average PSNR value of the denoised results is polled as a function of α. We can see from Fig. 3 that the average PSNR value increases at the initial stage, then reduce gradually after reaching a peak value α = − 1 3 . Although α = − 1 3 achieves the best result, any α ∈ [−1, 0) can still achieve satisfactory results. We choose α = − 1 3 for our method. Each weighted direction index is sensitive to the edge aligned with a given direction. From the observation, we see that the standard deviation of pixels along edges should be small, otherwise it should be large. Inspired by this observation, Since the standard deviation describes how tightly all the values are clustered around the mean in the set of pixel, l min shows that the four pixels aligned with this direction are the closest to each other. Therefore, the center value should also be close to them in order to keep the edges intact. Hence, let Ω 0 p = Ω p \{p}, we assign a weight to these pixels, and denote an extensional rank candidate set E p as: where Ψ(q) = θ, q ∈ S lmin 1, q / ∈ S lmin , A p (q) = |y q − y p |, and the operator ♦ denotes repetition operation [7]. θ usually takes its value in the range [2, h max ]. Taking into account the influence of the impulse noise, we use θ = 2. Then we define a statistic WROD, short name of "Weight Rank-Ordered Difference", i.e., where the operation Sort represents sorting the A p (q) values in increasing order, and m ∈ [2, (2V + 1) 2 − 2] denotes the first m values amongst sorted result. In [16], it is suggested that we can use m=12 and the 5-by-5 sliding window if the noise ratio is high. We focus on high density noise in this paper, therefore we use m=12 in our simulations. With WROD, we can define a noise detector by employing a threshold T : a pixel p is detected as noisy if W ROD m p (q) > T , and noise-free if otherwise.
We will discuss why this local statistic can be used to enhance detection capability through analyzing the following four cases.
I. When p is an edge pixel, and suppose its neighbors are all clean. Though for some directions, A p (q) may be large, the intensities of neighbor pixels in the same edge are very close to the center pixel p. Our l min method increases the proportion of small absolute differences between p and the same edge pixels in the candidate set E p so that the W ROD will be small. II. When p is noisy, and suppose its neighbors are all clean. Then W ROD will be large because every A p (q) is large.
III. When p is an edge pixel with some neighbor pixels corrupted by impulse noise. Because in the same edge, some neighbors are not contaminated by noise, the W ROD is still small. Let us raise the noise ratio to 60% with 5 × 5 sliding windows. The total number of pixels is 25, and l min contains 5 pixels. So the possible count of pixels corrupted by impulse noise in l min is 5 25 × 60% = 1.2. Thus most pixels in the optimized direction l min are clean. The weight Ψ(q) will bring more small values from the optimized direction l min into the set E p . Thus, the W ROD is still small.
IV. When p is noisy, and suppose some of its neighbors are corrupted by impulse noise too. In this case, it is clear that the W ROD is large.
It is worth noting that, in case IV, the W ROD may be small if one or more pixels in the same edge are corrupted by similar noise values as those of the center pixel. However, such a case may occur only with a very small probability since the Random-valued impulse noise can choose any value in [0, 255]. Thus, this case can be ignored.
4. WROD-EPR filter. WROD-EPR filter is a variational method. The essence of our method is first to detect the corrupted pixels and then, at the second phase, to restore the image only by using those pixels that are not corrupted. With a good noise detector WROD, we can combine it with the edge-preserving regularization, forming a two-stage method, denoted as WROD-EPR. To ensure a high accuracy of detection, it is executed iteratively with decreasing thresholds, see [11]. At early iterations, with large thresholds, WROD will identify pixels that are most likely to be noisy. In the subsequent iterations, we decrease the threshold to include more noise candidates. Suppose the noisy image is y. Our algorithm is as follows: Algorithm The proposed WROD-EPR filter for random-valued impulse noise 1: Initialization Set k = 0 and x 0 = y, where k is the iteration number and y is the observed image. 2: Noise detection a. Compute the W ROD values for each pixel p k in image x k . b. If W ROD(p k ) > T k , p k is identified as a noise pixel, and p ∈ N k (the noise candidate set); otherwise, p k is noise-free. To ensure high accuracy of detection, we adopt recursive method to decrease threshold as suggested in [14]: where T 0 takes the value that is close to the mean W ROD of the noisy pixels, since the mean reflects the trend of how intensively the impulse noise is. For the statistics on image "Lena" and "Bridge" with m=12 and 5-by-5 sliding window, we got the mean WROD value for the noisy pixels are in [6.7,15.8]. The midpoint of the mean range should be better, thus we choose the midpoint 11.3 as the initial value of T, namely, T 0 = 11.3. 3: Noise restoration Restore all pixels in N k by minimizing the following function [10]: where Ω 0 p is the set of the closest neighbors of p, and ϕ is an edge-preserving potential function [12]. For all p / ∈ N k , take x p . We only have the regularization term and no data-fitting term. It is because the data are fitted exactly for uncorrupted pixels; see, e.g., [10]. Some methods can be used to minimize it. In [20], a dual method is presented. The Newton's method with continuation is adopted in [10]. In this paper, we use the following function as ϕ ϕ(z) = |z| β , (16) where β = 1.3 according to [25,11]. In order to improve the computational efficiency, we use a Conjugate Gradient (CG) type method [8] to minimize the objective function given by Eq.(15).

4: Iteration
Stop the iteration when k > K max or P SN R k − P SN R k−1 ≤ δ, where K max is the maximum number of iterations and δ is taken as 0.01. Otherwise, set k = k + 1, and go to Step 2. According to our simulation, the K max = 9 is enough. 5. Experimental results. In this section, the noise detection and restoration capability of our method is evaluated and compared with many other existing methods. We have tried a group of 512 × 512 8-bit gray-scale images corrupted by random-valued impulse noise with various ratios. Our method can remove noise effectively while preserving details, and it has out-performed all other methods we tested.

5.1.
Comparison of noise detection. The capability of noise detection is very important for good performance. We compare our detection method with all the methods listed in Table 2 that have noise detectors. The table lists the detection results of each method in three parts: the number of undetected noisy pixels ("miss" term), the number of false detected noisy pixels ("false-hit" term), and the total number. A good detector should be able to find most of the true-noisy pixels, meaning that the detector should reduce both the number of "miss" and "false-hit" pixels.
In Table 2, although some methods, such as the Luo and ACWM filters, generate less "false-hit" or "miss" than ours, their total numbers of error detection are more than ours which indicate the presence of noticeable noise patches or blurred detail. In contrast, our method can strike a balance between "false-hit" and "miss". Our method can distinguish more noises with fewer mistakes even the noise ratio is as high as 60%. Fig. 4 indicates the changes of the total numbers with respect of the increase of the noisy level. We note that the proposed method is the slowestgrowing one, while for other methods, including Luo and ACWM, the increase is much rapidly. Thus our method has better precision and stability when the noisy ratio is high.

Restoration performance measurements.
Restoration results are quantitatively compared by using the peak signal-to-noise ratio (PSNR) [6] which is defined as:  Table 3 lists the PSNR values obtained from various methods for the three images with noise densities from 40% to 60%. The best values are marked by boldface. From Table 3, it is observed that our method generates the best results for all the tested images and noise ratio. To compare the results subjectively, we show images restored by various methods listed in Table 3. Fig 5 shows the output results of 'lena' image corrupted by 40% impulse noise. In order to give a better visual impression, we enlarge portion of the images. From the figure, ACWM, Luos method and ASWM methods produce many noticeable artifacts noise patches. Although no noticeable noise is observed by the DWM filter, the image becomes ugly in the edges. The ROAD-trilateral filter and ROR-NLM can remove most of the noise. However they also blur image. ROLD-EPR has a good capability of preserving the image edges, but there exist some noticeable noise, especially in the nose area. In contrast, our method performs much better. The results produced by our method have superior visual quality in regards of noise suppression and detail preservation.  Table 4 and Fig 6 are the cost comparison between detection and removal phase with different density noise. As can be seen from the graph and table, the time cost of detection phase is almost constant, independent of density. Because our detection method traverses every pixel whatever it is noise or not, the noise has no effect on time cost. The proposed will speed more time when noise density is low, however has advanced when the density is high. The proportion of detection cost reduces with growing of noise density.  Fig. 7 show the cost comparison between detection and removal phase with different scale image. We adopt 4 kinds of scale "Lena" images, 64 × 64, 128 × 128, 256 × 256, 512 × 512. The time cost grows rapidly for removal, but slowly for detection. The time of detection is relatively small in the total time.
Through the above analysis, we believe that the extra cost in detection phase for accuracy is acceptable.  6. Conclusion. In this paper, we proposed to use a robust local image statistic, called WROD. With which we can distinguish image details and edges from impulse noise effectively. Therefore it can identify noisy pixels more accurately. Then, we combined it with the edge-preserving regularization to remove random-valued