Overcomplete representation in a hierarchical Bayesian framework

Figure 1.  The generative model (left) and the blurred and noisy data vector $ b \in {\mathbb R}^{46} $ (right)

Figure 2.  Reconstruction of the signal $ x $ (left) and the count of CGLS steps per outer iteration of the global hybrid IAS (right)

Figure 3.  Vector $ \alpha_i $ (left panels), corresponding scaled variances (middle panels) and contribution of the signal $ {\mathsf W}_i \alpha_i $ (right panels) for $ i = 1 $, i.e. representation in terms of increments, (top row) and $ i = 2 $, i.e. representation in terms of cosine transform, (bottom row)

Figure 4.  Original image (left), observed data (middle) and reconstructed image (right)

Figure 5.  Vector $ \alpha_i $ (left panels), base-10 logarithmic plot of the corresponding scaled variances (middle panels) and vector $ {\mathsf W}_i\alpha_i $ contributing to the final restoration (right panels) for $ i = 1 $, i.e. vertical increments representation, (top row), $ i = 2 $, i.e. horizontal increments representation, (bottom row)

Figure 6.  Top row: original $ 512\times512 $ test image (left), observed data (middle) and denoised image (right). Bottom row: respective close-up(s)

Figure 7.  Vector $ \alpha_i $ (left panels), base-10 logarithmic plot of the corresponding scaled variances (middle panels), and vector $ \mathsf{W}_i\alpha_i $ (right panels) for $ i = 1 $, i.e. vertical increments representation, (top row) and $ i = 2 $, horizontal increments representation, (bottom row)

Figure 8.  Value distributions of the coefficients corresponding to vertical (left) and horizontal (right) increments in logarithmic scale. The blue distributions correspond to the original image which does not allow sparse representation in the basis, which is reflected in the high percentage of non-vanishing coefficients, while the red distribution is the sparse denoised reconstruction, in which the percentage of coefficients above the negligible threshold value is reduced by an order of magnitude

Figure 9.  Original image (left), observed data (middle) and reconstructed image (right)

Figure 10.  Representation vector $ \alpha_i $ (left panels), base-10 logarithmic plot of the corresponding scaled variances (middle panels) and vector $ {\mathsf W}_i\alpha_i $ contributing to the final restoration (right panels) for $ i = 1 $ (first row), $ i = 2 $ (second row), $ i = 3 $ (third row) and $ i = 4 $ (fourth row)

Figure 11.  A schematic representation of the dictionary learning example. The digit on the right is the non-annotated image $ b $, which is approximated in terms of the annotated atoms $ w_i $ on the right. The coefficients $ \alpha_i $ can then be used to identify the digit

Figure 12.  Dictionary learning results. The first row shows the test images of the digits to be classified by the dictionary learning algorithm (vector $ b $), the true annotation indicated in the figure, the second row the vectors $ \theta $ after the IAS iteration with the first hyperprior, and the third row after the iteration with the second hyperprior. The fourth row represents the synthesis $ {\mathsf W}\alpha $ approximating the original digit, and finally, the fifth row gives the histogram of the annotations of the atoms corresponding to coefficients above a threshold $ \tau = 0.01 $. The annotation is done by majority vote, choosing the largest of the bins. In this example, the standard deviation of the noise representing the mismatch was $ \sigma = 0.01 $

Figure 13.  The rows are as in Figure 12. In this example, the standard deviation of the noise representing the mismatch was $ \sigma = 0.05 $. Observe that the approximation becomes sparser

Figure 14.  The rows are as in Figure 12. In this example, the standard deviation of the noise representing the mismatch was $ \sigma = 0.1 $. The increased sparsity here is traded with an increased number of misclassifications, such as in the first and the third columns