A Bayesian multiscale deep learning framework for flows in random media

Figure 1.  Schematic of the fine-scale and coarse-scale grids. Thick lines represent the primal coarse-grid $ \bar{\Omega}_j $, and the blue line indicates a block of the dual coarse-grid $ \Omega_k^D $. Thin lines define the fine-scale elements $ f_i $ that constitute the fine-grid $ \{\Omega_i\}_{i = 1}^{n_f} $

Figure 2.  A schematic of the multiscale framework

Figure 3.  (a) Discretization of the domain: fine-scale domain (black bold lines correspond to the coarse-grid ($ \bar{\Omega}_j $) and thin lines correspond to the fine-grid $ \Omega_i $), coarse-blocks and fine-cells (b) Local triangulation (indicated in purple) and coarse-block centers (indicated in black) (c) Cells inside the support region are indicated in blue patch, the support boundary is indicated in green patch and the coarse center node is indicated in black for the corresponding coarse-block and (d) Global boundary (indicated in gray) and coarse-block center (indicated in black)

Figure 4.  The basis functions for the interior and non-interior support regions: (a) Coarse-blocks ($ 3\times3 $) and basis function for coarse-block $ 5 $ (basis function for the interior support region), (b) Coarse-blocks ($ 3\times3 $) and basis function for coarse-block $ 1 $ (basis function for the non-interior support region) and (c) Illustration of the interior support regions (shown in the green patch) where the basis functions are computed using the Deep Learning surrogate, and non-interior support regions (shown in the blue patch) where the basis functions are computed using the multiscale solver

Figure 5.  A schematic of the DenseED network. (a) The top block shows the DenseED architecture with the encoding layers, and the bottom block shows the decoding layers containing convolution, Batch Norm, and ReLU. The convolution in the encoding layer reduces the size of the feature map, and the convolution (ConvT) in the decoding layer performs up-sampling. (b) The dense block also contains convolution, Batch Norm, and ReLU. The main difference between the encoding or decoding layer and the dense block is that the size of the feature maps is the same as the input in the dense block. Lastly, we apply the sigmoid activation function at the end of the last decoding layer

Figure 6.  Comparison of the standard multiscale framework with the data-driven hybrid multiscale DenseED framework

Figure 7.  A schematic of the hybrid deep neural network- multiscale framework using DenseED. Parameters $ \mathit{\boldsymbol{A}} $, $ \mathit{\boldsymbol{q}} $ and $ \bar{\mathit{\boldsymbol{\Phi}}}^{non-int} $ are obtained from MRST [35] (magenta dashed line) and the network is trained using Pytorch (implementation is marked in blue dashed line)

Figure 8.  Permeability field KLE$ -100 $ (top left), KLE$ -1000 $ (top right), KLE$ -16384 $ (bottom left) and channelized (bottom right)

Figure 9.  Permeability coarse block (top) for KLE$ -100 $, KLE$ -1000 $, KLE$ -16384 $ and channelized field and the corresponding basis functions (bottom)

Figure 10.  HM-DenseED model: Prediction of KLE$ -100 $ with $ 32 $ training data: first row (from left to right) shows the target (pressure, $ x- $velocity and $ y- $velocity components), the second row shows the corresponding predictions, and the last row shows the error between the corresponding targets and predictions

Figure 11.  HM-DenseED model: Prediction of KLE$ -100 $ with $ 96 $ training data: first row (from left to right) shows the target (pressure, $ x- $velocity and $ y- $velocity components), the second row shows the corresponding predictions, and the last row shows the error between the corresponding targets and predictions

Figure 12.  HM-DenseED model: Prediction of KLE$ -1000 $ with $ 64 $ training data: first row (from left to right) shows the target (pressure, $ x- $velocity and $ y- $velocity components), the second row shows the corresponding predictions, and the last row shows the error between the corresponding targets and predictions

Figure 13.  HM-DenseED model: Prediction of KLE$ -1000 $ with $ 128 $ training data: first row (from left to right) shows the target (pressure, $ x- $velocity and $ y- $velocity components), the second row shows the corresponding predictions, and the last row shows the error between the corresponding targets and predictions

Figure 14.  HM-DenseED model: Prediction of KLE$ -16384 $ with $ 96 $ training data: first row (from left to right) shows the target (pressure, $ x- $velocity and $ y- $velocity components), the second row shows the corresponding predictions, and the last row shows the error between the corresponding targets and predictions

Figure 15.  HM-DenseED model: Prediction of KLE$ -16384 $ with $ 160 $ training data: first row (from left to right) shows the target (pressure, $ x- $velocity and $ y- $velocity components), the second row shows the corresponding predictions, and the last row shows the error between the corresponding targets and predictions

Figure 16.  HM-DenseED model: Prediction of channelized field with $ 160 $ training data: first row (from left to right) shows the target (pressure, $ x- $velocity and $ y- $velocity components), the second row shows the corresponding predictions, and the last row shows the error between the corresponding targets and predictions

Figure 17.  The basis function for KLE$ -100 $. The first row shows the ground truth basis function, HM-DenseED model predicted basis function, and the error between them for the model trained with $ 32 $ training data. The second row shows the ground truth basis function, HM-DenseED model predicted basis function, and the error between them for the model trained with $ 64 $ training data. The last row shows the ground truth basis function, HM-DenseED model predicted basis function, and the error between them for the model trained with $ 96 $ training data

Figure 18.  The basis function for KLE$ -1000 $. The first row shows the ground truth basis function, HM-DenseED model predicted basis function, and the error between them for the model trained with $ 64 $ training data. The second row shows the ground truth basis function, HM-DenseED model predicted basis function, and the error between them for the model trained with $ 96 $ training data. The last row shows the ground truth basis function, HM-DenseED model predicted basis function, and the error between them for the model trained with $ 128 $ training data

Figure 19.  The basis function for KLE$ -16384 $. The first row shows the ground truth basis function, HM-DenseED model predicted basis function, and the error between them for the model trained with $ 96 $ training data. The second row shows the ground truth basis function, HM-DenseED model predicted basis function, and the error between them for the model trained with $ 128 $ training data. The last row shows the ground truth basis function, HM-DenseED model predicted basis function, and the error between them for the model trained with $ 160 $ training data

Figure 20.  The basis function for the channelized field. Here, we show the ground truth basis function, HM-DenseED model predicted basis function, and the error between them for the model trained with $ 160 $ training data

Figure 21.  Distribution estimate for the pressure for KLE$ -100 $. Location: $ (0.6,0.4) $. Here, the Monte Carlo result is shown in the blue dashed line; the hybrid DenseED result is shown in green circles, and the DenseED model result is shown in magenta dashed line

Figure 22.  Distribution estimate for the pressure for KLE$ -1000 $. Location: $ (0.6,0.4) $. Here, the Monte Carlo result is shown in the blue dashed line; the hybrid DenseED result is shown in green circles, and the DenseED model result is shown in magenta dashed line

Figure 23.  Distribution estimate for the pressure for KLE$ -16384 $. Location: $ (0.6,0.4) $. Here, the Monte Carlo result is shown in the blue dashed line; the hybrid DenseED result is shown in green circles, and the DenseED model result is shown in magenta dashed line

Figure 24.  Distribution estimate for the pressure for channelized flow. Location: $ (0.6,0.4) $. Here, the Monte Carlo result is shown in the blue dashed line; the hybrid DenseED result is shown in green circles, and the DenseED model result is shown in magenta dashed line

Figure 25.  Distribution estimate for the $ x- $velocity component (horizontal flux) for KLE$ -100 $. Location: $ (0.6,0.4) $. Here, the Monte Carlo result is shown in the blue dashed line; the hybrid DenseED result is shown in green circles, and the DenseED model result is shown in magenta dashed line

Figure 26.  Distribution estimate for the $ x- $velocity component (horizontal flux) for KLE$ -1000 $. Location: $ (0.6,0.4) $. Here, the Monte Carlo result is shown in the blue dashed line; the hybrid DenseED result is shown in green circles, and the DenseED model result is shown in magenta dashed line

Figure 27.  Distribution estimate for the $ x- $velocity component (horizontal flux) for KLE$ -16384 $. Location: $ (0.6,0.4) $. Here, the Monte Carlo result is shown in the blue dashed line; the hybrid DenseED result is shown in green circles, and the DenseED model result is shown in magenta dashed line

Figure 28.  Distribution estimate for the $ x- $velocity component (horizontal flux) for channelized flow. Location: $ (0.6,0.4) $. Here, the Monte Carlo result is shown in the blue dashed line; the hybrid DenseED result is shown in green circles, and the DenseED model result is shown in magenta dashed line

Figure 29.  Distribution estimate for the $ y- $velocity component (vertical flux) for KLE$ -100 $. Location: $ (0.6,0.4) $. Here, the Monte Carlo result is shown in the blue dashed line; the hybrid DenseED result is shown in green circles, and the DenseED model result is shown in magenta dashed line

Figure 30.  Distribution estimate for the $ y- $velocity component (vertical flux) for KLE$ -1000 $. Location: $ (0.6,0.4) $. Here, the Monte Carlo result is shown in the blue dashed line; the hybrid DenseED result is shown in green circles, and the DenseED model result is shown in magenta dashed line

Figure 31.  Distribution estimate for the $ y- $velocity component (vertical flux) for KLE$ -16384 $. Location: $ (0.6,0.4) $. Here, the Monte Carlo result is shown in the blue dashed line; the hybrid DenseED result is shown in green circles, and the DenseED model result is shown in magenta dashed line

Figure 32.  Distribution estimate for the $ y- $velocity component (vertical flux) for channelized flow. Location: $ (0.6,0.4) $. Here, the Monte Carlo result is shown in the blue dashed line; the hybrid DenseED result is shown in green circles, and the DenseED model result is shown in magenta dashed line

Figure 33.  Training and testing RMSE plot for KLE$ -100 $ ($ 64- $training data), KLE$ -1000 $ ($ 96- $training data), KLE$ -16384 $ ($ 128- $training data) and channelized field ($ 160- $ training data)

Figure 34.  Comparison of test $ R^2 $ scores (for pressure) for HM-DenseED (left) and DenseED (right) for KLE$ -100 $, $ -1000 $, $ -16384 $ and channelized permeability field and for various training data

Figure 35.  Prediction of KLE$ -100 $ with $ 32 $ training data (left), and $ 96 $ training data (right); For individual prediction statistics, from left to right (first row): For single test input $ \mathit{\boldsymbol{K}}^{*} $, test output (ground truth) $ t^{*} $, Predictive mean $ \mathbb{E}[\widehat{{\mathit{\boldsymbol{P}}}_f}^{*}|\mathit{\boldsymbol{K}}^{*},\mathcal{{{D}}}] $, from left to right (second row) Error: Predictive mean and test output (ground truth), and predictive variance $ \text{Var}(\widehat{{\mathit{\boldsymbol{P}}}_f}^{*}|\mathit{\boldsymbol{K}}^{*},\mathcal{{{D}}}) $

Figure 36.  Prediction of KLE$ -1000 $ with $ 64 $ training data (left), and $ 128 $ training data (right); For individual prediction statistics, from left to right (first row): For single test input $ \boldsymbol{K}^{*} $, test output (ground truth) $ t^{*} $, predictive mean $ \mathbb{E}[\widehat{{\boldsymbol{P}}_f}^{*}|\boldsymbol{K}^{*},\mathcal{{D}}] $, from left to right (second row) Error: Predictive mean and test output (ground truth), and Predictive variance $ \text{Var}(\widehat{{\boldsymbol{P}}_f}^{*}|\boldsymbol{K}^{*},\mathcal{{D}}) $

Figure 37.  Prediction of KLE$ -16384 $ with $ 96 $ training data (left), and $ 160 $ training data (right); For individual prediction statistics, from left to right (first row): For single test input $ \mathit{\boldsymbol{K}}^{*} $, test output (ground truth) $ t^{*} $, Predictive mean $ \mathbb{E}[\widehat{{\mathit{\boldsymbol{P}}}_f}^{*}|\mathit{\boldsymbol{K}}^{*},\mathcal{{{D}}}] $, from left to right (second row) Error: Predictive mean and test output (ground truth), and Predictive variance $ \text{Var}(\widehat{{\mathit{\boldsymbol{P}}}_f}^{*}|\mathit{\boldsymbol{K}}^{*},\mathcal{{{D}}}) $

Figure 38.  Prediction for channelized field; For individual prediction statistics, from left to right (first row): For single test input $ \boldsymbol{K}^{*} $, test output (ground truth) $ t^{*} $, predictive mean $ \mathbb{E}[\widehat{{\boldsymbol{P}}_f}^{*}|\boldsymbol{K}^{*},\mathcal{{D}}] $, from left to right (second row) Error: Predictive mean and test output (ground truth), and predictive variance $ \text{Var}(\widehat{{\boldsymbol{P}}_f}^{*}|\boldsymbol{K}^{*},\mathcal{{D}}) $

Figure 39.  MNLP of test data

Figure 40.  Non-Bayesian and Bayesian test $ R^2 $ scores for KLE$ -100 $, $ -1000 $, $ -16384 $ and channelized field (Hybrid DenseED model)

Figure 41.  Bayesian HM-DenseED: Training and testing RMSE plot for KLE$ -100 $ ($ 64- $training data), KLE$ -1000 $ ($ 96- $training data), KLE$ -16384 $ ($ 128- $training data) and channelized field ($ 160- $ training data)

Figure 42.  (Left) Uncertainty propagation for KLE$ -100 $ ($ 32 $ training data). We show the Monte Carlo output mean, predictive output mean $ \mathbb{E}_{\mathit{\boldsymbol{\theta}}}[\mathbb{E}[\mathit{\boldsymbol{y}}|\mathit{\boldsymbol{\theta}}]] $, the error of the above two, and two standard deviations of the conditional predictive mean $ Var_{\mathit{\boldsymbol{\theta}}}[\mathbb{E}[\mathit{\boldsymbol{y}}|\mathit{\boldsymbol{\theta}}]] $. (Right) Uncertainty propagation for KLE$ -100 $: ($ 64 $ training data) we show the Monte Carlo output variance, predictive output variance $ \mathbb{E}_{\mathit{\boldsymbol{\theta}}}[Var(\mathit{\boldsymbol{y}} | \mathit{\boldsymbol{\theta}})] $, the error of the above two, and two standard deviations of the conditional predictive variance $ \text{Var}_{\mathit{\boldsymbol{\theta}}} (\text{Var}(\mathit{\boldsymbol{y}} | \mathit{\boldsymbol{\theta}})) $

Figure 43.  Uncertainty propagation for KLE$ -100 $ ($ 96 $k__ge training data). (Left) We show the Monte Carlo output mean, predictive output mean $ \mathbb{E}_{\boldsymbol{\theta}}[\mathbb{E}[\boldsymbol{y}|\boldsymbol{\theta}]] $, the error of the above two, and two standard deviations of the conditional predictive mean $ Var_{\boldsymbol{\theta}}[\mathbb{E}[\boldsymbol{y}|\boldsymbol{\theta}]] $. (Right) We show the Monte Carlo output variance, predictive output variance $ \mathbb{E}_{\boldsymbol{\theta}}[Var(\boldsymbol{y} | \boldsymbol{\theta})] $, the error of the above two, and two standard deviations of the conditional predictive variance $ \text{Var}_{\boldsymbol{\theta}} (\text{Var}(\boldsymbol{y} | \boldsymbol{\theta})) $

Figure 44.  Uncertainty propagation for KLE$ -1000 $ ($ 64 $k__ge training data). (Left) We show the Monte Carlo output mean, predictive output mean $ \mathbb{E}_{\boldsymbol{\theta}}[\mathbb{E}[\boldsymbol{y}|\boldsymbol{\theta}]] $, the error of the above two, and two standard deviations of the conditional predictive mean $ Var_{\boldsymbol{\theta}}[\mathbb{E}[\boldsymbol{y}|\boldsymbol{\theta}]] $. (Right) We show the Monte Carlo output variance, predictive output variance $ \mathbb{E}_{\boldsymbol{\theta}}[Var(\boldsymbol{y} | \boldsymbol{\theta})] $, the error of the above two, and two standard deviations of the conditional predictive variance $ \text{Var}_{\boldsymbol{\theta}} (\text{Var}(\boldsymbol{y} | \boldsymbol{\theta})) $

Figure 45.  Uncertainty propagation for KLE$ -1000 $ ($ 128 $k__ge training data). (Left) We show the Monte Carlo output mean, predictive output mean $ \mathbb{E}_{\boldsymbol{\theta}}[\mathbb{E}[\boldsymbol{y}|\boldsymbol{\theta}]] $, the error of the above two, and two standard deviations of the conditional predictive mean $ Var_{\boldsymbol{\theta}}[\mathbb{E}[\boldsymbol{y}|\boldsymbol{\theta}]] $. (Right) We show the Monte Carlo output variance, predictive output variance $ \mathbb{E}_{\boldsymbol{\theta}}[Var(\boldsymbol{y} | \boldsymbol{\theta})] $, the error of the above two, and two standard deviations of the conditional predictive variance $ \text{Var}_{\boldsymbol{\theta}} (\text{Var}(\boldsymbol{y} | \boldsymbol{\theta})) $

Figure 46.  Uncertainty propagation for KLE$ -16384 $ ($ 96 $k__ge training data). (Left) We show the Monte Carlo output mean, predictive output mean $ \mathbb{E}_{\boldsymbol{\theta}}[\mathbb{E}[\boldsymbol{y}|\boldsymbol{\theta}]] $, the error of the above two, and two standard deviations of the conditional predictive mean $ Var_{\boldsymbol{\theta}}[\mathbb{E}[\boldsymbol{y}|\boldsymbol{\theta}]] $. (Right) We show the Monte Carlo output variance, predictive output variance $ \mathbb{E}_{\boldsymbol{\theta}}[Var(\boldsymbol{y} | \boldsymbol{\theta})] $, the error of the above two, and two standard deviations of the conditional predictive variance $ \text{Var}_{\boldsymbol{\theta}} (\text{Var}(\boldsymbol{y} | \boldsymbol{\theta})) $

Figure 47.  Uncertainty propagation for KLE$ -16384 $ ($ 160 $ training data). (Left) We show the Monte Carlo output mean, predictive output mean $ \mathbb{E}_{\boldsymbol{\theta}}[\mathbb{E}[\boldsymbol{y}|\boldsymbol{\theta}]] $, the error of the above two, and two standard deviations of the conditional predictive mean $ Var_{\boldsymbol{\theta}}[\mathbb{E}[\boldsymbol{y}|\boldsymbol{\theta}]] $. (Right) We show the Monte Carlo output variance, predictive output variance $ \mathbb{E}_{\boldsymbol{\theta}}[Var(\boldsymbol{y} | \boldsymbol{\theta})] $, the error of the above two, and two standard deviations of the conditional predictive variance $ \text{Var}_{\boldsymbol{\theta}} (\text{Var}(\boldsymbol{y} | \boldsymbol{\theta})) $

Figure 48.  Uncertainty propagation for channelized field ($ 160 $ training data). (Left) We show the Monte Carlo output mean, predictive output mean $ \mathbb{E}_{\mathit{\boldsymbol{\theta}}}[\mathbb{E}[\mathit{\boldsymbol{y}}|\mathit{\boldsymbol{\theta}}]] $, the error of the above two, and two standard deviations of the conditional predictive mean $ Var_{\mathit{\boldsymbol{\theta}}}[\mathbb{E}[\mathit{\boldsymbol{y}}|\mathit{\boldsymbol{\theta}}]] $. (Right) We show the Monte Carlo output variance, predictive output variance $ \mathbb{E}_{\mathit{\boldsymbol{\theta}}}[Var(\mathit{\boldsymbol{y}} | \mathit{\boldsymbol{\theta}})] $, the error of the above two, and two standard deviations of the conditional predictive variance $ \text{Var}_{\mathit{\boldsymbol{\theta}}} (\text{Var}(\mathit{\boldsymbol{y}} | \mathit{\boldsymbol{\theta}})) $

Figure 49.  Comparison of DenseED (first row) and fully-connected network (second row) for a test set

Figure 50.  Distribution estimate for the pressure at location $ (0.96, 0.54) $