An out-of-distribution-aware autoencoder model for reduced chemical kinetics

Figure 1.  Elemental mass conservation errors. An elemental error of 20% is expected to cause large errors in temperature prediction thus false extinction/ignition events in engine simulations. With OOD-aware model, non-physical solutions with conservation laws violated can be detected before leading to poor engine designs

Figure 2.  Estimation of 95% PI prediction in toy regression task $ y = x^3 + \varepsilon $ with asymmetric noise $ \varepsilon $. The 95% PI produced by PI3NN captures about 95% of training data with tight bounds within training region, while DE produces an unnecessarily wide lower bound. Both methods produce reasonably wide PIs in the OOD region

Figure 3.  Architecture diagrams of AE with DE and PI3NN. Neurons are represented by the circles with different edge colors for different types of layers, i.e., red for input and output layers, green for bottleneck layer, and blue for other hidden layers. The red circles filled with gray are outputs for variance/standard deviation variables. The diagrams shown here are simplified for illustration only. The actual numbers of layers and neurons in each layer vary in different experiments

Figure 4.  Comparison of OOD detection accuracy in MNIST (ID) and Fashion-MNIST (OOD) test sets between DE and PI3NN methods. Both methods capture the difference between ID and OOD with larger uncertainties for OOD samples. The confusion matrices show that the majority of test samples are correctly detected as OOD (true positive) or ID (true negative)

Figure 5.  Receiver operating characteristic curves of DE and PI3NN in OOD detection. All methods can capture the OOD samples well in the image experiment. Though DE with ensemble size of 2 shows better accuracy than PI3NN, it requires two runs while PI3NN only needs a single run. DE with a single run shows lower OOD detection accuracy than PI3NN

Figure 6.  Normalized joint histogram of predicted values by AE (with $ n_z = 2 $) and true values for 12 thermo-chemical state variables, i.e., temperature and mass fractions of 11 species, in PSR test set. The dark red area along the diagonal line shows that AE can reduce the state dimension of syngas CO/H$ _2 $ combustion from $ n_x = 12 $ to $ n_z = 2 $ without much loss of accuracy

Figure 7.  Predictive uncertainty (the width of PI) vs. predictive error of the ID (red) and OOD (blue) test sets from the PI3NN and DE methods. Scatter points are samples randomly selected from the two test sets. The filled contours show the normalized joint histogram for samples in ID/OOD test sets, with light red/blue colors representing fewer samples and dark red/blue representing more samples. DE with a single run $ \left((a)-(c)\right) $ fails to capture the difference in uncertainty for ID and OOD samples. DE with 10 runs $ \left((d)-(f)\right) $ shows improved but still limited separation of OOD samples from ID samples and it fails to produce the uncertainty-error correlation. PI3NN $ \left((g)-(i)\right) $ shows a strong correlation between the uncertainty and the error and clearly demonstrates that OOD and ID have different uncertainty magnitudes

Figure 8.  Receiver operating characteristic curves of PI3NN and DE ($ M = 1, 10, 20 $) methods in OOD detection. PI3NN shows the best OOD detection accuracy. DE shows improved detection accuracy with increasing ensemble size ($ M $)