Gaussian Process Regression + Deep Neural Network Autoencoder for Probabilistic Surrogate Modeling in Nonlinear Mechanics of Solids

This work introduces an approach that combines autoencoder networks with the probabilistic regression capabilities of Gaussian processes. The autoencoder provides a low-dimensional representation of the solution space, while the Gaussian Processes (GPs) provide a probabilistic mapping between the low-dimensional inputs and outputs. We show the applications of the proposed approach in predicting the non-linear deformation of solids along with associated uncertainties.

Supplementary data is available on

Framework Overview

The framework takes force inputs and predicts full-field displacements of solid bodies along with uncertainties.

The proposed framework has two stages

1. Training:

• Autoencoder Network Training: Train the autoencoder network on full field displacement data and use the encoder part to obtain latent representations of the full field displacements.
• Gaussian Processes Training: Use the latent representations obtained using the autoencoder network to train independent GPs. The input features for the GPs are body force density vectors, each with a dimension of 3.

2. Prediction:

• Latent Displacement Prediction: Once both models are trained, for a given input force, predict distributions for latent displacements using the GPs.
• Full Field Projection: Project the latent displacements to the full field using the decoder part of the autoencoder network.

Usage Instructions

Download data

Download pretrained models and training-test data from Zenodo as follows:

curl -L -o data.zip "https://zenodo.org/records/14563089/files/data.zip?download=1"
unzip data.zip

Prediction using Pretrained Models

To predict the probabilistic displacement output for a given input force in its latent representation, use the predict.py script. By default, the script uses optimized autoencoder weights and trained GP models.

To use different pretrained weights or specify a particular test case, provide the model weights and define the test number using the following CLI arguments:

python src/predict.py --wts_path path/to/weights --test_no test_no_to_predict

Pretrained weights can be downloaded from Zenodo and are located in the data/pretrained_models directory. The autoencoder weight file is named best.h5, and the optimized GP models for all latent units are stored as GP_{latent_unit_no}.pkl in the src/pretrained_models/best_GPs directory.

Note - When using custom-trained autoencoder and GP models, it's essential to always use the same autoencoder during both training and prediction phases. This ensures consistency when reconstructing the full-field solution. Using a different autoencoder can produce different latent representations for the same data, leading to incorrect results.

Postprocessing of results

The output file generated by running predict.py can be visualized using the AceFEM library with the visualisation.m Mathematica notebook provided in the 'results' directory. Running this notebook will generate multiple plots to visualise the prediction of the GP + Autoencoder framework. They are stored in the directory corresponding to the respective test example.

Training from scratch

Training is performed in two step

1. Training of autoencoder: Optimised weights and corresponding compressed (or) latent representations are saved after the training:

python src/train.py --model autoencoder

2. Training of GPs: Train on the input forces (originally provided) and latent displacement data (latents obtained in the first step):

python src/train.py --model gp

Instead of running the above scripts manually one can directly submit jobs using the train.sh (SLURM launcher script) provided in the 'bash' directory

sbatch train.sh

The above step submits jobs for training both autoencoder and GPs in the respective order.

Once the training is finished, optimised models are used to make predictions on new cases by providing the path of the autoencoder weights file (identified with a unique timestamp generated while training and present in the 'data/training_data' directory) using predict.py.

Note: All essential parameters are located in the config.py file present in the gp_auto package and can be modified as needed.

Dependencies

The scripts have been tested running under Python 3.10 with the following packages installed (along with their dependencies).

• tensorflow==2.12
• keras==2.12
• scikit-learn==1.2.2

You can install dependencies using the 'requirement.txt' (tested for CPU), or 'gpauto.yml' (tested for GPU on Linux-based systems) files provided as follows:

1. Clone the repository:

git clone https://github.com/gp-auto-regression.git
cd gp-auto-regression

2. Set up the virtual environment and install dependencies:

Create a virtual environment with pip or conda and install dependencies using 'requirement.txt' file as follows:

conda create -n gpauto python==3.10
conda activate gpauto
pip install -e . 
#or 
pip install -r requirements.txt

Alternatively, you can create a virtual environment and install dependencies using the provided yml file (tested on Linux systems):

conda env create -f gpauto.yml
conda activate gpauto

3. AceFEM installation for data generation and postprocessing

Non-linear finite element simulations to generate the synthetic data are performed using the AceFEM library. AceFEM, which is based on Mathematica, requires Mathematica to be installed beforehand. Detailed installation instructions for AceFEM is available here.

Datasets

Training data is provided in the 'data/training_data' directory. Datsets have been obtained through non-linear FEM simulations using AceFEM framework. The pipeline to create numpy arrays of datasets will be provided soon.

Originally, the following four dataset arrays are provided.:

Data	Shape	Description
train_full_outputs.npy	(n_train, dof)	Full field displacements of training set (originally provided)
test_full_outputs.npy	(n_test, dof)	Full field displacements of test set (originally provided)
train_latent_inputs.npy	(n_train, 3)	Latent input forces of training set (originally provided)
test_latent_inputs.npy	(n_test, 3)	Latent input forces of test set (originally provided)

Compressed (encoded) representations of `Y_train.npy` and `Y_test.npy` obtained using the encoder part of the optimised autoencoder network (with best weights) are saved in the same data folder as:

Compressed Data	Shape	Description
best_train_latent_outputs.npy	(n_train, latent_dim)	Latent displacements of train set (obtained after encoding Y_train)
best_test_latent_outputs.npy	(n_test, latent_dim)	Latent displacements of test set (obtained after encoding Y_test)

For training from scratch: Compressed displacement are saved as {timestamp}_latent_outputs_train(test).npy where 'timestamp' refers to the unique timestamp generated at the start of the training procedure. It is present in the timestamp.txt. By default it is set to best.

Reference

@article{DESHPANDE2025117790,
title = {Gaussian process regression + deep neural network autoencoder for probabilistic surrogate modeling in nonlinear mechanics of solids},
journal = {Computer Methods in Applied Mechanics and Engineering},
volume = {437},
pages = {117790},
year = {2025},
issn = {0045-7825},
doi = {https://doi.org/10.1016/j.cma.2025.117790},
url = {https://www.sciencedirect.com/science/article/pii/S0045782525000623},
author = {Saurabh Deshpande and Hussein Rappel and Mark Hobbs and Stéphane P.A. Bordas and Jakub Lengiewicz},
keywords = {Surrogate modeling, Deep neural networks, Gaussian process, Autoencoders, Uncertainty quantification, Finite element method}
}