Robust Optimal Well Control using an Adaptive Multigrid Reinforcement Learning Framework

Atish Dixit*, Ahmed H. Elsheikh

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)
17 Downloads (Pure)


Reinforcement learning (RL) is a promising tool for solving robust optimal well control problems where the model parameters are highly uncertain and the system is partially observable in practice. However, the RL of robust control policies often relies on performing a large number of simulations. This could easily become computationally intractable for cases with computationally intensive simulations. To address this bottleneck, an adaptive multigrid RL framework is introduced which is inspired by principles of geometric multigrid methods used in iterative numerical algorithms. RL control policies are initially learned using computationally efficient low-fidelity simulations with coarse grid discretization of the underlying partial differential equations (PDEs). Subsequently, the simulation fidelity is increased in an adaptive manner towards the highest fidelity simulation that corresponds to the finest discretization of the model domain. The proposed framework is demonstrated using a state-of-the-art, model-free policy-based RL algorithm, namely the proximal policy optimization algorithm. Results are shown for two case studies of robust optimal well control problems, which are inspired from SPE-10 model 2 benchmark case studies. Prominent gains in computational efficiency are observed using the proposed framework, saving around 60-70% of the computational cost of its single fine-grid counterpart.

Original languageEnglish
Pages (from-to)345-375
Number of pages31
JournalMathematical Geosciences
Issue number3
Early online date4 Nov 2022
Publication statusPublished - Apr 2023


  • Adaptive
  • Multigrid framework
  • Reinforcement learning
  • Robust optimal control
  • Transfer learning

ASJC Scopus subject areas

  • Mathematics (miscellaneous)
  • Earth and Planetary Sciences(all)


Dive into the research topics of 'Robust Optimal Well Control using an Adaptive Multigrid Reinforcement Learning Framework'. Together they form a unique fingerprint.

Cite this