Learning to navigate a crystallization model with Deep Reinforcement Learning

In this work, a combination of a Convolutional Neural Network (CNN) based measurement sensor and a reinforcement learning (RL) framework that speeds up the control loop is presented. The objective of the controller is to reach a target mean size and to reduce the variability of the crystal sizes. The CNN based sensor improves the quality of crystal size measurement and reduces the time to process images while the RL framework learns to navigate the crystallization model optimally even in the face of disturbances. The proposed data driven strategy is validated against an unseeded crystallization of sodium chloride in water using ethanol as antisolvent in an experimental bench-scale semi-batch crystallizer. We find that the RL-based controller can be trained offline to optimize multiple target conditions while the CNN provides accurate feedback for the controller to recompute the optimal actions in the face of disturbances and guide the system towards the target.