Applying a Surrogate Model to an RL-Controlled Pulse

Applying a Surrogate Model to an RL-Controlled Pulse

After conducting an experiment on the DIII-D tokamak with our Reinforcement Learning (RL) model for plasma control, and training a surrogate model to reconstruct plasma shape, we became curious how these models work together.

So, we applied the surrogate model to reconstruct the shape of the plasma controlled using our RL controller. The surrogate model demonstrated promising performance in reconstructing the plasma boundary, achieving a mean boundary point displacement of 0.07 meters during the discharge. Interestingly, this is slightly lower in quality compared to our model’s reconstructions for DIII-D experiments with conventional control algorithms, where the model achieves a mean error of 0.05 meters.

This observation reminds us the importance of providing representative and diverse training datasets for ML models, as their generalization power is closely tied to the quality and coverage of the data used during training. Overall, we see the great potential in integrating supervised ML and RL techniques in fusion research to advance plasma control and optimization.

See the chart with the comparison of the ground-truth boundary (blue) and the reconstructed boundary (red), illustrating the capabilities of our model.