Abstract
Background and Aims
If individuals with diabetes want to be able to take measures in anticipation of forthcoming hypo- and hyperglycaemic events, it is essential to predict future blood glucose levels. We now report the first application of deep reinforcement learning to improve prediction of blood glucose made by a pre-trained deep neural network based on long-short term memory (LSTM).
Methods
In support of a pre-trained LSTM sequential model, an adjustment module based on Double Deep Q-Learning (DDQN) was introduced to fine tune the prediction of blood glucose. The algorithm was trained and tested on retrospectively collected data from 6 real-patient (OhioT1DM Datasets). Both LSTM and DDQN models use 4 history glucose data (look-back, sampling time 5 minutes); the former predicts a “basic” glucose value for a prediction horizon of 30 minutes, and the latter adjust, if needed, the “basic” prediction.
Results
As shown in Table 1, the DDQN model improved the performance of the LSTM sequential model. The Time Lags were decreased - without obvious influence on the Root Mean Square Error or the Correlation Coefficient (CC).
Conclusions
A deep reinforcement learning algorithm has been introduced to enhance the performance of a pre-trained deep LSTM network in predicting blood glucose. The preliminary results are promising, especially with respect to the delay between predicted value and real data.
