Background and Aims

For an advanced artificial pancreas, the automation of insulin treatment is of great interest in clinical and research aspects. Despite recent algorithmic advances, there are fundamental challenges to overcome, such as uncertainties from unannounced meals and delays from "insulin stacking." In this study, we propose a reinforcement based automated insulin infusion algorithm.

Methods

We devise a bio-inspired reinforcement learning approach where a reward function mimics the temporal homeostatic objective of β-cells and a discount rate reflects individual specific pharmacokinetics/pharmacodynamics (PK/PD) characteristics. The proposed method is evaluated on 10 adults from the US-FDA approved UVa/Padova simulator under a single-meal with preprandial fasting scenario.

Results

The trained algorithm achieves a mean glucose of 117.21 mg/dl. At the beginning of the fasting phase, the average patient's sensor glucose is 121.61 mg/dl, and it is stabilized to basal state with 100.41 mg/dl under the automated regulation. For the postprandial phase without any meal announcement, the trained algorithm automatically regulates the postprandial glucose with a sudden peak of the insulin infusion rate which is followed by a gradually decreasing rate. About three hours after meal-intakes, the infusion rate becomes the lowest and results the patients to avoid hypoglycemia.

Conclusions

The proposed insulin infusion algorithm based on the bio-inspired reinforcement learning approach allows for fully automated glucose control with hypoglycemic avoidance.

Acknowledgments: This work was supported by the MSIT, Korea, under the ICT Consilience Creative program (IITP-2019-2011-1-00783), the Basic Science Research Program through the NRF funded by the MSIT (NRF-2017R1A5A1015596), and the POSCO Green Science Project funded by POSCO, Korea.