Mobile health (mHealth) technologies aim to improve distal outcomes, such as clinical conditions, by optimizing proximal outcomes through just-in-time adaptive interventions. Contextual bandits provide a suitable framework for customizing such interventions according to individual time-varying contexts, intending to maximize cumulative proximal outcomes. However, unique challenges such as modeling count outcomes within bandit frameworks have hindered the widespread application of contextual bandits to mHealth studies. The current work addresses this challenge by leveraging count data models into online decision-making approaches. Specifically, we combine four common offline count data models (Poisson, negative binomial, zero-inflated Poisson, and zero-inflated negative binomial regressions) with Thompson sampling, a popular contextual bandit algorithm. The proposed algorithms are motivated by and evaluated on a real dataset from the Drink Less trial, where they are shown to improve user engagement with the mHealth system. The proposed methods are further evaluated on simulated data, achieving improvement in maximizing cumulative proximal outcomes over existing algorithms. Theoretical results on regret bounds are also derived. A user-friendly R package countts that implements the proposed methods for assessing contextual bandit algorithms is made publicly available at https://cran.r-project.org/web/packages/countts.

翻译：移动健康（mHealth）技术旨在通过即时自适应干预优化近端结局，从而改善远端结局（如临床状况）。情境赌博机为根据个体时变情境定制此类干预措施提供了合适的框架，旨在最大化累积近端结局。然而，在赌博机框架中对计数结果进行建模等独特挑战，阻碍了情境赌博机在移动健康研究中的广泛应用。本研究通过将计数数据模型融入在线决策方法中解决了这一难题。具体而言，我们将四种常见的离线计数数据模型（泊松回归、负二项回归、零膨胀泊松回归及零膨胀负二项回归）与流行的情境赌博机算法汤普森采样相结合。所提出的算法受Drink Less试验的真实数据集启发并进行评估，结果显示其能提升用户对移动健康系统的参与度。进一步在模拟数据上评估所提方法，发现其在最大化累积近端结局方面优于现有算法。此外，还推导了遗憾界的理论结果。一个用户友好的R包countts（用于评估情境赌博机算法所提方法的实现）已在https://cran.r-project.org/web/packages/countts公开提供。