Two observational methods are currently being used to monitor post-deployment vaccine effectiveness: the obvious crude method comparing rate testing positive per head of vaccinated population with that rate per head of unvaccinated population; and the test-negative case control (TNCC) method. The two methods give very different results. We want to know whether either method is reliable. We assume either a homogeneous population or one partitioned into two homogeneous subsets which differ only in their not-directly-observable healthcare-seeking behaviour including probability of getting vaccinated. We first consider uniform independent priors on the probabilities of being hospitalised conditional on subset, vaccination status, and infection status. We simulate from the resulting model and observe the TNCC estimate, the crude estimate, and the Bayesian central 95% confidence interval on vaccine effectiveness represented as log ratio of odds ratios for infection with and without vaccination. With these wide open priors, even when the population is homogeneous, the Bayesian 95% confidence interval typically has a width of nearly 4 nats (55-fold), implying too much uncertainty for the data collected to be of any use in monitoring effectiveness. There do exist some tight priors under which the data is useful: some lead to TNCC being more accurate while with others the crude estimate is more accurate. Thus using only data from those spontaneously choosing to be tested, we find that neither method is reliably better than the other, and indeed that the desired information is not present in this data. We conclude that effective monitoring of vaccine effectiveness and side-effects requires either strong information on the population's behaviour, or ongoing randomised controlled trials (RCTs), rather than just choosing whichever of TNCC and crude estimate gives the result we prefer to find.

翻译：目前有两种观察性方法用于监测疫苗部署后的有效性：一种是显而易见的粗略方法，即比较已接种人群中检测阳性率与未接种人群中检测阳性率；另一种是检验阴性病例对照（TNCC）方法。这两种方法得出的结果差异很大。我们想知道这些方法是否可靠。我们假设人群是同质的，或分为两个同质子集，这两个子集的差异仅在于不可直接观察的求医行为（包括接种疫苗的概率）。首先，我们假设在给定子集、接种状态和感染状态条件下住院概率服从均匀独立先验分布。通过从该模型模拟并观察TNCC估计值、粗略估计值以及贝叶斯中心95%置信区间（以接种与未接种感染比值比的对数比表示疫苗有效性）。在如此宽泛的先验条件下，即使人群是同质的，贝叶斯95%置信区间的宽度通常接近4纳特（55倍），这意味着数据收集带来的不确定性过大，无法用于有效性监测。确实存在一些紧先验使数据具有实用性：某些情况下TNCC更准确，而另一些情况下粗略估计更准确。因此，仅利用自发选择接受检测者的数据，我们发现两种方法并无可靠优劣之分，且所需信息实际上并不存在于这些数据中。我们得出结论：有效监测疫苗有效性和副作用需要强有力的人群行为信息，或持续进行的随机对照试验（RCTs），而非仅凭偏好从TNCC和粗略估计中选取结果。