Ming's List

View Full Calendar

Statistics Seminar - Nathan Kallus " Near-Optimal Non-Parametric Sequential Tests and Confidence Sequences with Possibly Dependent Observations

Event Type
Department of Statistics
wifi event
Apr 4, 2024   3:30 pm  
Originating Calendar
Department of Statistics Event Calendar

Title: Near-Optimal Non-Parametric Sequential Tests and Confidence Sequences with Possibly Dependent Observations

 Abstract: Sequential tests and their implied confidence sequences, which are valid at arbitrary stopping times, promise flexible statistical inference and on-the-fly decision making. However, strong guarantees are limited to parametric sequential tests, which can suffer high type-I error rates in practice because reality isn't parametric, or to concentration-bound-based sequences, which are overly conservative so we get wide intervals and take too long to detect effects. We consider a classic delayed-start normal-mixture sequential probability ratio test and provide the first asymptotic (in the delay) analysis under general non-parametric data generating processes. We guarantee that type-I-error rates approach a user-specified α-level (primarily by leveraging a martingale strong invariance principle). Moreover, we show that the expected time-to-reject approaches the minimum possible among all α-level tests (primarily by leveraging a new identity inspired by discretizing Itô's lemma). Together, our results establish these (ostensibly parametric) tests as general-purpose, non-parametric, and near-optimal. We illustrate this via numerical experiments and a retrospective re-analysis of A/B tests at Netflix. Time permitting, I will discuss the application to inference using data collected adaptively by a contextual bandit algorithm.

Papers: Will mostly talk about https://arxiv.org/abs/2212.14411

Time permitting will also discuss https://arxiv.org/abs/2106.00418 and https://arxiv.org/abs/2106.01723

 Bio: Nathan Kallus is an Associate Professor at the Cornell Tech campus of Cornell University in NYC and a Research Director at Netflix. Nathan's research interests include the statistics of optimization under uncertainty, causal inference especially when combined with machine learning, sequential and dynamic decision making, and algorithmic fairness. He holds a PhD in Operations Research from MIT as well as a BA in Mathematics and a BS in Computer Science from UC Berkeley. Before coming to Cornell, Nathan was a Visiting Scholar at USC's Department of Data Sciences and Operations and a Postdoctoral Associate at MIT's Operations Research and Statistics group.

link for robots only