Title: On Estimation of ROC Curves from Likelihood Ratio Observations.
Abstract: The optimal receiver operating characteristic (ROC) curve, giving the maximum probability of detection vs. the probability of false alarm, is a key information-theoretic indicator of the difficulty of a binary hypothesis testing problem (BHT). The optimal ROC curve for a given BHT, corresponding to the likelihood ratio test, is theoretically determined by the probability distribution of the observed data under each of the two hypotheses. In some cases, these two distributions may be unknown or computationally intractable, but independent samples of the likelihood ratio can be observed. This raises the problem of estimating the optimal ROC for a BHT from such samples.
Four estimators of the ROC curve will be discussed. (1) The empirical estimator, based on separate estimates of the likelihood ratio distribution under each hypothesis, (2) The maximum likelihood estimator, and (3-4) two variants of the maximum likelihood estimator we call the split and fused estimators. All four are shown to be consistent, and finite sample size bounds are given for the empirical estimator and the variants of the maximum likelihood estimator. Based on joint work with Xiaohan Kang.