Title: Do GRE's Predict Physics PhDs?
A Case Study in Damned Lies and Statistics
Abstract: A recent widely publicized paper in a prominent journal used multiple regression methods to conclude "that lower than average scores on admissions exams do not imply a lower than average probability of earning a physics Ph.D". Although the authors had a large data set and checked results via two software packages, their conclusion was opposite to the implication of their data. One key misstep was failure to specify the causal question they were trying to answer. Specific common errors include introduction of collider stratification bias, variance inflation by collinearity and range restriction, and blurring the distinction between failure to reject a null and accepting a null, as well as some less common errors. The take-home lesson is the importance of getting the basic logic correct before starting to crunch numbers.
