Using computational methods to study social structure and behavior at scale requires researchers to make a plethora of decisions, including how to sample and preprocess data, implement algorithms, and validate results. I present findings and lessons learned from my group’s work on assessing the impact of some of these choices, especially related to data provenance and selecting variables and metrics, on understanding research collaborations and validating social science theories in contemporary settings. I highlight sources of biases and strategies for mitigating biased insights.