Data quality issues are common in insurance data. As actuaries we are very familiar with common data problems – missing values, wrong values, incorrectly coded data, gaps in the data – those which arise in our day-to-day work. Such data “hygiene” issues have received considerable attention in actuarial circles.
This session investigates a different type of data issue, often referred to as selection bias or sampling bias in the statistical literature. This issue has significant implications for the performance of predictive models and has not seen much coverage in the actuarial literature. Selection bias arises from the discrepancy between the sample on which the model is built and the target population to which the model will be applied. For example, can a model developed in Massachusetts in 2010 to predict whether a workers’ compensation claim is fraudulent be extrapolated to other states and future time periods? When selection bias is present in the modeling data, the results of traditional modeling approaches will reflect this inherent bias. It is important to understand and consider the potential for bias whenever the model is intended to make a prediction that will be applied to a broader population.
Gaetan Veilleux will discuss selection bias in the context of insurance modeling from a traditional perspective. He will define selection bias, provide some history on the topic, illustrate real-world examples inside and outside of insurance, and examine techniques that have been proposed to identify and/or adjust for the presence of selection bias.
Herbert Weisberg will discuss a new interpretation of the nature of selection bias based on the “counterfactual” perspective, which brings notions of causality back into statistical analysis. He will argue that complex probabilistic modeling cannot provide a quick-fix for selection bias. The fundamental problem is primarily conceptual rather than technical. From the counterfactual perspective, this problem is inherent in the limitations of the classical statistical framework. As a practical matter, judgment based on subject-matter expertise, and focused application of empirical validation techniques, are the most effective prophylactics against selection bias.