Fixed effects estimators are frequently used to combat selection bias by eliminating large swaths of variation in observational data. For example, it is well-known that unit fixed effects in panel data discard all between-unit variation, resulting in estimates of an independent variable's effect as it changes within units over time. In this article, we replicate several recent studies which used fixed effects estimators to show how descriptions of the substantive significance of results can be improved by precisely characterizing the variation being studied and presenting plausible counterfactuals---i.e., shifts in the independent variable likely to occur within the variation being used for estimation---when assessing the substantive impact of the treatment. We provide a checklist for the interpretation of fixed effects regression results to help avoid these interpretative pitfalls.