Data mining reconsidered: encompassing and the general-to-specific approach to specification search

ABSTRACT

This paper examines the efficacy of the general-to-specific modeling approachassociated with the LSE school of econometrics using a simulation framework. A mechanical algorithm is developed which mimics some aspects of the search procedures used by LSE practitioners. The algorithm is tested using 1000 replications of each of nine regression models and a data set patterned after Lovell's (1983) study of data mining. The algorithm is assessed for its ability to recover the data-generating process. Monte Carlo estimatesof the size and power of exclusion tests based on t-statistics for individual variables in the specification are also provided. The roles of alternative sizes for specification tests in the algorithm, the consequences of different signal-to-noise ratios, and strategies for reducing overparameterization are also investigated. The results are largely favorable to the general-to-specific approach. In particular, the size of exclusion tests remains close to the nominal size used in the algorithm despite extensive search.