Prof. Bryan Caplan

Econ 345

Fall, 1998

Week 15: Being An Intelligent Consumer of Econometrics

  1. Economic Significance vs. Statistical Significance
    1. For most of the class, we've checked for statistical significance - i.e., whether the results were likely to occur purely by chance.
    2. But it is at least as important to know if the results are economically significant - i.e., are they big?
    3. If you have a lot of data, almost everything will be statistically significant, but still may not be economically significant.
    4. If you have little data, little will be statistically significant, but it may still be economically significant.
  2. Econometrics One Tool of Empirical Research
    1. Econometrics is only one tool of econometric research.
    2. More important: economic history. Econometrics may help you understand economic history better, but statistics are not a good substitute for knowing the historical facts.
    3. Knowing history helps you to distinguish correlation and causation.
  3. Data-Mining
    1. A lot of statistical work is produced by "data-miners" who torture the facts until they confess. So whenever you see statistical results, ask yourself:
    2. What data was used?
    3. Does the data actually capture what it is supposed to measure?
    4. How many alternate specifications were tried? Of these, how many were shown?
    5. Does the study confuse causation with correlation?
  4. Economic Theory and Econometrics
    1. Sometimes, you use econometrics to test economic theories.
    2. However, when an economic theory is fundamental, you can reverse this: use the theory to test whether the econometrics works well.
    3. Example: I am convinced by all of my personal experience and historical study that demand curves slope down. If a study claims that the minimum wage increases employment, it just makes me think that econometrics is unreliable. (How come no one checks to see if the demand for asparagus is negatively sloped?)
    4. While some will call this dogmatic, I don't think that it is. Econometrics is itself a sort of theory that merits testing and must be double-checked against other sources of knowledge.