About handy, handmade and handsome models

Statistical inference deals with natural variation between units and with uncertainty due to sampling, measurement error and random assignment to treatments. Almost always it does so within one model, assumed fixed and valid. Here, however, variation of conclusions due to variation in model choice will be discussed. Heuristics to find appropriate models are seldom taught, and model fit tests and model fit diagnostics are far from optimal. The use of more than one model for the same data is illustrated by examples from Item Response Theory.