Expected Precision of Estimation and Probability of Ruling Out a Hypothesis Based on a Confidence Interval

Interval estimation is an important objective of most experimental and observational studies. Knowing at the design stage of the study how wide the confidence interval (CI) is expected to be and where its limits are expected to fall can be very informative. Asymptotic distribution of the confidence limits can also be used to answer complex questions of power analysis by computing power as probability that a CI will exclude a given parameter value. The CI‐based approach to power and methods of calculating the expected size and location of asymptotic CIs as a measure of expected precision of estimation are reviewed in the present paper. The theory is illustrated with commonly used estimators, including unadjusted risk differences, odds ratios and rate ratios, as well as more complex estimators based on multivariable linear, logistic and Cox regression models. It is noted that in applications with the non‐linear models, some care must be exercised when selecting the appropriate variance expression. In particular, the well‐known ‘short‐cut’ variance formula for the Cox model can be very inaccurate under unequal allocation of subjects to comparison groups. A more accurate expression is derived analytically and validated in simulations. Applications with ‘exact’ CIs are also considered.