Statistical Integration Through Metadata Management

Faster and more versatile technology is fuelling user demand for statistical agencies to produce an ever wider range of outputs, and to ensure those outputs are consistent and mutually related to the greatest extent possible. Statistical integration is an approach for enhancing the information content of separate statistical collections conducted by an agency, and is necessary for consistency. It has two aspects-conceptual and physical-the former being a prerequisite for the latter. This paper focuses on methods for achieving statistical integration through better management of metadata. It draws on experiences at the Australian Bureau of Statistics in the development and use of a central repository (the "Information Warehouse") to manage data and metadata. It also makes reference to comparable initiatives at other national statistical agencies.
The main conclusions are as follows. First, a prototyping approach is required in developing new functionality to support statistical integration as it is not clear in advance what tools are needed. Second, metadata from separate collections cannot easily be rationalised until they have been loaded to a central repository and are visible alongside one another so their inconsistencies are evident. Third, to be effective, conceptual integration must be accompanied by physical integration. Fourth, there is great scope for partnerships and exchange of ideas between agencies. Finally, statistical integration must be built into the ongoing collection processes and viewed as a way of life.