The construction and initial validation of a measure of expatriate job performance

Based on Campbell's (1990, Handbook of Industrial and Organizational Psychology, 2nd ed., Palo Alto, CA: Consulting Psychologists Press, 687–732) multi-factorial model of job performance, the Expatriate Performance Scales were developed to measure components of expatriate performance. Item generation for the scales was informed by job performance theory, content analysis of interviews with expatriates and item sorting by subject matter experts. The scales (48 items) were administered to 106 Australian expatriates in the Special Administrative Regions of China. Data were subjected to exploratory factor analysis, reliability analysis as well as tests for criterion, convergent and discriminant validity. These procedures resulted in 32 items measuring an amended model of expatriate performance with six components: task performance, communication performance, demonstrating effort, maintaining personal discipline, team and leadership performance and management and administration performance. Results provide initial psychometric evidence of criterion, convergent and discriminant validity.     

Initial assessment and validation of an instrument to measure student perceptions of leadership skills

This article reports on development and validation of an instrument for use in measuring student perceptions of leadership skills. The article describes the development process of an instrument through use of subject matter experts for measuring leadership skill perception among students and use of an exploratory factor analysis technique to validate the resulting instrument. These data were obtained from a survey of 214 students in a small liberal arts college of business. The article presents an instrument for assistance with measuring leadership skill perception, development, and importance as well as further reports on the initial validity and reliability measures resulting from the analysis. In addition, the usefulness of the instrument for educators and workplace trainers is explored.     

Face validity in personality tests: psychometric instruments and projective techniques in comparison

Face validity is a controversial kind of test validity. Personality tests are divided into two big categories: projective techniques and psychometric instruments. They differ also for face validity, which influences the perception that people have of tests themselves. The article reports the scientific debate on face validity, and the results of a study carried out on naive subjects in order to let them compare projective techniques and psychometric instruments on the mere basis of their surface. An ad hoc questionnaire was administered. It asked subjects to compare projective techniques and psychometric instruments by using 13 adjectives. The sample, accidental, is composed of 238 participants, 45 males and 193 females. The data were substantially analyzed through techniques of Correspondence Analysis. Personality tests are principally judged through two dimensions: the aesthetic and the efficacy. The first dimension characterizes in particular projective techniques; the second, psychometric instruments. Although participants acknowledge that psychometric instruments are credible and scientific, there is a clear preference for projective techniques, principally by females, people younger than 22 and participants with lower education. Personality tests have an appearance that is judged by those who look at them. The aesthetic seems to prevail on the efficacy perception, but it would be suitable to carry on the same research with a sample stratified in respect of the personal details measured by the questionnaire.     

Leadership competency inventory: A systematic process of developing and validating a leadership competency scale

This article reports psychometric evaluation of the Penn State Leadership Competency Inventory (LCI). The 32‐item LCI was validated on a sample of 323 managers in the health care industry. Preliminary validity and reliability evidence of the LCI was established through exploratory factor analysis (EFA), item‐total correlations, Cronbach's alpha coefficients, and confirmatory factor analysis (CFA). The four‐factor leadership competency scale, comprising supervisory and managerial competencies, organizational leadership, personal mastery, and resource leadership, accounted for 58% of variance. According to CFA results, the model fit of the four latent factors of the LCI was confirmed to be appropriate. Cross‐validation with other populations is needed to confirm the factor structure. Limitations and further research recommendations are discussed.     

Acceptability of energy sources using an integration of the Delphi method and the analytic hierarchy process

The research presented in this paper is structured on the basis of the extensive literature on the world energy shift and, in particular, the dynamics of the political, social, market, and community acceptability of different energy sources. This study attempts to determine which sources of energy are most accepted or preferred by political systems, public opinion, and the market and the weights that these dynamics of acceptability have on the global energy shift. In this theoretical framework, we present an application and propose a methodology resulting from the integration of two methods: the Delphi method and the analytic hierarchy process (AHP). While Delphi seeks a convergence of opinions among a group of experts, the AHP helps solve complex decision problems using a hierarchical structure. To integrate the Delphi and AHP methods, we propose an innovation in the way the feedbacks are circulated among the participants. The experts’ judgements are given in a box-plot using a slider, which simplifies the procedure because the respondents have all the most important information on the distribution of responses from the previous round. At the same time, our method eliminates the problem of the choice of one of the diverse methods of aggregation, which are necessary when the AHP is applied in a group setting. The application of this method to energy issues and the dynamics of the acceptability of the different sources in particular revealed the feasibility and advantages of the instrument and produced interesting results.     

Development and application of the Leadership Archetype Questionnaire

The leadership assessment questionnaire (LAQ) is a 360 degree survey instrument designed to help organizational leaders identify their own style of leadership and formulate appropriate development objectives. It is designed to provide avenues for development of an executive team in which multiple leadership archetypes are represented. The LAQ is based on eight leadership archetypes – strategist, change-catalyst, transactor, builder, innovator, processor, coach, and communicator. These archetypes are representations of ways of leading in a complex organizational environment. In this article, we discuss the development, design, and psychometric analysis of the LAQ. We detail the conceptual foundations of the questionnaire and the psychometric methods used to confirm the validity and reliability of the instrument. We conclude with avenues for future research.     

EMPLOYEE PARTICIPATION IN POLLUTION REDUCTION: A SOCIO-TECHNICAL PERSPECTIVE

Is industrial pollution reduction solely a matter of applying technical solutions to technical problems, or do organizational strategies and human factors also play an important part? In 1991 the US Environmental Protection Agency began collecting data on organizational strategies in pollution control as part of its Toxic Release Inventory database. An exploratory analysis of the 1991–1992 Toxic Release Inventory data revealed evidence that internally based, socially driven organizational strategies such as employee participation can have a dramatic impact on pollution reduction. The results are reported here of the second phase of this research: the approach is broadened to define general ‘internal’ versus ‘external’ strategies, and ‘social’ versus ‘technical’ approaches. There is strong evidence that the combination of internal and external strategies is superior to either alone; there is even stronger evidence that a socio-technical approach is better than either alone, and still stronger support for a combination of all of these criteria.     

How to examine construct validity of item banks

In this study data is brought to support an approach for assuring the content validity of item banks and for examining their construct validity. The method is an application of Guttman's concept of using mapping sentences for theory construction in the domain of “simple fraction’. The construct validity of the theory used for the creation of the item bank was tested by selecting two samples of items which are referenced in the same structuple (criterion). The Yule weak monotonicity coefficients of the items' connections were calculated and represented in two different matrices and in two different space diagrams. The results show clearly a similarity in both the structure of the space diagrams of any two parallel samples and in the partition of the points into regions, i.e., regions according to the classification of the items represented by them on the basis of the facets defined by the mapping sentences. Consequently, it may be concluded that the results support the construct validity of the theory on the basis of which the bank has been created.     

Methodological criteria for the internal validity and utility of practice oriented research

For conducting practice oriented research different research strategies are available. In this paper a typology is developed for differentiating between practice oriented research strategies that are data based and participatory research strategies. The former category involves quantitative and qualitative data based research strategies. The latter category, participatory research strategies, includes knowledge based research and practice based research. At present, methodological criteria for assessing the quality of practice oriented research heavily rely on those developed for theory oriented research, in particular internal and external validity and reliability. However, we argue that for assessing the results of practice oriented research other criteria are necessary. In this paper, methodological criteria are formulated for evaluating the internal validity and practical utility of practice oriented research with the help of a Delphi study using research methodologists as experts. They agree upon the criteria of verifiability, comprehensibility and acceptance of the results, as well as holism. Moreover, different categories of participatory and data based research strategies are compared to these criteria. Practice based research and qualitative data based research are best equipped to fulfill these criteria. These findings may enable researchers to make a more deliberate choice for a specific research strategy in practice oriented research.     

Development and Psychometric Analysis of an Instrument to Measure the Reasons that Oncological Patients give when Participating in a Psychological Intervention

This work presents the psychometric study of a questionnaire on participation in psychological therapy (QPPT). The study is applied to oncological patients and the questionnaire was designed to evaluate the reasons given by chronic patients when participating or not in programmes of psychological treatment. The questionnaire includes six rational areas which resulted from the confirmatory factorial analysis.The internal consistency of the scales was analysed through Cronbach’s Alpha, and their discriminatory capacity between criterion groups of oncological patients was tested by means of t-tests and discriminant analyses. The results support, to a reasonable extent, the reliability and validity of the QPPT as an instrument to evaluate the reasons of participation or refusal of psychological treatment in oncological patients.     

What's in a broken promissory obligation? Developing and testing a multiple component form measure of psychological contract breach

While the literature has suggested the possibility of breach being composed of multiple facets, no previous study has investigated this possibility empirically. This study examined the factor structure of typical component forms in order to develop a multiple component form measure of breach. Two studies were conducted. In study 1 (N = 420) multi-item measures based on causal indicators representing promissory obligations were developed for the five potential component forms (delay, magnitude, type/form, inequity and reciprocal imbalance). Exploratory factor analysis showed that the five components loaded onto one higher order factor, namely psychological contract breach suggesting that breach is composed of different aspects rather than types of breach. Confirmatory factor analysis provided further evidence for the proposed model. In addition, the model achieved high construct reliability and showed good construct, convergent, discriminant and predictive validity. Study 2 data (N = 189), used to validate study 1 results, compared the multiple-component measure with an established multiple item measure of breach (rather than a single item as in study 1) and also tested for discriminant validity with an established multiple item measure of violation. Findings replicated those in study 1. The findings have important implications for considering alternative, more comprehensive and elaborate ways of assessing breach.     

The development of the Personality Audit: a psychodynamic multiple feedback assessment instrument

The Personality Audit (PA) was developed to meet a need for a relatively simple multiple feedback instrument that could clarify the various motivational needs of executives. Using a psychodynamic approach to leadership, the PA allows the test-taker to assess him- or herself in seven personality dimensions important in human behaviour and to identify personal ‘blind spots’. The resulting insights can be used to formulate appropriate leadership development goals.

The objective of this paper is to describe the design and psychometric properties of the PA. This instrument, in contrast with other tools that can be used to clarify the inner theatre of individuals, is designed not only to report information given by the test-taker but also to reflect the perceptions of observers representing both the test-taker's public and private spheres. This paper describes in detail the conceptual foundations of the questionnaire, the psychometric methods used to confirm its validity and reliability, and possible directions for future research.     

Identifying Optimal Items in Quality of Life Assessment

Quality of life has drawn increasing attention in health science and more efforts have been devoted to develop instruments that are valid and reliable to assess quality of life. The World Health Organization (WHO) has devised an instrument World Health Organization Quality of Life Survey- Abbreviated Version (WHOQOL-BREF) to assess quality of life, but the psychometric property of each individual item has not been studied yet. Using item response theory, we examine the properties of the WHOQOL-BRIEF Taiwan version. Samejima’s graded response model was fitted, and item parameters were calculated. The items were ranked base on their discrimination power, and the best items are identified. Several data with subset of items (22, 20, 18, 16 and 14 items) were created by omitting items with lower discrimination power. The test information function of the full questionnaire and the subsets were compared. The results showed there were significant positive correlations between the full questionnaire and the subsets of items and the distributions are similar. The test information function showed the maximum amount of test information spaced over two ends of the theta continuum, and this suggested that the WHOQOL-BREF provided more information for groups with either lower or higher satisfaction of quality of life, while it is less discriminating for individuals in the middle range.     

In search of content validity: facet analysis as a qualitative method to improve questionnaire design

In this paper, two scales are examined for the measurement of self-regulation. The item content is evaluated for its specific usefulness for measuring self-regulation of obesity patients. Qualitative facet analysis (QFA), an application of Guttmann’s Facet Theory, is used to investigate whether item content sufficiently covers the intended content area. This form of content analysis constitutes a systematic, effective and critical tool for improving the content validity of questionnaires. It discovers topics that are underrepresented or overrepresented. The method offers a way to clarify the relationship between concrete item phrases and the desired content as defined by construct and theory. It allows the researcher to improve his or her understanding of the items as a stimulus and of the similarities of items, and allows the formulation of a possible structure for confirmative factor analysis. Various ways to improve the content of both scales are identified.     

The Employee Engagement Scale: Initial Evidence for Construct Validity and Implications for Theory and Practice

Interest in the employee engagement construct has gained increasing attention in recent years. Measurement tools focused on nuanced areas of engagement (i.e., job engagement and organizational engagement) have been offered; however, no measure of employee engagement has been advanced despite persistent calls in the research. We present the development, method, and results of a three‐dimensional employee engagement measurement tool developed for use in the human resource and management fields of study. Across four independent studies, the employee engagement scale (EES) was found to consist of three subfactors (cognitive, emotional, and behavioral) and a higher‐order factor (employee engagement). Across a series of four studies, we explored the factor structure and reliability of the EES (Study 1), then refined the scale, confirmed the factor structure, and examined reliability and both convergent and nomological validity evidence (Study 2). Next (Study 3), we completed a final reduction in scale items and examined additional evidence of reliability and nomological validity as well as evidence of discriminant validity. Finally (Study 4), we tested for evidence of incremental validity. In the implications for theory and practice section, we discuss the importance of an employee engagement measure aligned alongside an agreed‐upon definition and framework. Limitations and future directions for research—such as the need for further psychometric testing and exploring issues of measurement invariance—are discussed. © 2016 Wiley Periodicals, Inc.     

Item analysis of university-wide multiple choice objective examinations: the experience of a Nigerian private university

Teachers and Students worldwide often dance to the tune of tests and examinations. Assessments are powerful tools for catalyzing the achievement of educational goals, especially if done rightly. One of the tools for ‘doing it rightly’ is item analysis. The core objectives for this study, therefore, were: ascertaining the item difficulty and distractive indices of the university wide courses. A range of 112–1956 undergraduate students participated in this study. With the use of secondary data, the ex-post facto design was adopted for this project. In virtually all cases, majority of the items (ranging between 65% and 97% of the 70 items fielded in each course) did not meet psychometric standard in terms of difficulty and distractive indices and consequently needed to be moderated or deleted. Considering the importance of these courses, the need to apply item analyses when developing these tests was emphasized.     

Validation of an instrument to measure absorptive capacity

Absorptive capacity is an ability firms should develop if they wish to adapt to changes in an increasingly competitive and changing environment and to achieve and sustain competitive advantage. Despite the increase in literature on absorptive capacity, some ambiguity remains in determining the dimensions that shape the construct. Thus, no measurement instrument can be adapted to these dimensions. The aim of this paper is to contribute to the literature on absorptive capacity by using a resource-based view to present an alternative measurement instrument for absorptive capacity. This instrument differentiates between the phases of acquisition, assimilation, transformation and exploitation of knowledge, as well as between the two dimensions of absorptive capacity (potential and realized), to reduce the problem of measuring and identifying the dimensions that shape this important construct. The instrument’s validity and reliability are guaranteed and have been tested using data from 168 Spanish organizations.     

How Competitive Action Mediates the Resource Slack–Performance Relationship: A Meta‐Analytic Approach

The fungibility of organizational slack provides firms significant latitude in addressing both internal and market pressures. A vast literature suggests that slack influences firm performance; however, the empirical record is mixed, and the underlying mechanism linking slack to performance remains ambiguous. We address these issues by theoretically expanding the slack–performance model to include mediation. Specifically, we develop and test a model in which a firm’s competitive behaviours direct the utilization of slack toward the realization of firm performance. Our meta‐analytic‐based structural equation model supports partial mediation, showing that competitive behaviours provide some resolution to the conflicted understanding of how slack affects performance. Further, we provide value to the slack literature by consolidating the evidence for the effects of various types and forms of organizational slack. Beyond providing robustness to our theoretical model, doing so offers a more complete understanding of how operationalizations of slack and performance outcomes matter.     

Improving the wisdom of crowds with analysis of variance of predictions of related outcomes

Decision-makers often collect and aggregate experts’ point predictions about continuous outcomes, such as stock returns or product sales. In this article, we model experts as Bayesian agents and show that means, including the (weighted) arithmetic mean, trimmed means, median, geometric mean, and essentially all other measures of central tendency, do not use all information in the predictions. Intuitively, they assume idiosyncratic differences to arise from error instead of private information and hence do not update the prior with all available information. Updating means in terms of unused information improves their expected accuracy but depends on the experts’ prior and information structure that cannot be estimated based on a single prediction per expert. In many applications, however, experts consider multiple stocks, products, or other related items at the same time. For such contexts, we introduce ANOVA updating – an unsupervised technique that updates means based on experts’ predictions of multiple outcomes from a common population. The technique is illustrated on several real-world datasets.     

An improved subordinate appraisal of bank manager's competence

The three-factor, forty-five item instrument measuring manager's competence in retail banking returns a poor comparative fit index when modelled as an orthogonal or oblique model. Revisiting the original scale development study published in 1996 with a new data set and following a revised research design yields a considerably improved model. A second-order factor, overall managerial competence, is demonstrated to capture a meaning common to the five first-order factors, namely, entrepreneurial style , interpersonal skills , intellectual capacity , participative management style and emotional maturity . The revised model with substantially improved construct validity and reconfirmed instrument reliability can now be used with more confidence by human resource managers. The revised instrument can become a diagnostic tool in identifying areas of incompetence in management, as well as an instrument of feedback to the manager from subordinates. Allowing subordinates to voice their opinions about their immediate supervisor can become a welcome addition to the practice of participative management. The emergence of interpersonal skills and participative management style as the key factors explaining overall managerial competence indicates the importance of assessing managerial candidates on these factors.