Alt

Published Articles

Here are abstracts of articles we have published in peer-reviewed scientific journals. Full-text versions of the articles are available through various publishers.

The Measurement and Prediction of Managerial Creativity

Authors: Linda S. Scratchley and A. Ralph Hakstian
Journal: Creativity Research Journal
Date: 2000-2001

A Creative Management Assessment Battery consisting of cognitive tests of Divergent Thinking Ability and a personality scale measuring Openness to Change, Risk and Ambiguity successfully predicted on-the-job creative management performance in a sample of 221 managers from a variety of organizations. For organizations concerned with their management team’s capacity for change management and creative problem solving, impressive improvements in these aspects of the management force are possible through use of the Creative Management Assessment Battery in a personnel-selection context. Choosing the top quarter of management candidates, for example, on the basis of their scores on the Creative Management Assessment Battery could increase by 20 – 24% the proportion of an organization’s managers who are satisfactory-or-better in managerial creativity.

Improving Salesforce Performance: A Meta-Analytic Investigation of the Effectiveness and Utility of Personnel Selection Procedures and Training Interventions

Authors: Seonaid Farrell and A. Ralph Hakstian
Journal: Psychology and Marketing
Date: 2001

The research literature on systematic selection and training was examined using meta-analytic techniques to identify the most effective way to improve salesforce performance. Among the different selection and training practices, the greatest improvements in salesforce performance were achieved by selection with multiple assessments, yielding a net return-on-investment of approximately $43,000 per year for each salesperson hired with multiple assessments. Single assessments for selection were effective, on average, in predicting performance; however, not all types of single assessment were effective. Similarly, training improved salesforce performance on average, although not all types of training were effective and it was demonstrated that training by itself cannot compensate for invalid selection. The most effective way for organizations to improve the performance of their salesforce is to combine selection by multiple assessment with training, which could yield a return-on-investment to the organization of approximately $70,000 per year for each salesperson hired and trained.

Selection of Telemarketing Employees by Standardized Assessment Procedures

Authors: A. Ralph Hakstian, Linda S. Scratchley, Allison A. MacLeod, Roger G. Tweed, and S. Siddarth
Journal: Psychology and Marketing
Date: 1997

A Telemarketing Assessment Battery measuring two cognitive abilities (Perceptual Speed and Accuracy and Divergent Thinking Ability) and three personality traits (Self-Acceptance, Socialization, and Achievement via Conformance) provided solid prediction of future job performance in a sample of 85 applicants for telemarketing positions. Through its positive impact on sales and other aspects of telemarketer job performance, use of the Telemarketing Assessment Battery in employee selection proves to be an important aspect of salesforce management. Choosing the top 25% of telemarketing applicants, for example, on the basis of their scores on the Telemarketing Assessment Battery could increase by 18.5% the proportion of an organization’s telemarketers who are considered good or excellent performers and could lead to a 15% increase in yearly sales for the telemarketing department.

Validity of a Large-Scale Assessment Battery in an Industrial Setting

Authors: A. Ralph Hakstian, Lorette K. Woolsey, and Marsha L. Schroeder
Journal: Educational and Psychological Measurement
Date: 1987

An assessment battery was developed to select personnel for managerial-level positions. The battery consists of ability tests (two measures of general intelligence, reading and writing skill tests, a quantitative ability test, and two measures of innovative thinking), two multi-scale batteries that measure personality/motivational traits, and an In-Basket exercise that assesses administrative skill variables. The battery was designed to provide information about different aspects of management potential, much like an assessment centre does. None of the measures included require subjective scoring and the battery can be group administered. The battery was administered to 238 BC Tel first-level supervisory personnel. The purpose of the study was to develop a set of reliable composite dimensions and to assess their concurrent validity for predicting success in first-level management positions. Six personality dimensions, three ability dimensions, and two administrative skill dimensions were constructed and were found to be significantly related to performance appraisal ratings provided by participants’ supervisors. The results suggest that the measures constructed in this study would contribute significant, worthwhile information to the selection of high-potential management candidates.

Management Selection by Multiple-Domain Assessment: I. Concurrent Validity

Authors: A. Ralph Hakstian, Ross M. Woolley, Lorette K. Woolsey, and Barbara R. Kryger
Journal: Educational and Psychological Measurement
Date: 1991

This study extended the research conducted by Hakstian, Woolsey, and Schroeder (1987). Bio-demographic data and supervisory judgment measures were added to the predictive battery developed in 1987 and a revised administrative skill measure was used. A total of 321 first-level managers employed by the Manitoba Telephone System (MTS) completed the battery. Behaviourally focused performance appraisal ratings provided by senior managers were combined to form 12 appraisal dimensions to measure the management competencies used at MTS. A number of significant relations among ability, personality, and administrative skill composites and the 12 competency dimensions were found. Composites constructed from the bio-demographic data and supervisory judgment data were also related to the competency dimensions. The test battery data were used to construct 12 linear composites that were positively related to the competency dimensions, supporting the concurrent validity and usefulness of the test battery for selecting managerial personnel.

Management Selection by Multiple-Domain Assessment: II. Utility to the Organization

Authors: A. Ralph Hakstian, Ross M. Woolley, Lorette K. Woolsey, and Barbara R. Kryger
Journal: Educational and Psychological Measurement
Date: 1991

This study investigated the utility of adopting the multiple-domain assessment approach used in the MTS study. Utility analysis quantifies the gains an organization would realize from adopting the assessment program. The overall predicted management potential composite was used to estimate the probability that a selected employee would be successful on the job. Three different methods of assessing the utility were applied. The Taylor-Russell approach showed that using the test battery would increase the proportion of successful employees hired, even when the base rate (proportion of individuals who would succeed without using the test battery for selection) was high. The Naylor-Shine approach demonstrated that using the battery would result in higher average employee performance than would be realized by placing employees without using the battery. The Brogden-Cronbach-Gleser utility model was used to quantify the predicted net annual dollar benefit and the productivity gains per employee that would accrue from use of the test battery. The estimated annual dollar benefit ranged from approximately $4,000 to $25,000. This model also forecast that individual productivity would increase by approximately 4 hours per week.

Development and Application of a Quickly-Scored In-Basket Exercise in an Organizational Setting

Authors: A. Ralph Hakstian, Lorette K. Woolsey, and Marsha L. Schroeder
Journal: Educational and Psychological Measurement
Date: 1986

An In-Basket exercise, consisting of 22 items, was developed and scored for two key dimensions: Productivity (amount of material processed), and Content (effectiveness of the actions taken). Productivity was scored as (1) the number of items dealt with and (2) the approximate number of words generated across all items. Ten key items of the 22 were scored for Content according to a 0-1-2 point (per item) scoring scheme, where very effective responses received two points, somewhat effective responses received one point, and ineffective responses received zero points. Both the Productivity and Content scores were found to predict various aspects of on-the-job supervisory performance in a sample of 238 first-level supervisory employees, with the Productivity score slightly outperforming the Content score.

Assessment of In-Basket Performance by Quickly-Scored Methods: Development and Psychometric Evaluation

Authors: A. Ralph Hakstian and Karen P. Harlos
Journal: International Journal of Selection and Assessment
Date: 1993

Building on the In-Basket exercise/scoring system developed by Hakstian, Woolsey and Schroeder (1986), a 21-item In-Basket exercise was scored for three performance dimensions (Productivity, Quality of Judgment, and Understanding of Situation) and five stylistic dimensions: (a) Arranging, Scheduling and Coordinating, (b) Interpersonal Relations, (c) Leadership in a Supervisory Role, (d) Managing Personnel, and (e) Analysis and Synthesis in Decision Making. A listing was made of all the actions taken by a large number of participants in response to each In-Basket item. Panels of senior managers keyed these actions for Quality of Judgment (effective, neutral, or ineffective) and assigned each action to one or more of the five stylistic dimensions. Dimension scores on Quality of Judgment and on the five stylistic dimensions were derived by identifying all of the actions taken by a candidate and summing the dimension scores assigned to those actions. Productivity was scored as the total number of actions taken. Understanding of Situation was assessed using a multiple-choice In-Basket post-test designed to measure understanding of the issues, problems, personal dynamics, etc., present in the In-Basket exercise. To reduce scoring time, the eight In-Basket items whose Quality of Judgment scores predicted on-the-job management performance most effectively were earmarked for scoring, with the other items serving as contextual filler. A measure of Overall In-Basket Performance which was derived by combining all of the dimension scores from the eight scored In-Basket items was found to be an effective predictor of on-the-job management performance in the sample of 321 first-level managers that were used in developing the scoring system, and also in a second, independent sample of 296 first-level managers.

In-Basket Assessment by Fully Objective Methods: Development and Evaluation of a Self-Report System

Authors: A. Ralph Hakstian and Linda S. Scratchley
Journal: Educational and Psychological Measuremen
Date: 1997

In the third generation of In-Basket development work that began with Hakstian, Woolsey and Schroeder (1986), a follow-up booklet was developed to enable the self-reporting of actions. After having completed the In-Basket exercise in the usual way, participants checked off all of the actions that they have undertaken while working on the In-Basket, and then chose a maximum of three alternative actions that they considered to be effective. The increased information yield that resulted from having participants’ scores based upon both what they actually did while working on the In-Basket exercise and some chosen alternatives led to increased reliability of the exercise. In-Basket scores resulting from the self-report and traditional scoring systems were substantially and equally effective in predicting on-the-job management performance in a sample of 258 first- and second-level managers. The results suggest that the high face validity of the In-Basket exercise can be combined with the scoring ease and objectivity of the self-report format. The practical implication of using the self-report-based scoring is that the exercise takes longer to administer, but results in substantially reduced scoring time for the assessor.

The Assessment of Counterproductive Tendencies by Means of the California Psychological Inventory

Authors: A. Ralph Hakstian, Seonaid Farrell & Roger G. Tweed
Journal: International Journal of Selection and Assessment
Date: 2002

An 80-item scale developed from the California Psychological Inventory (CPI) successfully predicted self-reported counterproductive behaviours in a sample of 1,019 university undergraduates, documented counterproductive job behaviours in a sample of 91 armored-car company employees, and supervisory ratings of work conscientiousness in samples of 196 management employees and 79 telephone service representatives. The CPI-Cp scale developed in this study is superior to existing standard CPI scales with respect to predicting counterproductive activity, and has approximately equal validity to existing special-purpose overt and personality-based instruments. The advantage of the CPI-Cp scale over stand-alone measures of counterproductive tendencies lies in its ability to normalize the measurement of counterproductive tendencies, since, with the CPI-Cp, the latter are assessed along with many other mainstream personality dimensions. Given that counterproductive behaviour in the workplace has enormously costly consequences, it is important that organizations be able to add the assessment of counterproductive tendencies as one more component in a comprehensive selection/assessment program.