Evaluation Techniques (Part I): Overview

Posted on April 2nd, 2014

iStock_000015274873SmallIt is my belief that one of the most important roles of an HR professional is to continuously proselytize the principles of job-relatedness and consistency. These two principles are absolutely critical to the implementation of talent selection best practices and lay the foundation for all HR processes, including performance management, leadership development, and succession planning.

In order to understand this perspective, we first have to define these principles to ensure we have a common framework for discussion. When people talk about job-relatedness they are referring to the concept of “validation”. Validation is essentially a research process where one collects evidence to demonstrate that performance on an assessment tool is related to performance on the job. According to legal and professional guidelines, validation is required for any tool or test that is used in making employment-related decisions. (As an aside, these same guidelines interpret the concept of “test” perhaps more broadly than most assume. For example, minimum qualifications that are frequently published in job postings would be considered a test (because recruiters use them to screen applicants out of the hiring process) and thus require evidence of job-relatedness.)


There are several different types of validation strategies but the two that are most commonly used are criterion and content validation. With the former, researchers collect test and performance data (e.g., from current incumbents) in an effort to demonstrate that higher test scores predict higher performance on the job. In effect, the data allow you to explore a statistical relationship between the test and job performance. With a content approach, researchers are asking job experts to determine the degree to which the test overlaps with critical pieces of the job. Essentially they are establishing that the test measures key aspects of the target position.

Consistency, on the other hand, refers to the extent to which standardization and structure have been built into the talent selection process. Consistency covers things like defining assessment tools in advance, treating all candidates in the same way, and requiring that assessment results be quantifiable and that those scores be justified and documented.

Consistency requires planning, organization, and, particularly for large companies, systems support. Evaluation criteria have to be identified (for more information about this topic check out my March 2014 blog post), assessments have to be developed and validated, scores have to be collected and data-based, and managers have to be trained.

Hiring managers often complain that consistency takes away their flexibility. They would like to be able to ask any question of any candidate. Having provided litigation support to companies involved in employment-related class action lawsuits, it is often the case that such unbounded flexibility and autonomy become the source of a company’s vulnerability. I consider the hiring process as a path in which HR allows a type of controlled meandering but that has sufficient structure to help identify the best candidates as well as protect the company and its stakeholders.

quoteIn discussing these principles with Erica Spencer, PhD, the Director of Talent Management Operations at Novo Nordisk, she revealed that her company makes great efforts to design and implement talent selection processes which are standardized and job-related. “If it is determined that a selection tool is needed,” Dr. Spencer says, “Talent Management then works with the line of business to further define the business need, identify the appropriate tool, conduct the necessary validation work and then implement the tool in a standardized manner.”

As a final note, I consider job-relatedness and consistency as interdependent such that a talent selection process could not be considered best practice if it followed one principle but not the other.  In other words, job-related processes which are executed inconsistently will struggle to identify the appropriate candidates and expose the company to litigation – so too will a consistently implemented process that lacks validation evidence.

In next month’s blog, I will do a deep dive into evaluation techniques to cover qualifications review, testing and in-person assessment approaches. As always, I encourage you to leave your comments and questions in the text box below and to participate in a 3-minute anonymous survey both of which will help drive discussion and inform subsequent blog posts.

To discuss your specific talent selection issues and challenges, please contact me at 203-817-7522.

Leave a Reply

Connect with us: