When it comes to assessing the validity of behavioral event interview questions, we need to understand what we are attempting to do. Some people loosely use the term “validity” to mean “we had the questions reviewed by our legal counsel, or senior management, or our consultant, to determine whether they met standards for selection in the Uniform Guidelines.” This approach is a first step in ensuring validity of a question, but it certainly doesn’t meet the full criteria. In this context, however, there are a few points to check as you begin to assess validity of your questions.
- Does the question ask the candidate about JOB or ROLE SPECIFIC information? If the question isn’t about the job or about a factor that you can show through correlative studies is related to success on the job, you should probably avoid its use.
- Is the behavior, skill or experience related to the successful performance of the job? If you ask candidates about the most innovative idea they ever created at work and the job you are hiring for does not require creativity, your question could be challenged. Same thing goes for asking candidates to complete a test of spelling when spelling or writing is not required in the job.
- Does the question gather information about the personal background, marital or child status, preferences or sexual orientation of the individual? If so, better not ask that one either.
- Does the question ask about hobbies, church groups, sports activities, or other free time activities? If there is no relationship between the job and these categories, then this is a question to avoid as well.
This list could surely be increased, but the rule of thumb for your first screen of questions is to ensure that they are job related and that they focus on the skills, knowledge and experiences that you know are linked with success in the job.
Next, one way to establish validity would be to ask existing people in the job in your company to answer the questions within your interview as if they were being interviewed. Note their responses. How did best performers answer the questions? How did average and below average performers answer the questions? You may wish to use key word analysis to group the responses and find out if these two groups answer the question in the same way or differently. If the whole group tends to answer the question the same way, you need to decide whether the information you are asking for is basic information you have to have of the candidate or if the question in fact is really worth asking.
Over time, you can track how various candidates answered your questions and which questions seemed to be able to identify better potential candidates. It is a good idea to show how you did your analysis and to keep the documentation you used to draw your conclusions. Of course if you can show some statistical correlation between the responses of your candidates, using key word analysis, and the relation to actual performance, then you are on your way to establishing some real validity in your questioning guides.
If you don’t have the time, wherewith-all, or interest in exploring this area, you can always call ECI. We enjoy the challenge of identifying differentiating items within the selection process and tracking validity of selection systems. Happy hiring!
