Standard Setting
An improperly developed passing score will invite legal
scrutiny
Recall the school days when teachers set the universal "70%" as the minimum score
needed in order to pass the examination. Although that may be sufficient for a high
school science class, using this cut score without sound methodology is the easiest
way to invite legal challenges to your examination program.
Typically, the two most common areas of legal vulnerability for a high-stakes exam
is (1) inadequate job analysis methodology or (2) improperly established performance
standards. There are several important factors to consider when setting the passing
standard:
1. What is the difficulty of the examination? Experts developing a high-stakes
exam usually write questions with a wide range of difficulty, from very easy to
very difficult. As such, it is advisable to document a quantitative evaluation of
the exam difficulty, even if it is based on opinion from a panel of experts.
2. What is the minimum level of performance that can be considered "competent"?
Observations of employees on the job will typically yield a range of performance,
from exceptional (the superstars) to inadequate (the incompetent). The goal is to
establish the minimum set of knowledge and skills someone would need to possess
in
order to be labeled "competent" on the job.
3. What is the experience level of the target population with the material being
tested? Another consideration is the experience with the exam content of
those who will be taking the examination. If the exam contains concepts recently
identified as important due to a change in job role or expectations, the target
population may not have had the experience to effectively master it yet. As a result,
the passing rate may be unreasonably high
until proper training has occurred. Therefore,
it is important to be aware of any new knowledge or skills being tested that would
limit the performance of target population due to experience or exposure.
4. What is the market demand for practitioners in the organization, industry
or public?
A passing standard set too high will produce too few practitioners
to adequately meet the needs of the organization, industry, or public. If too few people are meeting the performance standard
to meet the objectives, it may be advisable to revisit the appropriateness of the
passing standard.
Our team of consultants is very experienced in facilitating standard setting workshops
in a variety of industries including healthcare, information technology, social
services, retail, manufacturing, pharmaceutical, biotechnology, and exercise science.
Next Step
What is a typical passing score for an exam?
For high-stakes exams, there is no such thing as a "normal" or "typical" passing
score
for an exam. As described, many factors affect a passing standard for an exam.
For example, programs with very easy exams or a need to select only the very best
individuals may have a passing score requiring more than 85% of the questions to
be correctly answered. Other programs with very difficult exams or the need to select
many individuals may set the passing standard at fewer than 50% of the questions.
As such, it is impossible to recommend an acceptable score (or even a reasonable
range) without considering all of the factors involved.