Job/Task Analysis
The foundation of a quality high-stakes
exam
An often overlooked component of a high-stakes examination is a Job/Task Analysis
(JTA). In this step, the job's duties and requirements are analyzed to develop a
framework for exam's content. Without this crucial first step, it
is impossible to establish a case for validity of the exam results.
Typically, the process begins with a discussion among the key stakeholders about
the exam's strategic purpose, expected target populations, and general scope of the
job's duties and requirements. This is followed by an expert panel discussion to
outline the domains, tasks, knowledge and skills required for competent performance
on the job. This outline is used as a blueprint for all exam development activities
to follow. ExamDeveloper provides all of the tools necessary to build this important
description of work online.
The next step in the process is to have a group of practitioners validate the outline
using a survey containing several quantitative scales such as importance, criticality, frequency, difficulty
of acquisition, etc. These results will provide a confirmation that the components
in the JTA outline are relevant to competent performance on the job. Exam Design can facilitate this survey process for you. All that is needed
is the names and email addresses of as many practitioners necessary to complete
the survey, which is typically 10-20% of the target population for the examination.
At the conclusion of the survey, the practitioner ratings are combined together
to create an exam blueprint, which specifies the number of questions needed for each
area of the JTA outline. ExamDeveloper creates this question distribution automatically
using the results of the practitioner survey and the total question count of the
exam.
The final step of a JTA study is the documentation of the entire process in a technical
report. This documentation is important step in guarding against legal challenges
of your exam. This report contains an overview of the stakeholder's strategy
discussion, the process used to develop the JTA outline, the validation data associated
with the JTA outline, and the final exam blueprint. Exam Design's team of experts is
available to assist in preparing and publishing this report to fulfill psychometric
and legal requirements.
If this seems complicated or time-consuming, don't worry. Our
ExamDeveloper software simplifies the process and contains tutorials to
instruct users through every step of the process. In addition, feel free to
contact one of our PhD-level consultants, as they have led many organizations
toward creating a solid foundation for a high-stakes examination.
Next Step
Can a JTA be skipped?
Very often, clients ask about the importance of a Job/Task Analysis in the exam
development process. Faced with time or cost constraints, they often ask if it is
possible to skip the JTA process and proceed directly with the question writing.
A reasonable analogy is to ask: How important is a foundation in the design and
construction of a building? Anyone knowledgable about construction will tell you
that a foundation is the most important component of a building. It would be
foolish to try to construct any building without a solid foundation to support it. In addition,
a faulty foundation is usually the most difficult and costly problem to remedy after
the building is complete.
The ExamDeveloper software was built to guide an organization through the steps
necessary for building a solid JTA as the foundation for their exam.