MultiStage Testing is a dynamic approach to assessment that blends aspects of adaptive testing and traditional (linear) testing. Instead of adapting the difficulty of the test after every item, we pre-define blocks of items (called Testlets, Panels, or Modules), and adapt after each Testlet.
Consider the following example, which has 3 stages, using a 1-3-3 design.
- Stage 1: A vocabulary test of 15 items
- Stage 2: A nonfiction reading passage with 5 associated items (3 possible passages: Low, Medium, High difficulty)
- Stage 3: A fiction reading passage with 5 associated items (3 possible passages: Low, Medium, High difficulty)
The purpose of MST, in the example above, is to deliver an appropriate level reading passage to each examinee.
i.e. If a "high" difficulty level passage is delivered to an entry level student or vice versa, the student will be lost.
The traditional approach of giving all examinees a middle difficulty passage is not ideal either, since it does a disservice to both the top and lower level students.
How to implement multistage testing in Assess.ai
Assess.ai deals with the complex construction by treating each Stage as a Test, and each possible Testlet / Panel / Module as a Section. The second (and, if needed, third) stages must have a Test that is specifically "Testlet-based." The initial test, which is called a Routing Test in the literature, is a standard Linear Test in Assess.ai.
Here are the steps to implement a MultiStage Test.
Creating a Test
Create a new test, and set the Test Type to be "Testlet-based."
You can do this on the initial Test creation screen, or edit the About screen of the test as seen below.
Scoring:
All Tests in a MultiStage Test must be scored with Item Response Theory (IRT) using the same parameters.
To edit the Scoring Properties of a test:
- Select the Test
- On the Edit Screen Select Scoring
- Scoring method = IRT score (theta)
- Select Maximum Likelihood or Bayes Modal
2. Create a Section for each of the intended Testlets.
Note: To edit the name / description of a Section click on the Cog Icon (It helps to name the sections appropriately).
As the Sections (Stages) are added to a MultiStage test the Infographic will populate with Information about the items in each Section (Stage). (Future Release)
- Stage 2 Easy
- Stage 2 Medium
- Stage 2 Hard
Creating a Multi-Stage Session
3. Create a Session that is a Multi Stage Session Type.
3a. Include the necessary tests. An example is shown below.
Note: Tests must be locked before being added to a Session
The Stage 2 and Stage 3 tests have both been constructed as Testlet-Based and have three sections. (Future Release)
Stage 2: Stage 3:
Multi-Stage (this is an additional step in the workflow for a Multi-Stage Session)
Option 1 - IRT Scoring - i.e. The results of the previous Section/Testlet will determine the next Section/Testlet.
Option 2 - Set the branching rules for the next Section / Testlet.
In the above example Stage 1 to 2:
In the example above Stage 2 - 3
Notes:
During Multi Stage Test Delivery the candidate will be able to skip a question, however will only be able to submit once all questions are answered.
Unscored Items can be included in Multistage Tests.
If you have questions about this process, please contact our Support team.
Comments
0 comments
Article is closed for comments.