Creating a New Test
Select Tests from the Main Menu
To create a new Test click on the Add Test button:
The Add new Test window will display, enter a unique Test Name and Description.
Select the Test Type:
|Linear||Test items / questions are delivered in the same order for each examinee.|
|Different items are delivered to each examinee from a pool of items defined by content constraints in the test setup. Each examinee receives the same number of items.|
|Adaptive (CAT)||Test adapts to the performance of the Candidate.|
|Testlet Based (MST)||Similar to CAT, however, items are selected in groups of items (Testlet) rather than by individual Items.|
then click on Add new.
Note: there are three tabs at the top of the screen - Assembly, About and Scoring.
- The Assembly tab is where the items for the Test are set up.
- The About tab stores key information about the test (e.g. Test Name, Description, Time Limits and Tags).
- The Scoring tab is where the scoring requirements for the test are setup.
The Assembly Tab
The Assembly tab has a substantial amount of important functionality; its purpose is to allow you to assemble psychometrically sound test forms by finding appropriate items in your item banks.
New Tests will be created with a default : First test section, this can be renamed and edited by clicking on the cog icon.
Enter the Section Name and Description.
- Check the Shuffle order of items check box to shuffle the order of items in the section.
- Enter the Time Limit for the Section.
- For individual item time limits the Section Check the : * Utilize item time limits check box. This will allow the user to specify a time limit per item (see note below about setting item time limits).
- Check the Display message to examinee that they are entering a timed section check box to enable this examinee delivery message.
- Check the Linear on the fly (LOFT) section check box for the section to be a LOFT delivery.
- Select the Section Template. (See notes on Section Templates: Automated Test Assembly)
* Note: Setting the item time limits is done during the Test Assembly, after adding Items to the section:
- Delete unused Sections by clicking on the delete icon . Note: This will delete the section as well as all items contained in that section.
To Add Items to the Section click on the plus icon .
Finding and Selecting Items
There are several search tools available to find the items for a test.
Search: type in the name or part of the name of any item.
Filters: Item Banks and Tags
- Item Banks: The default setting is All Items Banks, use the drop-down arrow to select an Item Bank to search (e.g. Math Item Bank or Biology Item Bank).
- Tags: narrow the search by selecting a Tag within the Item Bank previously selected, (e.g. Item Bank = Math and Tag = Algebra).
- Click on Apply after making search selections.
The list of items that match the filter criteria will be displayed (bottom left). Click on the eye icon to preview the details of an item .
There are 2 ways to add items to the test:
- Click, drag and drop an item from the left panel over to the right panel.
- Select the items by ticking the check boxes next to each item and click on the Add Items button
- Click on the Add All (#) button to add all the items listed on the left to the test.
- The Remove Items button can be used to remove selected items from the test.
To rearrange the sequence of items in the test, drag and drop the items in the right side panel.
To Scramble Items in a section select Scramble Items:
The Import Items function can be used for a quick setup of a test.
Step 1. Export the Items from Item Bank
a. Go to the Item bank
b. Select the Items for the Test
c. Select Export Items
d. Enter a Name for the Exported Items file and select the File Type = Excel
e. Go to the Notifications and Download the Items Spreadsheet
Step 2. Import the Items to the Test Section
f. Continue adding items to the section or Save & Exit.
The About Tab
The About tab stores key information about the test (e.g. Test Name, Description, Time Limits and Tags).
|Test ID:||Short Name for the Test|
The status of the Test (e.g. Active)
Note: The Status options are: New, Review, Active, Retired
|Time Limit (in minutes)||
Enter the time limit for the test. This time is set in minutes, e.g. 60 minutes. Note: Test Sections can also have a time limit, the sum of the sections cannot exceed the Total Test Time.
|Enable Correct Answer Preview||This option is used by Test Administrators or Test Reviewers. When enabled this will allow the candidate to check their answers during test delivery.|
|Shuffle order of Items||
Select this option to Shuffle to delivery order of the Items.
Linear - all test questions in the same order to all candidates
Linear-on-the-Fly (LOFT) - Setup this type of test using Section Templates see notes: Automated Test Assembly
Adaptive - CAT (Computer Adaptive Test) see notes: Computerized Adaptive Testing (CAT)
Testlet Based - MultiStage Tests (MST) - MST Set up Notes
This option turns on or off the calculator in the toolbar for the test. If you want to have a calculator on specific items, add it as a Feature to that item.
|Description:||Enter a Test Description.|
|Add Tag:||Enter a tag(s) for the test. Tags make searching easier.|
The Scoring Tab
The Scoring Tab is where the scoring specifications for the test are setup.
There are 3 scoring methods available:
- Sum of Points - Score as a Raw total
- Percentage - Score as a percentage
- IRT score (theta) - Item Response Theory Score - used for Adaptive Tests (CAT and Multistage)
There are 3 methods for calculating the Cutscore of a test:
- Disable Pass/Fail Cutscore - No cutscore for the test
- Pass/Fail Cutscore based on overall Score (all sections): Calculate the cutscore based on a minimum Sum of Points:
- e.g. 39 out of 50 - Examinee must achieve a minimum of 39 out of a possible 50.
- The max score indicates the total score of the test.
- or Percentage - Calculate the cutscore based on a minimum Percentage
- e.g. 78% - Examinee must achieve a minimum of 78% on the test.
- Pass/Fail Cutscore based on multiple hurdles (raw score for each section)
- Multiple Hurdle Scoring. Sections have cut scores, if a candidate does not achieve in any one of the sections, regardless of overall score, the candidate will “fail” the test.
Scaled Scoring will alter the raw scores into an arbitrary scale, this is for reporting purposes.
Scaled scores are computed using the scaling function (detailed below) for the total number correct scores and/or the domain number-correct scores. Scaled scoring is often used to mask details about the test, such as exact number of items or raw cutoff score, or to express scores on a different scale than number correct. Most large-scale tests use a form of scaled scoring.
- Linear Transformation: The raw scores are first multiplied by the slope coefficient then the intercept is added to the product.
- For example: if you want the scores to be reported on a scale of 100 to 200 for a test of 50 items, the scaled score could be specified as SCALE = RAW × 2 + 100.
- Standardized Transformation: The raw scores are converted to have a mean of X and a standard deviation of Y. This form of scaling is useful to center the mean of the test around a constant value for use in a report.
- For example: the classic IQ scale with mean=100 and SD=15.
Click the Save & Exit button to save and exit the test.