Creating a New Test
Select Tests from the Main Menu
To create a new Test click on the Add Test button:
The Add new Test window will display, enter a unique Test Name and Description.
Select the Test Type:
|Linear||Test items / questions are delivered in the same order.|
|Adaptive (CAT)||Test adapts to the performance of the Test Candidate.|
|Testlet Based||Similar to CAT, however, items are selected in groups of items (Testlet) rather than by individual Items.|
then click on Add new.
Edit Assembly Screen
Note: there are three tabs at the top of the screen - Assembly, About and Scoring.
- The Assembly tab is where the items for the Test are set up.
- The About tab stores key information about the test (e.g. Test Name, Description, Time Limits and Tags).
- The Scoring tab is where the scoring requirements for the test are setup.
The Assembly Tab
The Assembly tab has a substantial amount of important functionality; its purpose is to allow you to assemble psychometrically sound test forms by finding appropriate items in your item banks.
New Tests will be created with a default : First test section, this can be renamed and edited by clicking on the cog icon.
Enter the Test Name and Description.
Allow examinees to pause the test during this section; this will stop the timer but not stop the delivery. Check this box to allow the examinees to pause their test, the time will stop but the delivery of the test will continue
Randomize item order for each examinee. Check this box to randomize the item order of the test for each examinee.
Delete unused Sections by clicking on the delete icon . Note: This will delete the section as well as all items contained in that section.
To Add Items to the Section click on the plus icon .
Finding and Selecting Items
There are several search tools available to find the items for a test.
Search: type in the name or part of the name of any item.
Filters: Item Banks and Tags
- Item Banks: The default setting is All Items Banks, use the drop-down arrow to select an Item Bank to search (e.g. Math Item Bank or Biology Item Bank).
- Tags: narrow the search by selecting a Tag within the Item Bank previously selected, (e.g. Item Bank = Math and Tag = Algebra).
The list of items that match the filter criteria will be displayed (bottom left). Click on the eye icon to preview the details of an item .
There are 2 ways to add items to the test:
- Click, drag and drop an item from the left panel over to the right panel.
- Select the items by ticking the check boxes next to each item and click on the Add Items button
- Click on the Add All (#) button to add all the items listed on the left to the test.
- The Remove Items button can be used to remove selected items from the test.
To rearrange the sequence of items in the test, drag and drop the items in the right side panel.
The About Tab
The About tab stores key information about the test (e.g. Test Name, Description, Time Limits and Tags).
|Test ID:||Short Name for the Test|
The status of the Test (e.g. Active)
Note: The Status options are: New, Active, Retired)
|Time Limit (in minutes)||Enter the time limit for the test. This time is set in minutes, e.g. 60 minutes. Note: Test Sections can also have a time limit, the sum of the sections cannot exceed the Total Test Time.|
|Description:||Enter a Test Description.|
|Add Tag:||Enter a tag(s) for the test. Tags make searching easier.|
Linear - all test questions in the same order to all candidates
Adaptive - CAT Tests (Computerized Adaptive Tests) - CAT Set up Notes
Testlet Based - MultiStage Tests (MST) - MST Set up Notes
The Scoring Tab
The Scoring Tab is where the scoring specifications for the test are setup.
There are 2 methods for calculating the Cutscore of a test:
- Sum of Points - Calculate the cutscore based on a minimum Sum of Points
- e.g. 39 out of 50 - Examinee must achieve a minimum of 39 out of a possible 50.
- Percentage - Calculate the cutscore based on a minimum Percentage
- e.g. 78% - Examinee must achieve a minimum of 78% on the test.
Scaled Scoring will alter the raw scores into an arbitrary scale, this is for reporting purposes.
Scaled scores are computed using the scaling function (detailed below) for the total number correct scores and/or the domain number-correct scores. Scaled scoring is often used to mask details about the test, such as exact number of items or raw cutoff score, or to express scores on a different scale than number correct. Most large-scale tests use a form of scaled scoring.
- Linear Transformation: The raw scores are first multiplied by the slope coefficient then the intercept is added to the product.
- For example: if you want the scores to be reported on a scale of 100 to 200 for a test of 50 items, the scaled score could be specified as SCALE = RAW × 2 + 100.
- Standardized Transformation: The raw scores are converted to have a mean of X and a standard deviation of Y. This form of scaling is useful to center the mean of the test around a constant value for use in a report.
- For example: the classic IQ scale with mean=100 and SD=15.
Click the Save & Exit button to save and exit the test.