Test Review for SME and Admin Reviewers

There are 2 options available for test review:

SME Review: Subject matter experts will be able to review each of the items in a test.  

Admin Review: Administrators will have access to Review and View the comments and Angoff(s) captured by the SME Reviewers.

• Both options are available in the Tests Module

Review the SME Results is done by selecting:

SME Review Results - This review shows the results of the review of items in a given test by SMEs.

NOTE: Only tests that have a Review Status will be available for SME or Admin Review.

Marking a Test for Review

• Go to Tests
• Select the Test
• Navigate to the About Tab
• Select Status and Change to Review
• Select Save and Exit to Save the Review Status

SME Review

• Go to Tests
• Navigate to the SME Review Tab
• Select the test to Review

The test review is delivered in the same format as the test takers format.

Select Start to begin the SME Review

Reviewing the Test Items

Angoff

Enter the Angoff scores for this item and select SAVE

Data Visible to Reviewer:

• Tags - any tags associated with the item
• Custom Fields - any custom field data associated with the item

Comments:

The reviewer can enter a comment for the item.

Click on Next to Review the next Item

Cancel / Save / Export Angoff

Cancel the Review

Save and Exit to the Tests to Review screen

Export Angoff scores - this file will display in the Notifications

example of Exported Angoff scores:

SME Review Results

• Go to Tests
• Select SME Review Results

• Select the Test
• Select the Field(s) to Review.  Fields Available are: Angoff 1, Angoff 2, Comments 

• Click on Display Results

The results will display at the bottom of the screen:

Change the View by selecting : Angoff 1, Angoff 2 or Comments

Admin Review

• Go to Tests
• Navigate to the Admin Review Tab
• Select the test to Review

Select Start to begin the Admin Review

The test review is delivered in the same format as the test takers format.

Flagging Item

Flag Options:

• Keep - Keep Item
• Revise - Re-review Item
• Drop - Remove Item from Test

Select the Flag for the Item and Select Save

Form Level Statistics:

Form Level Statistics are displayed at the bottom of the screen

The answer key will be displayed for each of the tests.

Select Next to go to the Next item

Bulk Edit Based on Flag

Once all the Items have been flagged select Bulk edit based on Flag

Select the Flag e.g. Drop

Select the Action for all the items in the test Flagged as Drop:

Add Tags

• Select the tag for the items flagged as Drop e.g. Status.Retired

Remove Tags

• Select the Tag to remove for the items flagged as Keep e.g. Status.New

Create new test from items with flag

Select the Flag e.g. Keep

Action: Create new test form items with flag

• Enter the “new” Test ID
• Enter the “new” Test Description
• Select the Test Type
• Click on Create

Create new test from items without flag

Select the Flag e.g. Drop

Action: Create new test form items without flag

• Enter the “new” Test ID
• Enter the “new” Test Description
• Select the Test Type
• Click on Create

Add a value to custom field

Select the Flag e.g. Keep

Action: Add a value to custom field

• Select the Custom Field
• Enter the Text for the Custom field
• Click on Edit

Change Status

Select the Flag e.g. Drop

Action: Change Status

• Select the new Status
• Click on Edit

Comments:

Comments entered by other reviewers will be visible

Select Add Comment to add additional comments:

Enter comment and select Add.