2.7 Review Items

As a Reviewer of ExamDeveloper, you may be working on several projects, having been assigned a certain number of items for each project. You can see your work assignments for each project in the Project Review Summary page.

Access the Project Review Summary page

1. Select the Develop Questions → Review menu.

2. The Project Review Summary page displays a count of the items included in the search criteria used to create the Review Assignment.
   
   
   The Project Review Summary page includes the following features:
   
   

   Column	Description
   	Include previously reviewed items by the user. Applicable only when the item is still in the Review state (possible only when more than one Reviewers are required to accept items). Icon This option can be used if a Reviewer decides to make changes to items that he/she has reviewed previously. However, if an item has been modified by a different user since the Reviewer modified the item, it is no longer included in the list.
   	Review the selected items.
   	View the HTML report of the selected group of items. Permission for Enabling the Print Report Button Icon In order for this button to appear on the Review Questions page, you need "Print Report" included with the "Review Questions" permission.
   Criteria	The search criteria used to assign the items for the Review Assignment.
   Remaining	The number of items that have been assigned for review or are remaining to be reviewed per the specific Review Assignment.
   Require Re-Review	This is the number of items that the Reviewer has reviewed once already, but the items have been modified by another user and now require re-review.
   Due Date	The date in mm/dd/yyyy format, by which the assignment must be completed.

   
   
   

   Icon

   The Project Manager decides the number of reviews an item must go through before moving to the Validation stage. Once this number is reached and all Reviewers have accepted the item, it is deleted from the current Reviewer's Reviewing Stack.

   However, if you make changes to an item, other Reviewers see a revised version of the item. This carries a system-generated Revision Code, and it goes back into their Reviewing Stack, in which case, the counts for the “Pending Review” are reset.

Review Items for a project

1. Click the  Edit icon to view the assignment from which the items must be reviewed.
2. In the Review Questions page, the Question Summary section displays the first item on your reviewing list. You can go back and forth between items by clicking the < Previous and Next > buttons, or by using the Jump to drop-down list. You don’t necessarily need to review the item that is displayed before moving on to the next item.

3. From the Review Questions page you can Advance an Item, Modify an Item, or Return an Item.
   Special steps must be taken when reviewing an item that does not have mandatory metadata on the item or reviewing an item with incomplete asset or asset without mandatory metadata.

   Icon

   When multiple reviewers are all viewing the item on the Review Questions page, a message states how many other reviewers are currently viewing the item. As soon as one of the other viewers moves to a different item or navigates away from the Review Questions page, the message is updated in near real-time for all the other users viewing the item. This makes it easier for reviewers to determine if they should work on the selected item, or move on to a different item that is not being reviewed by other users. This messaging does not include any project managers viewing the item from the Examine Questions page.

   

   Icon

   When you are viewing an item on the Review Questions page, if a new revision of the item is created because it was modified by another user, a message notifies you that, "Item 12345 has been updated. Refresh the page to load the latest data."

   

Scenarios for reviewing Items

1. Same Review States for projects:
   Assuming that two different projects have identical Review States. An item from the first project has gone through the initial Review States (not the last Review State) and is waiting for the next Review State. It retains its Review State at the same level when it is made editable in the second project. In a case, the order of the Review States in the two projects is different, the item moves to the pending Review State in the second project. The item retains its accept votes.
2. Same Review States between projects but moved to another project before fulfilling the number of required votes:
   Here also, we assume that there are two projects with identical Review States and an item from the first project is made editable in the second project.
   1. If an item in the first project has received the required number of accept votes per that project’s configuration, then making it editable on the second project still requires it to get one accept vote for its Review State to progress.
   2. If the same item, after getting the required accept votes in the second project, is again made editable in the first project, then it again requires one accept vote in the first project for its Review State to progress.
      
      

3. Different Review Sub-States for projects:
   In this third scenario, we assume that the two projects have dissimilar Review States. If an item from the first project has completed the initial Review States (except for the last Review State), and it is made editable in the second project, then it must go through all the Review States of the second project before it moves to a Validate State. The accept votes that the item has received in the first project are voided.

   Icon

   If this item is checked-out and therefore Locked in another project, you cannot Advance, Return, or Modify it. However, comments can be added to the item.

   
   

   Icon

   In the Jump to drop-down list the number in parentheses is the Question ID.

   The asterisks (*) against one or more Answer Options mark the correct answer(s).

   Icon

   If the Project Manager has enabled users to manually check-out items, the Check-out button appears on the Review Questions page. Reviewers check-out the item for 1 day, 1 week, or 1 month depending on how the project manager configured the setting. See the Review Workflow topic for details on enabling manual item check-out.

   

   See Reviewer Manually Check-out an Item for details on manually checking out items.

   Icon

   If an item is assigned to a case, the Belongs to Case section identifies the name of the Case and the Exhibit text for the case Item, any assets on the case item, and a list of all the items assigned to the case including the item you are reviewing.

   

   Click the  Edit Case icon to open the Edit Case pop-up window as shown below. Here you can modify the Exhibit text (as long as the item is editable in the project and not obsolete). You cannot modify the Case name, or the Exhibit ID. You also cannot add or remove items present on the case.

   

   Icon

   If Restrictions have been applied to Fill in the Blank type items, balance your mouse pointer over the Information icon and a pop-up Restrictions table summarizes the restrictions applied to the Answer Option.

   

   
   
   

Tabs below the Item Stem and Answer Options

1. Question Metadata
   Displays information about the item, including the Blueprint ID and any custom metadata such as Categories and Keywords.
2. User Tracking
   Displayed only for the Project Manager and when the “observer” configuration “Can observers see names of Writers, Reviewers and Validators?” allows it to be displayed for other users. This tab tracks the actions performed on the item by users including date and time of changes.
3. Comments
   Click the Add Comment button to enter your comments for the item. First, choose the type of comment (the Comment Flag), then enter your comment and click Done. Note that the Project Manager sets the available comment flags.
4. History
   Displays earlier revisions of the item. Click on a revision number to view the item in that iteration.

5. Statistics
   Displays the Analysis Group(s) and Analysis Set(s) attached to the item. This tab only appears for those items that have Statistics assigned to the item.

   Icon

   The Analysis Group and Analysis Set drop-down lists are pre-populated according to the selection made on the Statistics tab under Manage → Configuration.

   Icon

   If the Reviewer is assigned with the assignment search criteria "Belongs to case," then that case cannot be deleted. In order to delete such cases, the assignment with Cases for Reviewer must be deleted first by the Project Manager.

   
   
   

View Item with Enemies

1. Click the Question Metadata tab on the Review Question page.
   
2. Click the Membership (Exams, Sections, Enemies, etc.) link at the bottom of the tab.
3. Reviewers can view the Enemy Items that are submitted by other users in the pop-up window, depending upon the observer configuration settings.
   

4. This is irrespective of the users having the View Questions role.

   Icon

   The "Return" button text, "Return" text for button description in the review process explanation paragraph near the top of the page, "Return" text in User Tracking, title text of the Return Question Comment box, additional occurrences for returned Question: State text and Comment title text. For HTML Report obtained after rejecting an item during Review and clicking Print: Status text, "Returned By" text, and Comment title in the Question Report on the Review Questions page may appear using different words, depending on the text provided in the Return Button and Return State Text settings of the Item Bank's configuration.

   In ExamDeveloper versions prior to 1809, these buttons were labeled as Reject and Rejected. For existing projects after 1809, these buttons will retain whatever you named the Custom Label. If you had Reject and Rejected, that will remain as Reject and Rejected.