Contents

• Overview
• Set Up a Review Round
• The Feedback Types Panel
• The Feedback Panel
  • Scoring
  • Voting
  • Custom Feedback Form

Overview

This document will walk through the setup of configuring the assessment type for reviewers in Next. 

Set Up a Review Round

Determining and configuring the feedback type is part of the overall configuration process when setting up a Review round.  Please see the articles titled Set Up a Review Round and Creating Review Groups in a Review Round for the first steps of this process.  This article starts after the first and/or second panels of the review round set up where either no review group is selected or the type of review group is selected and has been configured.

The Feedback Types Panel

Depending on the review group selected, either the Feedback panel shows next, or the Feedback Types panel shows next.

The Feedback Types panel will open if either one of the following options for review groups is selected:

• Use a field in the Application as a filter for the groups
• Create customized groups that will review all Applications

In this panel, the feedback type for all groups can be configured simultaneously with one button if they will be the same, or separately for each individual group if they will be different.  

Selecting Set Up for All Groups will configure the feedback process the same way for all the groups.  

To configure the feedback process for each group or option if they will be using different evaluation methods, or if each group or option has their own custom feedback forms, select the Set Up button next to each group or option and select the appropriate feedback settings.

Selecting the Set Up button for any of the above options will open a sub-panel providing choices to the type of feedback that will be given.  These configuration options behave the same as the options on the Feedback panel.  

The Feedback Panel

When Use a field in the application or Customized groups are selected for distributing review assignments, the feedback type for each of the review groups will need to be set up per individual option or group, or all together in the All Groups option.  

That means the configuration settings to set these up are the same as outlined in the sections below, but are contained within each option or grouping on this panel.  The screenshot below shows an example of setting up the feedback type individually per option or grouping.

When using either no review groups or randomized groups for review assignments, a separate Feedback panel will display as shown in the screenshot below.

No matter the type of review group selected for distributing the applications to reviewers to evaluate, the options for feedback types is the same:

1. Scoring
2. Voting 
3. Custom Feedback Forms 

--- REMEMBER ---

When setting up the feedback types for Use a field in the application group option, or Customized groups option, individual groups or options can be set up with each having their own feedback type. Each will have to be saved independently as they are set up.

Scoring

A scoring feedback form allows the set up of individual criteria for evaluation, each having its own scoring rubric.  

Each scoring element can also be weighted, which is especially helpful when evaluating on multiple criteria that are not equally important.

Fields that are not yet fully configured will have yellow triangles with exclamation points in them indicating they need to be set up as shown in the screenshot below.

To set up a Score Field, follow these steps:

1. Select the Scoring option for the type of feedback in the review round.

2. Select the Set Up button next to the Score Field to define.

3. A pop-up window will appear providing access to all the customizable Score Field options.

4. Update the name of the Score Field to an appropriate label:

a. Select the pencil icon.

b. Enter a new name.

c. Select the Return/Enter key on the keyboard or the arrow button on the pop-up window.

5. The score range allows the Program Owner to set a minimum and maximum score for that score field.  

a. Please note that the pre-populated text in the text boxes needs to be replaced.

b. As a best practice, these ranges should be clearly defined in terms of their values.  It is recommended to add definitions of the scoring range in the Include a personal message as part of the invitation email section of the program member email invitations that are sent to reviewers.

6. Assign a weight value to the score field.  (If there is only one score field, enter 100 in the box.)

Weights across all Score Fields must add up to 100%.  As values are entered in the box, notice that the dynamic text below the box updates in real-time.  The percent remaining indicates how much of the weight value is left to be assigned to the remaining Score Fields.

7. Select the Save button when the fields are complete. 

8. Repeat the above steps for each Score Field needed.  
9. Determine if reviewers will have a single text box under all the score fields on the feedback form to record any notes about their decision. 

10. If using an application field or customized groups for reviews, select the Save button at the top of the sub-panel to save the settings before moving on.  If using randomized groups, move to the next step.

11. Select the Next button to move to the next panel to assign reviewers.

12. The next panel assigns reviewers to the round.  See the article titled Assign Reviewers to the Round for those steps.

Voting

A voting feedback form is a simple form that provides reviewers only two options.  This type of feedback form is helpful in situations like an administrative review that merely decides eligibility, or in a final decision round where reviewers are only voting yes or no.

To set up a Voting form, follow these steps:

1. Select the Voting option for the type of feedback in the review round.

2. Fill in the prompt to give more information or instructions to reviewers. If no new text is added, the text shown in the box will be the prompt.

3. Scroll down to edit the voting labels; the default labels for voting will be “Yes” and “No” unless they are edited to other labels.

4. Determine if reviewers will have a single text box under the voting fields on the feedback form to record any notes about their decision.

5. Select the Next button to move to the next panel to assign reviewers.

6. The next panel assigns reviewers to the round.  See the article titled Assign Reviewers to the Round for those steps.

Custom Feedback Form

Custom reviewer feedback forms would have to be set up before configuring the review round.  See the article titled Custom Review Feedback Forms for those instructions.

Once the custom feedback form has been created, it then can be set up in the review round.  

To set up a Custom Feedback Form, follow these steps:

1. Select the Custom Feedback Form option for the type of feedback in the review round.

2. From the dropdown menu, select the desired custom feedback form.

3. Average scoring:

a. If a program will average scores across all reviews, select the corresponding field(s) from the dropdown list; only field types that are allowed for this function will show on the dropdown list if the custom feedback form contains them.  For more information about average scoring, see the article titled Average Scoring (coming soon).  

b. If a program will not utilize the average scoring feature, this field can be ignored.

--- IMPORTANT ---

When fields are selected from the dropdown list, they will display below the list and will disappear from the selectable options.  See the screenshot below for an example.

4. Select the Next button to move to the next panel to assign reviewers.

5. The next panel assigns reviewers to the round.  See the article titled Assign Reviewers to the Round for those steps.

--- IMPORTANT ---

Once a review round has started, the review round settings cannot be reconfigured.  Only new reviewers can be added.