Response Format

Selecting response formats (i.e., the choices test-takers
have to respond to test items) for test items is another crucial
consideration when building a test. The first major response format
consideration typically encountered is whether to require open-ended or
closed-ended responses. The Rorschach inkblot test is an example of a
test that elicits open-ended responses. There are practically no limits
to what an examinee can say when responding to this test. At the other
extreme are tests that require the examinee to choose between a very
limited set of responses, such as true/false, yes/no, or multiple-choice
tests, in which only one of several options is correct.

There are advantages and disadvantages to open-ended response
formats. For instance, open-ended items can provide rich information,
but developing a reliable scoring system is often difficult. Scoring
systems for the Rorschach have developed over many decades, are
difficult to learn, and their utility is controversial. Individually
administered IQ tests, such as the Wechsler tests, contain open-ended
items in which examinees are asked to define vocabulary words. Creating
scoring criteria for each vocabulary word requires collection of typical
responses, expert judgment, and assessment of scoring criteria for
inter-rater reliability and accuracy. Test administrators need to have
extensive training to correctly administer and score this test.

A very common and useful closed-ended response format is Likert
scaling. This format asks for each item to be rated on a multipoint
scale, typically, 1: Strongly disagree, 2: Disagree, 3: Neutral, 4:
Agree, 5: Strongly agree. Variations include more or fewer points on the
scale, different scale labels, rating of frequency vs. intensity, and
inclusion or elimination of a neutral scale midpoint. Another
consideration is whether to include reverse-scored items. For instance,
an anxiety scale might include the item “I am nervous almost all of the
time.” Giving the item a high rating, such as 4 (Agree) would indicate a
higher anxiety level. You might also, however, include the item “I am
usually calm” on your anxiety scale, so that a 4 would actually indicate
a lower anxiety level. Reverse items are adjusted before statistically
analyzing the scale by subtracting scores from one more than the highest
rating on the scale. With a 5-point scale, ratings of a reverse-scored
item would be subtracted from 6, so that 5 becomes 1, 4 become 2, etc.

Other response formats, such as ipsative (also called forced-choice)
items are less common. These items ask the examinee to choose between
options or to rank the desirability of various choices.

To prepare for this Discussion, consider the appropriateness of
various response formats for the test you are proposing for your Final
Project. Select two different response formats to use for this
Discussion.

With these thoughts in mind:

write a description of two different
response formats. Then explain the pros and cons of each for measuring
your Final Project construct of interest. Finally, justify the use of
each format in terms of psychological appropriateness. Support your
response using the Learning Resources and the current literature.

Be sure to support your postings and responses with specific references to the Learning Resources.

Readings

• Kline, T. (2005). Psychological testing: A practical approach to design and evaluation. Thousand Oaks, CA: Sage.
  Kline, T., Psychological testing: A
  practical approach to design and evaluation. Copyright 2005 Sage
  Publications Inc. Books. Used with permission from Sage Publications via
  the Copyright Clearance Center.
  • Chapter 3, “Designing and Scoring Responses”
• Fisher, S. T., Weiss, D. J., & Dawis, R. V. (1968). A comparison of Likert and pair comparisons techniques in multivariate attitude scaling. Educational and Psychological Measurement, 28(1), 81–94.
  Fisher , S. T., Weiss, D. J., & Dawis,
  R. V., A comparison of Likert and pair comparisons techniques in
  multivariate attitude scaling, in Educational and Psychological
  Measurement. Copyright 1968 Sage Publications Inc. Journals. Used with
  permission from Sage Publications, Inc. via the Copyright Clearance
  Center.
• Lissitz, R. W., & Green, S. B. (1975). Effect of the number of scale points on reliability: A Monte Carlo approach. Journal of Applied Psychology, 60(1), 10–13.
  Retrieved from the Walden Library databases.
• McCrae, R. R. (1994). The counterpoint of personality assessment: Self-reports and observer ratings. Assessment, 1(2), 159–172.
  McCrae, R. R., The counterpoint of
  personality assessment: Self-reports and observer ratings, in
  Assessment. Copyright 1994 Sage Publications Inc. Journals. Used with
  permission from Sage Publications, Inc. via the Copyright Clearance
  Center.
• Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428.
  Retrieved from the Walden Library databases.