Designing effective research and survey instruments takes skill and experience. The scales you choose impact the quality of your data. Rating scales are very common but there are other response options to consider.

Why Consider an Alternative to Rating Scales

The use of rating scales (for example rate the following using a 1-10 rating, with 10 being extremely important and 1 being completely unimportant) is very common in measuring attitudes in market research. One challenge when using rating scales is that when you offer a respondent a series of questions designed to help determine the value of one attribute, capabilities, feature etc. over another a respondent might actually choose 10 or 1 for all the attributes. As a result you now know everything is important or nothing is, which isn’t very actionable. And for the respondent it may be that all of the features/attributes/capabilities truly are important. However, the value of research is to gain insights upon which you can make actionable decisions.

What might be an alternative to the rating scales? Researchers are coming to the conclusion that a viable alternative is to use a rank order scale. A rank order scale gives the respondent a set of items and asks them to put the items in some form of order. Rank order scales can ask respondents to rank brands, product attributes, companies, etc. in terms of preference, importance, effectiveness, and more.

Buy Your Best-Practices Workbook

Research experts have explored the differences if any between using rating scales vs. rank order scales and to see which approach is better able to predict or correlate with behavior. You can test the concept yourself using two matched groups. Have the first group use a 1-5 rating scale to assess product attributes, such as affordable, value, convenient, speed, variety, quality, etc. Have the second group rank each attribute. Then calculate the correlation of each measure with share.

Were your results similar to what the experts are learning? Studies re finding that rankings actually correlate more with behavior than ratings.

Use More Ranking Questions When Designing Your Next Research Instrument

Rankings can help you differentiate table stakes from actual preferences. Products, brands, etc. that are already a part of the customers portfolio have already rated high on the attributes. Buyers and decision makers make discrete choices every day, they make attribute and product trade- offs regularly. These may be difficult decisions to make but some brand or attribute has to win each time they make a decision.

The rankings approach actually allows you to tap into this process because this approach requires the respondents to explicitly and repeatedly examine the products and attributes of the products they use and then select the best one.

If you decide to create your own research and survey instruments see these tips. Prefer to leverage experience and expertise? We’d love to talk with you.

FAQ:

Q1: Why might rating scales be insufficient for actionable research?
A: Rating scales (e.g., 1–10 importance ratings) often lead respondents to rate all attributes equally high or low, making it difficult to discern true priorities or preferences, thereby limiting the actionability of the data.

Q2: What is a rank order scale and how does it differ from rating scales?
A: Rank order scales require respondents to order a set of items (brands, attributes, features) by preference, importance, or effectiveness. Unlike ratings, rankings force trade-offs, revealing relative priorities and clearer distinctions.

Q3: How do rank order scales correlate with actual behavior compared to rating scales?
A: Research indicates that rankings tend to correlate more closely with real-world behavior and choices, providing more predictive and reliable insights for decision-making.

Q4: When should I use rank order scales in survey design?
A: Use ranking questions when you need to differentiate between essential “table stakes” attributes and true customer preferences, especially when respondents must make trade-offs among competing options.

Q5: What are best practices for incorporating ranking questions into surveys?
A:

Limit the number of items to avoid respondent fatigue.
Clearly instruct respondents on ranking procedures.
Combine rankings with other question types to enrich insights.
Pretest your instrument to ensure clarity and reliability.

Q6: Can I test the effectiveness of rating versus ranking scales?
A: Yes, by conducting parallel surveys with matched groups—one using rating scales and the other ranking scales—you can compare correlations with actual customer behavior or market share.

Q7: Where can I get expert help to design research instruments that yield actionable insights?
A: VisionEdge Marketing offers expertise in survey design, including scale selection and question formulation, to maximize the quality and utility of your market research.