Podcast: Play in new window | Download (Duration: 8:27 — 4.7MB) | Embed

Subscribe: RSS | More

Well-conducted primary research can yield valuable actionable insights to guide strategic growth decisions. The quality of your insights often depends on the quality of your research, especially the research questions. Rating scales, especially Likert Scales, are common in research instruments. Composing these properly is the focus of this episode of What’s Your Edge? 

Conducting primary research has been a fundamental part of my career since the late 70’s and key part of VisionEdge Marketing’s work since we began in 1999. Effective research starts with identifying survey objectives tied to business decisions. This might include research to define and validate new products that are market worthy, identify ways to improve customer experience to increase share of wallet and/or referral rates. Ideally, when the survey instrument is fielded and the responses are analyzed, the data gleaned from the answers provide actionable insights. 

Buy Your Best-Practices Workbook

How Great Questions are Like Great Melodies 

In addition to designing and fielding the studies, our customers rely on our expertise in survey and discussion guide question development. How questions are composed is extremely important to the quality and usability of the responses. Just as a melody is critical to music. A melody, the main idea on a track, is a sequence of individual, consecutive notes or musical pitches that when combined in a particular order, create pleasing sounds. The same is true for questions used in primary research.

Songwriters and composers use melodies in their music to tell stories. In the business world, we organize insights to tell an actionable story from the research we conduct. There are some common characteristics between composing good melodies and crafting good research questions, especially ranking and rating questions. Rating scales help us quantify an intangible or abstract concept. 

Make Sure Your Rating Scales Produce a Good Melody 

To make sure we’re singing from the same sheet of music, let’s define our terms, beginning with what a rating scale is.

Rating scales, such as the Likert Scale, are a common type of survey question that uses closed questions to gain information from a respondent. Each question offers a range of choices that are constrained to a single value. Respondents choose a response from a series of statements that best reflects their position, opinions and/or feelings. For example, should you create a question that asks how likely a prospect or existing customer is to purchase a particular song, and the available responses offer a range of choices from extremely likely to extremely unlikely, you are using a rating scale. Rating scales are often used to gain actionable insight into choices around satisfaction level, degree of agreement, or recommendation likelihood.

Creating a research instrument and crafting rating scales share some common elements with music, especially melodies. There are four primary elements of melody that are relevant to research instruments.

The 4 Powerful Survey Elements 

1. Intervals. By definition a melody uses more than one note, so there’ll always be at least one melodic interval. The same holds true for research instruments. There is typically more than one question related to a topic. This allows researchers to capture nuances and ensure validity and reliability.
2. Contour. The shape and sequence of movements. For example, in an ascending contour, the notes in a melody increase in pitch. In terms of research, this would be the sequence of your questions. 
3. Scale. Most melodies are formed from scales, which are like a bank of the specific notes used to create a melody. Results from rating scales reflect an individual’s assessment of value. When results are collected the answers form a comparative dataset to examine trends.
4. Range. The distance between the highest and lowest note of the melody. For rating scales, the most common number of choices on the scale range are 1-5, 1-7, and 1-10, with the highest of the numbers typically reflecting a very strong preference, such as strongly agree, and the lowest number the opposite, such as strongly disagree. We’ll talk more about the range in a bit.

Just like musical notes which are always read from left to right, rating scales run left to right on a horizontal axis. 1’ always represents the lowest scale end. As a participant moves to the right of the scale, the numbers increase. Let’s delve deeper into range, one of the most important elements for rating scales. 

Why Extremeness Aversion Is Important When Creating Likert Scale Ranges 

A great deal of research has been done around scale points. When it comes to range, go for a wider scale range. Why? Because of the concept of Extremeness Aversion. Psychologists explain extremeness aversion as the tendency for people to avoid the outmost edge or ultimate poles of a situation, preferring a compromise or middle position.

A Duke University study provides a good example. Researchers asked people to go on a “virtual road trip.” As part of the exercise, participants were asked to make virtual meal choices along the way. In each meal option, participants were offered three drink choices: small, medium, and large, but the sizes of each were randomly changed throughout the trip. So, at one restaurant the small was 16 oz. and the large was 30 oz., and at another restaurant, the small would be 24 oz. and the large would be 55 oz., and so on. Researchers found that regardless of size, participants chose the middle-sized option.

Why Scales with a Middle Option Usually Fail to Provide Quality Results 

This is why we strongly advise against 5-point rating scales. Most people will avoid the two endpoints (1 and 5), leaving a 3-point scale (2, 3, and 4). With a 3-point scale, participants are most likely to choose the middle option (3). Too few response options/scale points, force participants to select the next best alternative which introduces a measurement error. Additionally, the analysis of a series of 3 from most of your participants will generally yield little to no actionable insights.

What about using 7- or 10-point scales? Some researchers argue that 7-point response scales are the maximum number that an individual can process. But if you eliminate 1 and 7 from the scale, 2-6 remain. Once again there’s a middle option, 4. The middle option is generally a neutral response. Use a 7-point scale only if neutral responses add value. 

Why a 10-Point Scale is a Powerful Option

We recommend a 10-point rating scale. While it may be hard to come up with “adjectives,” people better grasp the concept of something out of 10 options. Research has found that respondent preferences are highest for 10-point scales and allow respondents a greater range in which to adequately express their feelings. When 1 and 10 are eliminated, there is no middle choice. The 1 – 10 range forces participants to choose either a lower or higher score than the mid-point, which says a lot more about the participant’s point of view and is thus more actionable and valuable. The numbers 8 and 9 become far more meaningful as do numbers below 5. Plus, research suggests that the greater the number of response options, the more reliable the scale will be.

Now, it’s Up to You 

Composing winning rating scales takes the same elements as a good melody:

1. Having a healthy interval of questions to ensure reliability and validity 
2. Carefully considering the contour sequence of your questions 
3. Choosing the right scale of answer choices, and, most importantly,
4. Setting a wider range of options for respondents to choose from.

Successful melodies are not monotone. Listeners will better recognize the meaningful changes in pitch that make up the melody that they are hearing. Questions using rating scales provide valuable growth-related insights when properly crafted.

Let us know if you need help in any aspect of crafting effective primary research:

• Asking the right questions to define the objectives 
• Designing and fielding the survey instrument including ensuring your Rating Scales offer a wide range of highly relevant options 
• Interpreting the responses to provide more actionable business insights for you and your business going forward.