Survey design is difficult. There—we said it!
Every aspect of a survey needs to be carefully designed, which means you have a lot of things to juggle. Depending on your objectives, one way to make that juggling act a little easier is to carefully choose and utilize key drivers.
But what is a “key driver,” and how can it help your surveys shine? Let’s find out!
Key drivers defined
Key drivers are exactly what the name suggests: attributes that have been determined to drive your key metric. However, it’s important to differentiate between “key drivers” and their little siblings, “key attributes.” Here’s a closer look:
- Key attributes are like background information. They’re what you’ll use to determine your key drivers.
- Key drivers have had a correlation or regression analysis performed to show that they drive, for example, purchase intent.
So, where does all this attribute and driver information come from?
Turns out, it actually has a name of its own: key driver analysis. According to Survey Monkey, this process “looks at a group of factors and weighs their relative importance in predicting [...] performance indicators such as customer satisfaction, customer loyalty, or Net Promoter Score.”
We recommend starting with qualitative data analysis, like a focus group where you ask consumers what impacts their intent to purchase. If you have an existing survey, you can review its qualitative data. As The Balance explains, this isn’t about changing or controlling performance indicators—it’s about making accurate predictions regarding what drives those performance indicators.
The key driver discovery process
Discovering key drivers is a journey, not a destination. To avoid potholes along the way, run a correlation of your drivers to determine which are most relevant to your key metric.
There are three popular approaches to this process: regression analysis, Shapley value regression, and Relative Weight Analysis. At Alida, we prefer Shapley value regression—and here’s why.
Why Shapley?
To understand why we use Shapley value regression, let’s take a look at where it stands in comparison with the other two options.
Regression analysis
In regression analysis, we make a best fit line between our drivers and the key metric of interest. We then interpret the coefficient of each driver—the slope of the line—as its relevance. The higher the coefficient, the more important the driver.
Regression analysis is easy to run and interpret, but it suffers from one major flaw: It can be a complete and total liar. For example, when two drivers are highly correlated, regression analysis might give one driver a seriously overblown importance rating, throwing off your analysis. Worse yet, regression analysis might flip-flop its values, assigning negative importance to a driver with positive importance or vice versa.
Shapley value regression
Shapley value regression, on the other hand, is good at telling the truth—even when driver importance levels are highly correlated with each other.
In this model, we interpret the drivers as players cooperating in a game. The objective is to create the best regression model with the main metric of interest, which is measured using the R2 value, also called the coefficient of determination. A driver’s importance is a weighted average of the R2 value increase delivered to all possible “teams” of drivers.
Relative weight analysis
Shapley value regression is simple enough, but things get slow when datasets have a lot of drivers. In this case, it’s sometimes better to use relative weight analysis.
More complicated by far than Shapley value regression, relative weight analysis is faster and capable of handling larger datasets. While it often provides similar results as Shapley value regression, relative weight analysis is unique in that it can tell you whether a driver has positive or negative importance to the outcome.
Do key drivers have an expiration date?
You’ve done your qualitative research. You’ve identified key drivers. You’ve used Shapley value regression or relative weight analysis to make sure those key drivers are air-tight. Your work here is done, so you can kick back and relax—right?
Well, not quite.
It’s important to note that key drivers, while highly relevant, won’t be an accurate view of the truth forever. Depending on the topic, key drivers may have plenty of staying power—but for others, events like COVID-19 could render your research obsolete.
The best way to manage this is to do regular audits of your key drivers. Ask questions like:
- Does this data reflect our current target audience?
- Has the topic or area of our research been influenced by recent events?
- Do we need to refine the focus of our research questions?
- Is our existing approach effective pending a few updates, or do we need to start from scratch?
At the end of the day, key drivers need to be based on your audience’s view of the world; just remember how much that view can change from one moment to the next.
Putting it all together
Sure, survey design is difficult—but with key drivers to inform your choices, the process becomes a whole lot clearer. If only key drivers were that simple to nail down!
The key is to follow the three steps we’ve covered today:
- Perform key driver analysis.
- Study each driver’s relative importance for achieving target metrics.
- Review and update your key driver data as necessary.
Easier said than done, right? The good news is that you’re not alone in this journey. Keep an eye on our blog for all the guidance you need to build better surveys by researching, formulating, and choosing key drivers.
Contributors
Lisa Ketola is Alida’s NORAM Head of Professional Services where she leads the Implementation, Managed Services, and Research Services practices. Prior to this role, Lisa specialized in quantitative techniques as a marketing research consultant for more than twenty years.
With a master’s degree in electrical and computer engineering from UBC, Derek Dai has over 7 years of industry experience in product management and software engineering. He is passionate about building products that customers love by leveraging innovative technologies such as data analytics and AI.
Eli Yufest works in Customer Inspiration at Alida and is a part-time professor of market research, data mining & modeling and business metrics.
Sean La works in Alida’s AI team, where he develops data science solutions for Alida’s products and for internal use. In a previous life, Sean performed research in computational biology, with a particular focus on using data science to understand infectious disease epidemics.
Amy Ko is a Principal CX Consultant on the Professional Services team at Alida. She is responsible for thought leadership, best practices, and shaping the CX Practice.
