[Editor's Note: The following post by Ron Sellers was originally published by and is syndicated with permission by The GreenBook Blog.]
I often wonder whether, in research, we spend so much time navigating the complexities of gathering the data that we neglect the all-important field of communicating what we find. Issues such as online representativeness, phone response rates, and newer forms of data collection (mobile MR, social media sampling, etc.) take up so much of our mental bandwidth that it can be easy to give short shrift to clarity and accuracy in reporting.
One of the biggest and most potentially toxic issues is generalizing. Marketers dream about homogeneous populations – segments composed of consumers who are all looking to buy a new minivan, or who all have price as the number one criterion when choosing a cell phone provider. Because of the lure of homogeneity, it’s very tempting to generalize a segment that shows a greater proportion of certain people as being comprised solely of those people.
Geodemographic clustering falls prey to this quite easily. When I first learned about this technique a couple of decades ago, I was initially quite impressed that companies could identify clusters of people who were all “upscale Caucasians who are early adopters of technology.” It was a huge disappointment to find out that this segment, rather than being exclusively comprised of these people, simply contained 20% of these people, rather than the 8% who could be found in the general population (I’m making these numbers up). Although many purveyors of clustering clearly identify their methodology and how the technique is built, I’ve seen how this process is often used by marketers and researchers. Rather than discuss a cluster with a higher proportion of the desired target, they discuss the cluster as containing nothing but the desired target.
Research reports do this far too often, as well. For instance, in a major study of charitable donors for the Russ Reid Company (http://www.greymatterresearch.com/index_files/Heart_of_the_Donor.htm), Grey Matter Research found that donors under age 40 are more likely than older donors to have considered supporting a new (to them) organization for the first time in the last year (51% to 38%).
Now, this is a statistically significant difference that can be critical to marketers. Yet depending on the wording of the report, it could be very easy to lead readers to the feeling that all younger donors out there are running around checking out new organizations to support, while older donors don’t do this at all.
We see this all the time in real life: Democrats support same-sex marriages. Evangelical Christians are right-wingers. Engineers are socially awkward. Wealthy people own very expensive cars. Young people aren’t brand loyal. These are called stereotypes, and while they sometimes can reflect tendencies (certainly a wealthy person is more likely to own a Bentley than I am), they don’t hold water for every member of a group – sometimes not even for a majority of a group.
Let’s face it – it’s much easier to sit in front of a group of executives and explain, “Latinos simply aren’t interested in your product, while Asian-Americans love it” than it is to provide all the statistical details, differences in the data, etc. And sometimes marketing strategy can be as concise as saying, “Let’s pull resources from our Latino marketing and put them toward reaching the Asian-American market.” But it’s dangerous in reporting to over-generalize, or to lead the researcher user to believe that 25% versus 16% means one segment loves the product while the other doesn’t. Even when the generalization happens unintentionally, it still harms the effective communication (and therefore the effective use) of the research.
About Ron Sellers
Ron has 23 years of experience designing and conducting research that makes a difference. Thirteen of those years were spent running Ellison Research, which he acquired in 2008 and rebranded into Grey Matter. Twenty-three years. That’s over a thousand focus groups moderated (plus many more IDIs)…hundreds of quantitative studies successfully completed…over thirty published articles in industry magazines and trade journals…speeches given, seminars led, and classes taught…and scores of clients relying on his work in making key decisions. Ron’s work helped Chevrolet design and name the Chevy Equinox SUV. He helped the NHL’s Phoenix Coyotes design a new logo and branding scheme. He helped the American Red Cross design a successful holiday fundraising campaign. He helped General Motors rebrand all of their pre-owned vehicle lots. Ron’s work has been cited extensively by the international media, including Harvard Business Review, Associated Press, USA Today, Los Angeles Times, Financial Times of London, USA Radio Network, Washington Post, and a host of other media in the U.S., China, Sweden, Russia, New Zealand, Korea, Canada, the Philippines, Norway, Australia, England, and Central America. His work has even been used in panel hearings before the U.S. Senate and quoted in Jay Leno’s opening monologue on The Tonight Show.
The post Generalizing: The Bane of Insights appeared first on Research Access.
