Part of good survey design certainly also includes having inclusive demographic questions. But how much time and effort do market researchers put into their demographic questions? And how can we make them better?
I’ll dive into the topic in this article, which is largely based on an episode of “Reel Talk: The Customer Insights Show.”
What are inclusive demographic questions?
Inclusive demographic questions reflect the current ways people refer to themselves and are used in quant and qual studies. What those are specifically depends on the audience you are trying to reach and current practices on how people identify.
“There’s a base recommendation but then you also have to customize that by listening to who your base is,” said Shannon Danzy, director of research and strategy at RALLY – a communications firm.
This can include:
- Gender identity
- Sexual orientation
Why are inclusive demographic questions so important to focus on?
Inclusive demographic questions show that a brand cares about the people their consumers are, which can drive brand engagement and loyalty, which in turn can lead to better research.
Tchicaya Robertson, a 25-year research veteran and senior principal at Accenture for workforce transformation and diversity, equity and inclusion research, said demographic questions are important for a couple of reasons. One is that it can help businesses be more successful.
“We don’t just want to do the right thing,” she said. “We want to do the right thing for business.”
The inclusiveness of the research
Research is supposed to get insights from our communities and elevate their voices.
“It also means to allow people from different backgrounds to interject and contextualize the insights,” she said.
Better demographic questions help here. The questions are so important because there are practical decisions you have to make, she said. Unfortunately, some demographic questions aren’t created to the fullest because of budget restraints.
“But the more we do that, the less justice we do in our insights,” Tchicaya said. “You cannot create a sustainable program, if the demographic questions don’t reflect an identity we are trying to influence.”
“Having a place to start until you get it right is the right way to go,” she said.
How to know what demographic questions to ask
Tchicaya added that that process isn’t a one-size-fits-all.
“Our effort is about where do we start and how do we give guidance until we get the data where we can say ‘for this use case we need to ask these questions,'” she said. “And every demographic question is not required for every survey. At the end of the day, it’s about what are your hypothesis, what do you need to test and what’s the goal? What are the right demographic questions to get the right sample for that purpose?”
The evolution of people’s identities
Shannon said demographic questions are top of mind because there’s been an evolution of how people identify.
That evolutions includes:
- how people want to be identified
- what people want to be called
“You cannot not see that happening,” Shannon said. “Take what Tchicaya has been saying as the base and then pile on top of that what has been happening.”
These changes add to the importance of why researchers need to look at their demographic questions, she said.
That being said, a gen pop sample is not always the best solution.
“Gen pop is one way to look at it but better framing is to ask ‘what is the goal of the research?'” Tchicaya said. “If you are creating a product for Black people, do you really need a gen pop sample? You don’t.”
“We have to stop because it’s just not relevant for all the use cases,” she said.
How to come up with inclusive demographic questions?
“We are trying to figure out how to do research more inclusively,” said Tchicaya. “From the language, to the questions, to the hypothesis – the whole research cycle.”
Of course, it does start with awareness. Researchers need to be aware of the latest intricacies.
“I could be 14 different things – from nationality to race to ethnicity,” Tchicaya said. “That’s why we are trying to do it right. What are all the options?”
Then we need to evaluate our current demographic questions. That likely will include talking to consumers.
“We might be in a place where we make people uncomfortable because we are making people check a box that is there but doesn’t reflect who I am as a person,” she said. “Because the option isn’t there.”
It’s really about giving respondents the option to express themselves.
“How are we doing the right thing by the people that we are asking?” Tchicaya said.
The industry likely can’t be forced to change, she added, but understanding how people identify and keeping that in mind when sorting groups of respondents is the right thing to do.
Base your demographic questions on research, which is what the Insights Association tried to do with its demographic questions paper.
But keep in mind that the paper is just a start, added Shannon.
“The paper is not saying that this is the way,” she said. “The way questions are being asked is slowly evolving, too. There’s still a lot of research that’s being done. The paper is a summary of what are the best practices right now.”
Why the “other” checkbox isn’t the solution
Many surveys have tried to get around the issue by adding an “other” box, but really how many groups should go into “other?”
“I think we’ve been aware of doing it the wrong way for a long time and put in an option for ‘other,'” said Jenn Vogel, host of “Reel Talk” and vice president of marketing at Voxpopme. “Just by its own nature that says ‘I’m other than…'”
It kind of ignores people for what they are and how does “other” help us understand those respondents anyway? It really doesn’t.
“In terms of recommendations the ‘other’ shouldn’t be there and you should allow that person to define themselves,” Shannon said.
For example, add a phrase like “I identify as…” and then allow people to identify themselves. Use the right technology to group open ends, Shannon said. Plus, seeing the open-end answers can help you make the next round of demographic questions more inclusive.
“They are going to be more accurate for you to get the people that you want in your study,” Shannon said. “And it makes the people in your study realize that they are seen and they’ll want to participate more in your study. They’ll say this company gets me.”
“We need to treat people as humans whose opinions we really want to understand,” Tchicaya added.
The nuances of inclusive demographic questions
First, we need to understand what the different options for demographic questions mean.
For example, Tchicaya said there are differences between:
- Gender – often established ideas of male, female and now some surveys add a third or more options
- Gender identity – somebody’s internal sense of what they are
“They sound very nuanced but are very different things,” Tchicaya said.
Your demographic questions, in this case, could be:
- What sex were you assigned at birth? (answers gender question)
- Then allow people to describe themselves – male, female, transgender, or none of these, and what’s your current gender identity?
Each also includes an option of:
- Prefer not to answer
“That’s to give a human the opportunity not to feel every question they respond to is done in light of how they answered the question,” Tchicaya said.
A broader look at race and ethnicity
Keep in mind that the limited definitions of race and ethnicity we’ve used in the past, are no longer enough. Let’s take the example of being Hispanic. Historically, this question in surveys would be “are you Hispanic or not Hispanic?” We need more options in our surveys to be more inclusive. A Hispanic person could be, for example:
- from Spain
- a descendent from somebody that lived in Spain
- from Mexico
- a descendent from somebody that lived in South America
All these groups have had different experiences and aren’t the same group of respondents necessarily, yet we’ve bucketed them together historically.
“Being able to reflect that is important,” said Shannon. “We want to be able to reflect the nuances. Being more detailed on the front end will help you on the back end.”
Also, Shannon added, keep in mind that just because somebody appears to be from one group or another doesn’t mean they want to be called that.
For example, Black and African-American isn’t the same thing, though, it’s often listed as Black/African-American.
“That implies it’s the same thing, which it’s not,” Shannon said. “African-Americans are people who were born in the U.S. and choose to be identified as African-American. Just like there’s a movement with pronouns, there’s a movement in the Black and African-American community as to what we should be called.”
When should demographic questions be asked?
Many times they are asked at the beginning – where they can also be used as screeners when that’s necessary.
“It can be a turn-off to have them at the beginning,” said Shannon. “In other cases or many cases really it’s better to have them at the end. Get the information that you want and then ask these questions at the end.”
Consider the overall makeup of the sample
Elena Lyrintzis, marketing and culture insights lead, Devices and Services at Google, discussed on “Reel Talk,” the importance of getting the right collection of people into the final sample for a survey.
“It’s something to be hyperfocused on when you are creating that sample structure, that quota structure,” she said.
And consider how to reach people. Some people might not have the time to take a survey on their computer. Others might not have time for a remote interview.
Different demographic questions by region
Not every country is a melting pot like the United States. Because of the diverse population in the United States, the demographic questions discussion is an important one to have. But that might not be the case in other countries. For example, in countries that are predominantly one race, asking about race in your demographic question might not make sense. It might even be distracting. “Why are they even asking me this?”
“If you are in a predominantly black country asking if you are identifying as black is like ‘what do you mean – what am I?'” said Tchicaya. “You do have global considerations and that includes in which country can you even ask the race question.”
Analyzing inclusive demographic questions
Having updated questions is a start, a start that needs to continue into the analysis. If you just put them all in the traditional buckets of demographic data that’s not really using inclusive demographic questions to the fullest. But that can happen because we are under time pressure, and Ray Poynter previously predicted that expectations for speed will continue to increase in market research.
“So we take the care to ask all the right questions and then on the back end we aggregate them all like we haven’t taken the care,” said Tchicaya.
This is so important because putting people in the wrong buckets can negatively impact the insights we get. Jenn mentioned the example of Prince Charles and Ozzy Osbourne, who would be put in the same bucket based on demographics alone, but it’s important to consider they probably care about different things.
“They are very different people,” Jenn said. “But if you segment them in a survey response they could be in the same bucket.”
How often should demographic questions be evaluated?
“The word on the street is that research is old after six months,” said Tchicaya. “There’s never a destination. There’s always learning; always something to consider.”
We might miss a group to include in our demographic questions. Something new might be learned. It’s a never-ending experience and ongoing learning and evaluation project.
“We have to be comfortable challenging what we see as the authority on these questions,” she said.
Asking inclusive demographic questions can certainly make our research better, which in turn can help us provide better insights to our business. That can have a business impact and even help us get and keep support for more research.