Today, Origin Ventures posted about their recent investment in Voxpopme. We’re thrilled to have Origin as a partner supporting our growth, and are excited to see what the future holds. Take a look at why Origin is excited about Voxpopme!
We are pleased to announce Origin Ventures’ investment in Voxpopme, an enterprise software company that allows brands to capture video feedback to measure customer satisfaction and gather market research. Voxpopme enables brands to use the power of video – featuring full sight, sound, and motion – to better capture consumer sentiment in real-time.
Origin Ventures led a $9M Series A financing. The company – whose name is derived from the Latin phrase vox populi, or “voice of the people” – has offices in Salt Lake City, the U.K., and Australia. Customers include Coca-Cola, Nestle, Microsoft, Colgate, Verizon, and Airbnb.
“Be it customer service, product quality, or just the way the customers feel about the companies they do business with, customer experience rises to the top of whether or not the customer will decide to keep doing business with a brand.”
A Gartner survey of marketers found that 89% expect to compete mostly on the basis of customer experience, vs. 36% four years prior. Verticals like media & entertainment, travel & hospitality, and financial services deliver much of their user experience digitally, with less direct interaction with customers than ever.
Digital Natives and Video
The investment aligns with our thesis centered around Digital Natives, and the user behaviors of a generation that has enjoyed the Internet and a smartphone for their entire lifetime. Video consumption dominates Internet traffic. Cisco suggests that 80% of Internet traffic is video, while Facebook, Instagram, and Snapchat combine for over 20 billion video views per day. In a post titled “Texting is Out. Spontaneous FaceTime Is In.”, a 21-year-old college student says “You’ve gotta see their face,” while a 19-year-old says “FaceTime was released in 2011, when I was 11. My peers and I literally grew up with it.” We believe a logical extension takes video from social media to communication directly with brands. Airbnb employs Voxpopme to improve voice-of-the-customer insights and help its employees develop empathy for customers.
The investment extends Origin Ventures’ interest in both video and customer experience. Investments in Cameo and ViralGains also have video at their core, while Seattle-based Apptentive helps brands like Starbucks, eBay, and Disney capture customer feedback on mobile devices.
We look forward to working with CEO Dave Carruthers and the global team at Voxpopme to build the leading software company for connecting brands and consumers directly via video, and the red button on their smartphones.
When A.I. really blossoms, how will it affect your approach to research and the wider insights industry?
Welcome to Episode 18 of Perspectives!
A.I. promises to have a huge impact on the insights industry with some impressive solutions hitting the market in recent years. In this episode, our panel of experts discusses how A.I is impacting the industry and what the future holds.
Ready to find out more? Hit play on the video to discover the influence A.I. is likely to have on your research and the transformative effect it will have on how we view our insights.
As artificial intelligence gets smarter and smarter and starts guiding more and more of our purchase decisions, and guides us and the things that we buy and spend our money on, who is it really that we’re going to need to talk to from a research perspective?…@BaillieForGood
I think AI, once into its prime will let us start getting high degrees of confidence with potentially smaller amounts of people, if it’s the right people. So I think one of the big things that will change beyond making things quicker, better, faster, cheaper, is it will actually change what we think about as being the most valuable thing that we want to optimize for… @Werdnamai
So when I look at the role of AI today in the world generally but within the insights industry, I say that a lot of the focus is on automation and how AI can take people out of the loop. I think that has a role to play for the future of the industry. But I think far more exciting is the potential for AI to accelerate human expertise, not take people out of the loop…@Jonathan Williams
AI will have a huge impact on the insights industry. The main impact it will have on our insight business, is that it will enable us to inform more decisions across the business. At the moment, the amount of decisions that we can influence using customer insight is constrained…@pd_hudson
2018 was a hugely exciting year for us here at Voxpopme. From the launch of our video customer experience platform, VideoCX, to new partnerships and exciting new hires, it’s fair to say that it was a busy one! On reflection, it was an eventful year for the market research industry too, with trends such as automation and the continued rise of emerging methods, such as online communities and mobile surveys, taking center stage. Now that the dust has settled on 2018, we take a look back on an outstanding year in research and look forward to the future as we explain how using agile video in qualitative and quantitative research can revolutionize your insight program in 2019…
Getting close to the consumer
According to the final GRIT Report in 2018, forward-thinking companies embraced technology last year to get closer to what people really think. This resulted in far greater adoption of technologies that facilitate agile qualitative research, employed across a multitude of studies (quant and qual), as companies across the globe redefined their goals to focus on making the consumer accessible again. This was perhaps driven by the need to cut through the noise in a data-heavy world and researchers’ growing desire to understand and build empathy with consumers through real conversations.
Over time, there’s been a growing demand for greater speed and increased insight – but, at the same time, research budgets seem to be under constant pressure. And that’s why automation was one of the most notable trends of 2018. Is it a faster and cheaper way to carry out research? Absolutely. But in order to have staying power, automated tools must deliver insight that is impactful and memorable. Sure, research needs to meet the need for speed, but it also needs to go beyond facts and figures and find out the why. And that’s where automated, agile video research comes in to deliver in-depth insight, at speed and scale.
How agile video can revolutionize your research
Including video in qualitative and quantitative research can help companies humanize their insight and get to the bottom of what consumers truly think. The increasing popularity of video in consumers’ own lives is revolutionizing the way in which researchers are delivering scalable, affordable, qualitative research. Self-recorded consumer videos can now be collected in any study and when paired with video analytics platforms like Voxpopme, it’s possible to fast track end-to-end video projects.
This approach to video is a compelling way to deliver real human feedback. It’s an entirely different ball game to video market research of the past. Today’s automated video insight and analytics technology enable you to gather and understand hundreds of videos in just a few minutes so that you can find emotion faster. And there’s no need to hire a camera crew or to endlessly trawl through footage looking for insights either – in fact; you don’t even need to physically go to your consumers anymore. Instead, self-recorded videos are captured remotely, uploaded to your video analytics platform and instantly human transcribed, time-coded and quality checked, before being filtered and rated for sentiment. As a result, in a matter of clicks, you can discover the themes that matter most and share them with your stakeholders. Sound good? Here’s how video can transform your research in 2019:
Video empowers consumers
Video empowers consumers because they feel like you’re listening to them and hearing their individual needs. It allows them to share their experiences in their own words, which improves engagement and helps them feel closer to your brand. Most of all, video makes your consumers feel that their opinion really matters. Think about when you’ve written a text-based response – did you really write down everything you thought? How about ticking boxes in a survey, or measuring responses on a scale – did they represent what you were truly feeling? Probably not. And that’s why video is a game changer: because it affords consumers a greater level of expression that feels like it will be heard.
Video reduces survey fatigue
The days of traditional text-based surveys or lengthy tick-box questionnaires are numbered. Instead, consumers are turning to video for a frictionless and fun way to share their thoughts and opinions. It’s a much easier way for them to express themselves than closed-end questions or tired text boxes, and it’s much quicker too! Not only that, but video is fun – and thanks to the broad range of selfie style video responses available, there’s a variety of ways for people to get involved and share their thoughts including in-store shop-along videos, experience-based feedback, in-home product testing and more. With video, consumers can express and articulate their thoughts and feelings anytime, anywhere which significantly reduces survey fatigue – something which was a growing concern in the market research industry last year.
Video enriches consumer understanding
Video allows you to cut through the noise to obtain raw, unfiltered stories direct from your consumer. It delivers more depth and emotion than scores alone, helping to bring a richness to your results and allowing you to uncover the human stories that often get buried in a data-heavy world. It also gives you unrivaled access to rich verbal descriptions and expressive body language, which means you can understand verbal and nonverbal communication clues to get to the bottom of what your consumers are thinking. That means you can uncover the why behind the what and make sure you truly understand what makes your consumers tick.
Video brings your data to life
To drive real change with research, we’re often told to tell stories – but sometimes it can be hard to find those stories in our burgeoning datasets. Video can help by making consumers the narrators of their own stories, turning them into much more than a statistic. These stories are easily collected by putting agile video in qualitative research or integrating video directly into quantitative studies. Either route helps you tap into real, human feedback that is far more revealing than data alone. Humans are not rational information processors, and we can’t keep using data alone to second-guess what consumers think and feel. Instead, we need to humanize data and bring it to life, so we can build empathy and start to add context to the numbers.
Video engages executives
Forbes revealed that 59% of executives would rather watch a video than spend time reading text-based alternatives. But why? Well, video stands out in the age of ‘information overload.’ The fact 90 percent of the data in the world was generated in the last two years further emphasizes the need for more engaging ways of communicating research – especially if reports are going to be memorable. It’s not that executives aren’t interested in facts and figures anymore, they certainly have a role to play, but when we’re drowning in data, video cuts through the noise.
It can be too easy to ignore graphs and charts. However, by sharing real video snippets of real consumers sharing real opinions, you can command the attention of your stakeholders and make your research stand out. Ultimately, this ensures consumers’ stories are heard by the people that matter and drive positive change across all departments of your organization.
Ultimately, 2019 is set to be an exciting year for video market research. Executives’ insatiable desire for human insight is driving positive change in the space. While 2018 saw researchers lay strong foundations for agile video, the pace of adoption is likely to snowball in 2019 as we see fast, in-depth insight repeatedly contributing to critical business decisions.
Video has always been a powerful tool that gives consumers a unique and fun way to share their opinions with the brands that matter to them. But that’s not all: using global video market research also gives brands access to human feedback that’s deep in meaning and emotionally engaging so they can make informed, customer-centric decisions. However, although video is powerful in terms of the depth of feedback it provides, it has previously been geographically restrictive.
In the past, researchers needed a camera, and maybe even a crew, to capture customer insights via independent studies which took place in a specified location with a small number of respondents. All of this meant that some researchers were put off using video in research because it was seen as time-consuming, expensive, and with a limited reach. Powerful responses, yes – but dynamic, global studies? Not so much.
However thanks to today’s technology, things have changed. Researchers can now run global video studies without leaving their desks as feedback is captured straight from consumers’ digital devices, anytime and anywhere in the world. Businesses can now get to know the people behind their brand and uncover real human stories from around the globe, driving customer-centric decisions through agile and engaging video research without even having to leave the office. Sound good? Here’s how you can run global video market research studies…
Identify your audience
First things first, you need to decide where you want to conduct your video market research and which markets you want to speak to consumers in. If you are looking to carry out your research in different markets, you’ll need to decide how many territories you’d like to collect video in. Next, you need to decide how many video participants you need per market – and you’ll also need to factor in translation time for your surveys too.
If you’re working in multiple English language markets, you can often use on-demand video feedback communities – but if you’re collecting in several non-English speaking markets, then you’ll need another way to connect with respondents. You can approach this by inviting your existing community/customer groups, or by recruiting respondents through a panel partner – and your video insight vendor should even be able to connect you with panel partners and possibly also assist with the recruitment process.
To get the best possible results from your research, it’s important to have a sufficient sample size of high-quality respondents that best represent the segment or group you’d like to understand better, whatever location you choose. Agile video market research enables you to collect video from any audience, whether you want to integrate video open-ends into existing communities or surveys, set up new video-centric studies and recruit through panel providers or use video panels such as Voxpopme’s On-Demand community.
Basically, there are tons of options, no matter how specific your audience. So whether you want to run video diaries with a select audience in the US, conduct shop-alongs in Germany or add qualitative questions to quantitative surveys as far afield as Timbuktu, Voxpopme has an array of tools to capture, transcribe, translate and analyze video in any language.
Integrating video research into your global surveys
So, it’s time to get started and begin integrating video into your multilingual surveys – and the truly great thing about using video in research today is that capturing video feedback has never been so easy. Whether you want to collect videos alongside, within or after a survey to boost the impact of your results or set up agile qualitative studies, you can easily bring a human voice to your data. That means you can sit back and collect hundreds of videos from all over the world – without even having to leave your desk. There’s a capture solution that works for everyone:
Embed – Add video open-ends to any new or existing survey so you can easily capture feedback from any consumer device, anywhere in the world. All elements of the embed technology are available in multiple languages so your respondents can experience video feedback that’s tailored to their needs.
On-Demand Communities – With Voxpopme’s mobile app communities, you can capture hundreds of videos from your target audience – including analysis – in less than an hour, which means it’s perfect for English speaking projects in North America, Australia, and the United Kingdom.
Offline App – Voxpopme’s offline solution allows the seamless collection of video in environments with no connection, making it an excellent solution for consumer in-home interviews in developing countries or in-field studies with limited internet access. Once the capture is finished, you can sync your videos back to the platform when an internet connection is established.
Moments App – Voxpopme’s newest app gives you an agile video diary tool so you can pre-set video tasks spread across days or even weeks for moment-in-time experiences, pre-group homework tasks, in-home product experiences and more. With global availability in the app-store and multiple language options, the app opens up agile qualitative studies around the globe.
Upload – Already got focus groups and IDIs from multiple locations and in different languages? Voxpopme’s upload tool will enable you to add your existing content to the Voxpopme platform for instant transcription, translation, and analysis.
How does video analysis work?
Once you’ve captured your videos, it’s time to analyze them. Previously, researchers have been put off using video, especially in global studies, because it was thought to be too time-consuming and expensive. Because of manual time coding and transcription, language barriers and translations have been tricky in the past too – and that’s without even thinking about the vast amounts of footage to trawl through after interviewing customers from all over the world! Essentially, video analysis was previously too laborious – but that’s not the case anymore.
Today’s technology has automated video analysis making video research faster and more cost-effective than ever before, wherever in the world you want to conduct your studies. For starters, agile video market research allows transcription and translation into any language, and videos uploaded to Voxpopme are instantly human-transcribed, time-coded and checked for quality. Once your videos are in the platform, there’s an array of tools at your disposal so you can quickly find the best and most relevant responses for your business.
Search tools allow you to filter responses by keyword or variables such as date, age or location, while sentiment charts and word clouds allow you to explore responses even further. In addition, thematic analysis automatically organizes your content into key phrases and topics to make it easier than ever before to identify common themes across hundreds of videos. It works by automatically identifying the most important snippets of video and illuminating key findings, taking you straight to the snippets where respondents mention your chosen theme, so you can spend less time searching for answers and more time telling better stories.
If that wasn’t enough, sentiment analysis is also available to help you categorize video content by respondents’ true feelings. That means you can understand the sentiment behind every single sentence in every single response – or even take a step back and view sentiment at the theme level. The best bit? Videos can be transcribed and translated for more than 80 supported languages, so no project is ever too difficult. That means wherever in the world you want to run your market research study, Voxpopme has your back. So you can save time, increase efficiency, and deliver maximum impact – worldwide!
Sharing video research results with your global team
What about sharing content with global teams? Well, in the old days there probably would have been language barriers to overcome – and if you needed to share global research with other offices in other departments, it would have been near impossible to get everyone together to view your findings or easily socialize large, cumbersome video files. Not now though. Instead, once your video has been analyzed, you can select, customize and share your most compelling consumer stories with your stakeholders.
In-built editing tools allow you to subtitle responses in any language so you can piece together snippets and overlay a common language that can be used across your business, which not only helps you overcome language barriers to make your research engaging and entertaining, but it also allows you to build impactful customer narratives for presentations. It’s also quick and easy to generate a showreel from your project – simply drag and drop your cursor over any transcribed section of a response and add it to your latest showreel to easily assemble the most insightful video moments from all your feedback.
You can even use auto-generated themes to jump straight into clips that contain a specific topic. And in a few simple clicks, you can add your logo, color scheme, custom slides, subtitles, images and more to transform your video feedback into impactful stories. Once it’s complete, simply hit “generate” and share by embedding your showreel into your next report or presentation. If you prefer, you can set up a private, password-protected landing page and share your edited showreel using the web link – or you can even send it via email to colleagues around the world.
Although global video market research studies might have seemed daunting in the past, video insight platforms have ensured that it’s easier than ever before to access respondents around the world. Ready to revolutionize your research in 2019? Contact with the Voxbot in the bottom right of your screen to arrange your free demo and see how you can access customers’ thoughts and feelings worldwide through global video market research.
Using video market research is a fantastic way for companies to get to the bottom of what their customers are really thinking. Video is the single most powerful way to deliver human feedback and see how customers feel about your brand, products, and services – and thanks to today’s technology, video feedback can be captured quickly, at scale, in any location, straight from consumers’ digital devices. This agile approach to video market research means customers can express themselves in their own words, allowing you to see the people behind your data, reveal the true stories behind your statistics and drive real change right across your businesses.
However, despite the new found ease of video, there’s still a lot to think about when it comes to using video in research. As with any other study, it requires sufficient planning, defined research objectives, consideration of the available capture tools and even production of the right questions. So, it’s important to be aware of any potential pitfalls before your study begins to ensure you run a seamless video research project and obtain the insight you need. So without further ado, we give you our list of the five most common video market research fails (and what you can do to avoid them!)
1. Not choosing the right tools
When it comes to efficient, effective video market research, it all starts with the right tools. Knowing the video capture solutions available to you is the all-important first step in equipping your research toolkit with versatile video solutions. Today’s technology has evolved video into something of a quant/qual hybrid and there’s a range of different tools you can use to collect video content in a way that matches your own specific needs:
Embed – Add video open-ends to any new or existing survey to capture feedback from any consumer device.
On-demand Communities – Some video insight vendors will have mobile app communities available for quick-turn feedback, enabling you to capture hundreds of videos from your target audience in less than an hour.
Upload – Whether you have short video open-ends or lengthy focus groups and IDIs, you can often upload content to platforms like Voxpopme to use automated analytics on your existing video files.
Offline Apps – Offline app solutions will allow seamless collection of video without the internet or heavy-duty camera equipment. Once you’ve captured content offline, all you’ll need is an internet connection to sync your videos back your video insight platform.
Diary Apps – Tools like Voxpopme’s Moments app give you an agile video diary tool so you can pre-set video tasks across days or even weeks. This enables you to capture moment-in-time experiences, run pre-group homework tasks, see in-home product experiences and more.
2. Not introducing video research clearly
When conducting video market research, you need to present the video feedback element to your respondents in a clear, concise manner. It’s important to be upfront and honest about what is expected of them from the beginning and communicate what the video will be used for, who will see it, and why. This is particularly important if you’re embedding video questions into a quant survey. Our research suggests that the highest percentage of usable videos tend to be captured when participants are told about the use of video in both the email invitation and the survey welcome page too. At the same time, you should also make sure you big up the technology, fun, and ease of video market research too. Respondents are far more likely to engage with video if they can see that it’s quicker, easier and allows them to be more expressive than traditional feedback methods. The moral of the story? Providing advance notice and sharing clear instructions upfront encourages higher video response rates, but try to keep your messaging fun and concise to avoid overwhelming participants.
3. Not offering the right incentive
Finding the appropriate incentive levels can be a challenge for any research study – and when it comes to video market research, it’s no different. It’s a delicate balance: incentives are a great way to show your respondents that you are grateful for their time and reassure them that their opinion matters, but it’s also important to make sure you don’t under or over-compensate for their feedback. Having said that, video is a fun and engaging feedback style so you shouldn’t be daunted by the process of finding the right incentive. Here are some pointers to ensure you hit the right incentive levels for each project:
Firstly, you need to know your audience. If you have a highly engaged community, you might not need to incentivize at all, or you could be dealing with an entirely new group of respondents that are also new to video – in which case an incentive could be a good way to help them get comfortable with video. Be sure to consider your incidence rate too. If your desired respondents are a very niche audience within a broader segment, consider increasing incentives to attract the right people and increase the probability of project completion.
You’ll need to assess the difficulty of your video feedback. Asking someone to record multiple diary-style videos, across several days is inevitably going to require a more substantial incentive than a one-off response to a single open-ended question.
Finally, you need to make sure that your incentive reimburses respondents for any costs they’ll incur. For example, if you’ve asked them to go out to a shop and purchase your product, then you’ll need to cover their costs in addition to paying them a token to thank them for their time.
4. Not asking the right questions
Next up, you need to determine the questions you want to ask and in what format. When drafting your questions for potential video responses, you should make sure they are open-ended and easy to understand so your consumers can deliver valuable, visual insight. And don’t forget, if you need your respondents to complete a task, review a concept, or watch an ad in advance of providing their feedback, you’ll also need to make sure you provide them with clear, simple instructions of any preliminary work they need to complete too. You’re effectively the director in this relationship so give respondents all the pointers they need express themselves fully.
Clear instructions, paired with open questions will help you get closer to the truth by revealing the all-important “why” behind your consumers’ perceptions, motivations, and decisions. And even though consumer-recorded videos are captured without the aid of a moderator, questions that are sufficiently challenging and give people the opportunity to speak their mind will encourage them to provide the most thoughtful answers. Typically, the most insightful responses come from respondents who are challenged, so it’s less yes or no and more what, when, who, how and why. Check out this blog for some great ideas on open questions.
5. Not being realistic about time frames
End-to-end video insight platforms have transformed the speed at which video projects can be captured, analyzed and shared – delivering full video projects in hours and days, not weeks and months. But it’s still important to be conscious of how long your latest project will take because there isn’t a one-size-fits-all timeline for agile video research. Projects can vary depending on a number of factors, from your chosen audience to the volume of video required and even the type of video study.
However, some general timeline considerations to think about, in addition to the time it takes to conduct your research, are how long it will take for transcription and time-coding to take place post-recording. In platforms like Voxpopme, quick-turn, human transcription is applied to videos in minutes to ensure you get fast, highly accurate transcriptions – providing a solid foundation for all the following automated analytics.
For a non-English video project, allow additional time for any video translations you require (approximately 24 hours after collection) and make sure you plan the turnaround time for any additional reporting or custom showreels. Don’t worry though, if you are sticking to a tight schedule there are some things you can do to speed things up. Automated sentiment analysis and theme coding of your content can help you find insights faster and DIY editing tools within your video insight platform help you to generate showreels in just a matter of clicks.
Done right, using video in research can give you access to unbeatable insights so you can start to tell your customers’ stories and drive real change across your entire organization.
Young people are the brand influencers of today and tomorrow. As the leading actors of the digital era, Millennials make up 31.5% of the world’s 7.7 billion population, while Generation Z will account for 32% of the global population by 2019. As the spending power and influence of these young people continues to rise exponentially, so too does their importance in market research.
So how can market researchers get closer to this influential customer group and find out what makes them tick? Look no further… This short post takes a look at how modern video market research engages young people so they can start to share their story with your business.
Appealing to younger audiences
Do you remember listening to the sound of dial-up internet? Well in today’s world, you’re in the minority (sorry!). The majority of young people in 2018 struggle to remember a life without a continuous internet connection, let alone the whirring and beeping of the dial-up connection. Most of Generation Z can’t even remember life without social media. This generation is the first true group of digital natives, with over 91% already having a digital footprint and 98% owning a smartphone.
Thanks to platforms such as Snapchat, YouTube and Instagram, young people today are living – and sharing – their lives on the go. They are used to bite-size pieces of information they can process right away and like to express themselves in the same way. That means short, snappy audiovisual content, such as self-recorded videos, are a great way to engage this on-the-go generation in research.
The allure of video
Self-recorded video is growing in popularity by the day. Forty-eight hours of video is uploaded to YouTube every minute, and it’s estimated that 63% of internet users have watched live video – and unsurprisingly, it’s those aged 25-34 who have the highest video views at 90%. What does this all mean? That video creation and storytelling is no longer solely the property of huge film producers – everyone can now share their story, from anywhere in the world and we’re consuming it in larger volumes than ever before.
Most of us are never more than a few feet away from a smartphone or laptop with a built-in camera, meaning that we have a new generation that is entirely comfortable self-recording their thoughts and feelings. And with the growth of platforms such as Snapchat and Instagram Stories, it makes perfect sense that video is fast becoming the chosen communication tool for sharing these stories – especially when it fits in so well with the ‘on-the-go’ lifestyles of the younger generations.
In recent years, platforms such as Voxpopme have been able to leverage this shift in communication habits to benefit market research. By using technology, we’ve added structure to the way video feedback can be captured, analyzed and shared within enterprises – making video a hugely popular way for all generations to communicate with brands. Here are just a few reasons why agile video market research is proving to be so popular, especially with younger generations…
Young people today love to share their lives on social – and they’re becoming more and more confident using video to do so. With Snapchat boasting 186 million daily active users and Instagram Stories continuing its bid for world domination with 400 million daily users, it’s official: vlogging, video stories, and diaries are growing in popularity by the day. This self-recorded video style has lent itself to more structured communication between consumers and brands, with businesses now putting video feedback questions at the heart of their market research, providing a fun and engaging way for young people to take part.
After all, it’s fun to self-record a short video recounting an experience or expressing what they think about a product and why. Especially when the alternatives are a long survey or a 30-minute face-to-face interview in a specific location. And with the options provided by video insight platforms, there’s a type of video that appeals to everyone, that can be recorded in their preferred environment, on their terms. Consumers can capture selfie-style content during in-store shop-alongs, share experience-based feedback videos at events, test products, and complete diary studies from home or even share concept feedback in a variety of locations. Agile video feedback offers a wider range of fun and engaging ways for young people to get involved in research and share their thoughts.
Young people today live in a fast-paced world of continuous updates – they want information, and they want it now. They have grown up with technology and high-speed internet, and as a result, they expect things to work right away. This generation shares and receives information instantly – and they can lose interest just as fast too, which means they have no time for lengthy surveys. Survey fatigue is a real and growing challenge in the market research industry, and if other generations are struggling to complete exhaustive 30+ question surveys, you’ve got even less chance of young people doing so. Can you even remember the last thing you did for 30-minutes on your phone?
Our lengthy surveys are built with good intentions – we hope we can capture scales, scores and then give an opportunity for expression with open-ended questions. And to encourage respondents to share their real feelings, we’ve stuffed these open-ended text boxes throughout our surveys to enable respondents to share intimate thoughts, but in reality, they are pretty restrictive. The effort required to type out and edit one’s complex feelings often result in responses of just three or four words and little or no storytelling which leaves us back relying upon scores alone. But, video open-ends offer a quick and easy way for people to express themselves and share their opinions beyond the confines of their keyboard. It fits in around busy lives and allows customers to record anytime, anywhere, from their preferred digital device – so they can share in-depth stories easily.
Once transcribed, a typical video response gives an average of 460 character responses with a typical word count of 75. Video delivers six times more content and 65% more themes compared to text-based responses. Instead of scrolling through a survey and endlessly selecting options, or typing out long answers, video lets your customers get to the point, and fast – making it perfect for Millennials who want instant communication at their fingertips. Many go a step further, seeking out on-demand video feedback communities so they can express themselves to more brands, about the products and services they care about most.
Another benefit of conducting video market research with young people is that it empowers them by enabling them to express themselves and ensures they feel heard in our enterprises. Think about when you’ve given written feedback yourself – did you write down everything you thought? Did you feel like you could effectively portray emotion through your words? And did you feel like your opinion was genuinely going to get heard? Or how about when ticking multi-select boxes or ranking responses on a scale. Did they represent what you were feeling? Nope, we thought not.
Your customers need to believe their feedback is making a difference – otherwise, to put it bluntly, why should they bother? And that’s where video has the edge over other research methods, especially with young people who really want to make a difference and feel as if their opinion is valued. Video makes young people feel like they are more than data – and it’s much more personal too. By allowing your customers to portray emotion and express both their body language and facial expressions, video adds a human aspect to communication – making young people more engaged, empowered, and willing to express themselves.
How it works
Researchers are often nervous about how long video will take and how expensive it is. But, today’s technology enables you to begin capturing hundreds of videos from your target audience in just a matter of minutes. It can be used as an agile qualitative tool or within quantitative research, so whether you want to utilize video in a new or existing study, collect videos alongside, within or after a survey, or capture content online or offline, it’s a quick and simple way to bring a human voice to your data.
Once your videos have been captured and uploaded to a video insight platform like Voxpopme, they will be instantly human-transcribed, checked for quality and time coded so you can quickly find the responses you need. Advanced tools like automated thematic analysis will also organize your content by keywords and phrases to make it easier than ever before to analyze. In addition, sentiment analysis is also available to help you categorize video content by positive, negative and neutral feelings so you can understand the sentiment behind every single sentence in every response. Finally, once your videos have been analyzed, you can select, customize and share your most powerful customer stories in short showreels that bring the voice of younger generations into the boardroom.
In conclusion, video market research allows you to cut through the noise to obtain raw, unfiltered content so you can better understand your consumers’ true feelings. But it’s not just about the benefits for you – video market research is also an unbeatable way to reach out to young audiences. It’s fast, it’s fun, it’s empowering – and above all, it’s a fantastic way for you to reach out to the customers of tomorrow.
The GreenBook Research Industry Trends (GRIT Report) is one of the leading publications in market research thanks to its renowned survey of the industry, for the industry. It’s an incredibly useful resource that allows researchers to understand where the industry is now, where it’s going in the future and what businesses need to do to adapt, evolve and stay ahead of the competition.
So what’s the lowdown? Well, for starters, the market research industry remains under great transformative pressure – and this GRIT Report aims to get to the bottom of what’s happening, why it’s happening, and what it means for the market research industry tomorrow. So what’s new in the world of market research? And what does that mean for video market research? Read on to find out…
GRIT Report on the ongoing advance of automation
One of the biggest focuses of the report was how forward-thinking brands and agencies are harnessing technology to get closer to what customers really think. Automation continues to be one of the hottest conversations in the market research world, and in recent years many organizations have adopted automation platforms in quantitative and qualitative research. Why? Well, the main reason reflects the one sweeping goal all of the companies involved in the report have in common: the need to better understand the customer and discover rich, impactful insight at speed.
And in modern business, where markets change in the blink of an eye, automation delivers on those needs. Technology has been used to automate charting, analysis of survey data, delivery of sample, analysis of text data, analysis of social media and analysis of video – and those are just the tools that are achieving significant adoption. As a result, expectations for automation in the industry are very high, with almost 80% of researchers agreeing that automation will continue to grow in adoption in market research in the coming years, while 70% think it will grow within their own organization.
Over time, many traditional research methods have become either too expensive or too time-consuming, and that’s often been the driving force behind automation and innovation. This forced evolution happened to quantitative research first but has heavily influenced qualitative approaches more recently. In a fast-paced world, the resources required for qualitative research have become increasingly hard to justify. Yet the need for qualitative market research has never been greater. It’s all about breaking down the barriers and making the consumer accessible again in a world that’s awash with data. So what has automation done for qualitative research and more specifically, video insight?
GRIT Report on video market research in an automated world
The previous issue with automation was that it had a value problem. Yes, it’s always been faster and cheaper, but too often this speed often requires us to sacrifice the emotion and authenticity we need in our feedback if we want to make it memorable. Enter agile video…
Video market research has always had the ability to help companies humanise their research and get to the bottom of what really drives customer decisions. And the perfect blend of sociological and technological evolution has made video a scalable, quick-turn, qualitative research method. Thanks to the increasing popularity of video in consumers’ own lives, they are entirely comfortable with self-recording videos and sharing them with the world on social media. This comfort with storytelling has created a global community of respondents that are ready and willing to share video feedback, in an instant, in response to video surveys. The accompanying rise of video insight platforms facilitates the capture, analysis, and sharing of self-recorded videos in end-to-end solutions that overcome all of the hassle factors associated with video.
As a result, brands like Microsoft, Clorox and Verizon readily acknowledge that video is actually the single most powerful way to deliver real human feedback and access to how customers really feel about their brands, products and services at scale.
Using video to uncover real human stories can give context to data, get closer to what people think and help companies make informed, customer-centric decisions. It goes beyond the information and data provided by scores and scales by unlocking real insight and revealing the people behind the data. Ultimately, video gives automation the value and the staying power it needs and, in return, automation gives video the agility, scalability, and flexibility it needs to deliver robust, in-depth insights at speed.
GRIT Report on the future of sampling & surveys
So apart from automation, what else is new in the report? Well, pessimism about the future of sampling is a big concern as survey fatigue continues to cause problems in the industry. In fact, 39% of researchers expect sample quality to worsen over the next three years, with reasons behind this including the continued prevalence of lengthy surveys and questionnaires that fail to capture respondents’ interest.
Add that to changing behaviors, a shift in device usage and overburdened people receiving too many survey requests makes it clear that survey fatigue is understandably going to keep getting worse. People rarely sit at their desktops outside of work these days and they certainly don’t want to stare at a screen and complete a 30-minute survey anymore either. Until researchers make grander steps to address these issues, the same old data sample sets will continue to decrease in quality.
Perhaps we’re suffocating sample with a survey overload. There’s no real interest in the actual respondent experience, sample sizes are dwindling, and instead, it feels like a race to the bottom. And as consumer attention spans continue to get shorter, so too will the quality of their answers to traditional surveys. In summary, the traditional survey model becoming less effective – which means that researchers have to come up with new survey methods if they want to tackle the problem head-on and reduce survey fatigue and, in turn, improve sample quality.
What does survey fatigue mean for video?
As identified earlier, video is fast becoming a go-to tool for communicating socially and more importantly, when engaging with companies. It’s pretty easy to see why video market research is so popular: it’s fun, quick and easy to use and offers an engaging way for customers to share their experiences, helping them feel better connected to brands while fighting the ongoing battle of survey fatigue. Not only that, but it also empowers customers to show emotion and express themselves both visually and verbally.
Because there is such a broad range of video responses available, there are tons of ways for people to get involved and share their thoughts, from in-store shop-along videos, experience-based feedback videos, product testing videos, customer review videos and more. A typical, minute-long video response is six to eight times longer than a text-based response, which means that in just a matter of clicks companies can use video to see what their customers truly think, feel and do. And for the respondent, they can share more information in less time – reducing dropouts and fatigue. Finally, video puts a face to the feedback, overcoming the distrust many researchers have in the sample quality by ensuring respondents fit the profile they’re targeting.
In conclusion, if the GRIT report has accurately predicted that the market research industry wants to tackle survey fatigue and embrace automation to get to know what drives customer decisions, then video market research certainly ticks all the boxes. The secret to success lies in more profound insights and more advanced analysis – and that’s what video insight technology achieves.
Video market research engages and excites customers by encouraging them to share their thoughts and opinions in a fun and engaging way – and because they feel as if they are being listened to, it’s a great way to tackle survey fatigue. But that’s not all: because end-to-end video solutions such as Voxpopme are so efficient, video market research can also help businesses access deep insights at speed, helping them get to the bottom of what their customers are really thinking and making it easier to uncover more compelling stories than ever before.
Sound good? If you’d like to find out more about how video can transform your market research, connect with us using the Voxbot in the bottom right of your screen to book a free demo.
Video market research technology helps brands see the people behind the data and drive customer-centric decisions with engaging insight. Thanks to the automation of video feedback, in just a few clicks, you can now get closer to your customers than ever before by capturing, analyzing and sharing compelling, self-recorded customer stories.
Solutions like Voxpopme have turned video into a quick-turn, end-to-end solution – removing the painstaking processes associated with a traditional video market research project. And the benefits of adopting video-first research are clear. Self-recorded videos give consumers a unique way to share their stories with brands and allow brands to actively listen to and act upon authentic feedback.
By all accounts, video insight platforms are a natural evolution for video and pretty intuitive for modern researchers. However, as with any project, it always helps to have a checklist to hand when you’re in the thick of planning and delivery. So, If you’re thinking about conducting video market research using the latest tools and techniques, here are some key considerations to think about for your next project…
Your video market research project checklist
1. Planning your video market research project
First things first, you need to set a project outline and determine your research objectives. By setting clear goals from the very beginning of your project, you can make sure the insight you collect delivers the information you need to make customer-centric decisions. At this stage, you will also need to decide on whether you want to conduct agile qualitative research or collect video within quantitative studies – both of which can be achieved with Voxpopme. Whatever your discipline, you can choose where video best deliver on your objectives.
For quantitative researchers, video open-ends can be integrated directly into surveys, communities, websites and more – collecting videos from consumers in the platforms you already use like Zappi, SurveyGizmo, Qualtrics, Decipher, and FuelCycle. The speed of modern video market research makes it the perfect solution for pre, during and post quant work adding scalable qual into your quant. So if your quantitative research needs to provide a clearer picture of customers’ experiences, thoughts and feelings, consider where you can fit video into that process now that video opens can sit right within your studies.
For qualitative researchers, technology serves your studies up in a fraction of the time versus traditional approaches. For example, because consumers can share candid videos from wherever you need them, your shopalongs can now be self-recorded by consumers as they explore stores, or your IHUTs can show how they interact with your products in their home, without the influence of a moderator or videographer. Additionally, specialist apps like Moments even make it possible to run remote diary studies across days and weeks so you can probe further into your customers’ lives – and that’s just a few examples!
Still unsure which type of video is going to help you achieve your research goals? Contact you video insight vendor or Voxpopme to speak to a video insight expert and plan your project.
2. Question format and customizations
Next up, you need to determine the questions you want to ask and whether they are standalone video questions, whether you are adding video questions to an existing survey or whether you are building a new survey with video scripted in. Other things to think about include how many videos you would like to collect per respondent, as well as whether there are any specific tasks you need them to undertake and why.
Don’t forget to utilise open questions that encourage respondents to truly express themselves. And if you have specific tasks (shopalongs, product purchases, concepts to review) then make the pre-recording instructions as clear as possible so respondents know exactly what’s required from them.
3. Target audience
After deciding on questions and customisations, it’s time to decide who you would like to collect your video market research from. For example, if you are recruiting panelists, is there any screening criteria you need to use? What are the required quotas? And if you are looking to carry out your research in different markets, how many territories would you like to collect video in and how many video participants do you need per market? Will you need responses and transcripts to be translated?
Agile video research enables you to collect video from any audience, whether you want to integrate it into existing communities or surveys, set up new video-centric studies and recruit through panel providers or use panels like Voxpopme’s OnDemand community. There are tons of options, no matter how specific your audience – and by planning in advance, you can make sure everything runs smoothly once the research begins.
4. Timeline considerations
It’s also important to think about how quickly you require your video content. There isn’t a one-size-fits-all timeline for agile video projects as they tend to vary depending on your chosen audience, the volume of videos required, the type of video study you conduct and more. For instance, single market projects with ondemand video feedback communities, or gen-pop samples can complete in as little as 24 hours, with all videos collected and analysed. However, should you require a niche sample and plan to collect videos in multiple markets, you could be looking at a three to five day turnaround. Running a study in multiple languages? We would recommend you factor in time for translations (you’ll need to allow approximately 24 hours after collection for this) and if requested, factor in turnaround time for additional reporting or custom showreels. If you’re working to a tight deadline and are up against the clock, you could consider things such as using sentiment and theme coding to find insights faster as well as DIY editing so you can generate your own showreels in just a matter of clicks.
5. Budget considerations for your
Finally, you’ll need to consider budget for video responses plus possible recruitment fees for panel partners. If you’re running a multi-market study you should check translation costs for the required videos, and if you’ve ordered additional custom reports, showreels or analysis beyond the tools that are available in your chosen video insight platform you’ll need to factor in those costs too.
When thinking about the budget for the overall project you may also want to consider the cost of incentives – should you plan to use them. For example, if you’ve set a particularly extensive task, additional incentives can provide the appropriate compensation for the level effort required from your respondents. Take a moment prior to your project to work with your video insight vendor and evaluate whether or not additional incentives would help assist the successful delivery of your project.
Life today is hectic. No one has enough time – and in a world where we’re constantly connected, it’s pretty much impossible to ever switch off. If we’re not rushing from A to B then it’s the pressure of deadlines looming over us – and big data is always there threatening us with more information to digest than ever before too. It’s official: we live in a DRIP world that’s data rich and insight poor – and we’re struggling to keep our heads above water.
Despite all that, in this data-driven world, customer feedback has never been more important – which means that brands are frequently asking people to complete lengthy surveys in a desperate attempt to get to grips with what drives their decisions. The problem is that today’s customers just don’t have the time to fill these surveys in. Let’s face it, the last thing people want in addition to all the data and deadlines is to get caught up in a super long survey attempting to explain why they’re thinking and feeling the way they do.
The fact of the matter is no matter how many different question types you use, or how many pretty colors and fantastic images you add in to funk up your surveys, long surveys are boring and impersonal – and people just don’t have the time or the inclination to complete them. That means that survey fatigue is becoming a very real problem that’s growing – and fast.
Survey Fatigue & the trouble with traditional surveys
When it comes to getting to know what customers think, self-complete surveys have been the method of choice for over a decade. Our best attempt to go beyond scores and scales has been open-ended questions that ask for written responses, with the idea being that text-based responses give people the chance to elaborate on their thoughts and feelings and give more insights than scores alone. However, unfortunately, the reality is that text-based surveys are actually cumbersome, restrictive, and tricky to fill in – which only adds to mounting fatigue.
The stats say it all: open text boxes usually deliver responses of just three or four words and nearly always less than 50 characters – either that or they are skipped completely. As a result, questions go unanswered, participants become disengaged and survey fatigue continues to grow, and all the while researchers are still none the wiser on how they can understand, excite and engage their target audience. But there is another way…
Using video to engage with your audience
Video market research allows you to inject some personality into your surveys and gives your participants a new and exciting way to share information. We’ve already talked to you about the benefits of video from a business perspective – it can unlock insights, generate rich customer feedback and allow you to understand your customers better than ever before – and in this world of information overload, it will also get your executives excited too. But enough about you – what about your customers?
Basically, your customers are tired. They’re tired of deadlines, tired of data – and they’re absolutely sick and tired of completing exhaustive 30+ question surveys too. Instead, they are choosing to communicate with video more and more. Video is fast becoming a go-to tool for communicating socially and, more importantly, when engaging with companies as well.
A typical self-recorded video response is six to eight times longer than a text-based response, which means that in just a matter of clicks you can use video to see what your customers truly think, feel and do. It goes beyond standard survey questions by looking at things through a qualitative lens – offering much greater detail than rating scales such as Likert, Graphic or Semantive and scores such as NPS, CSAT, and PSAT alone.
Video is a quick and easy way for people to express themselves and for you to get a real understanding of what drives your customers – and crucially, it goes way beyond the confines of scores and rating scales by revealing the why behind the numbers. The proof is in the pudding, with video open-ends giving an average of 460 character responses and delivering 6x more content than text responses.
It’s pretty easy to see why video is so popular: it’s fun, quick and easy to use. Basically, it offers an engaging way for your customers to share their experiences and feel better connected to your brand, which in turn helps to fight survey fatigue. It also empowers your customers to show emotion and express themselves both visually and verbally, and because there is such a broad range of selfie style video responses available, there’s a variety of ways to get involved and share their thoughts. From in-store shopalongs and experience-based feedback videos to product testing and review videos, there are a number of ways that utilizing the power of video can help to minimize survey fatigue:
1. It makes your customers feel important
Video makes your customers feel like their opinions are important, that they are being listened to and that they are more than just data.
2. It’s easier for them to express themselves
Instead of answering closed questions or typing out tired text responses, video gives respondents a quick and easy way to express themselves.
3. It’s more personal than ticking boxes
Rather than just ticking boxes and getting lost in a sea of data, video adds a human aspect to communication that will excite and engage your respondents.
4. It’s fast
Instead of endlessly selecting options or typing out lengthy answers, video can cut down the questions and let your respondents get to the point fast, so it can easily fit in around their busy lives.
5. It can be done anytime, anywhere
Because respondents can record video anytime, anyplace, from their preferred digital device, it means that in-depth stories can be shared with ease and on the go
6. It allows them to show emotion
Body language and facial expressions allow your respondents to show how they really feel – something that just isn’t possible with traditional surveys.
7. It appeals to younger audiences
Younger audiences are keen to express their opinions across various media and making a quick reel reviewing a product or recounting an experience is much more appealing than a restrictive survey.
How it works
People are often nervous about how long video will take and how expensive it is – however today’s video market research allows you to capture hundreds of videos from your target audience in less than an hour. By just adding a simple block of code to whatever platform you’re using, you can integrate video-open ends and capture video insights alongside or after a survey to boost the impact of your results.
You can embed video open ends into any survey on any platform, website or app to enable video responses across any device – and you can even tap into specialist apps that make video the entire survey and give you access to video feedback communities. For example, Voxpopme’s on-demand community provides video feedback, in an instant, on a full range of market research studies via the Voxpopme app.
So whether you want to utilize video in a new or existing study, collect videos alongside or after a survey, in just a few clicks you can capture, analyze and share compelling customer stories across an array of quant and qual studies to boost the impact of your results. And once your videos have been recorded and uploaded, we’ll use time-coded, human transcription for outputs that are 97% accurate. Not only that, but Voxpopme software will then enable you to easily search themes, explore sentiment and create show-reels that excite your stakeholders. Find out more here.
To conclude, video is a powerful form of customerfeedback for any business, and adding Voxpopme video questions into ongoing studies can supercharge your surveys and provide a deeper understanding of your target audience, whilst setting up new surveys or app-based studies puts video right at the heart of your research. But it’s not just about the benefits for you (although there are a lot!) – it’s also about your customers. Basically, video gives your customers a voice. It makes them feel valued, appreciated, and that their opinion is important – and it’s fun too, which means it’s a great way to fight survey fatigue.
We live in a world that’s data rich and insight poor. We currently receive five times as much information as we did in 1986, and that number is only expected to climb as we become more bogged down with data. Why? Author Anthony ‘Tas’ Tasgal believes it’s because we live in an arithmocracy, a system where we worship numbers as gods and strongly believe in the safety of numbers and digits above all else. We’re afraid to make decisions without data, which is why we want facts – and perhaps why the art of storytelling in market research has been lost.
However, data alone isn’t enough. There’s no exact science to understanding people and what drives their behaviors; people aren’t predictable, controllable or susceptible to mathematical analysis and ration. So if researchers really want to start understanding why people behave the way they do, we need to abandon the worship of rationalism and start looking at the people behind the data. Basically, in order to unlock real, actionable insights, we need to start looking for the story behind the statistics.
Choosing insight over information
In this data-heavy age, it’s easy to think that the world revolves around numbers. But actually, it’s all about insight. Information (or data) is merely the fuel that drives the insight engine – and in order to access deep insights, the market research industry needs to make meaning the driving force behind brand and business development. Basically, in order to drive real change, we need to start telling stories and find the meaning behind the data once more.
Information is to be collected and insight is to be connected. Humans are not rational information processors, and that means we can’t use data to second-guess them. Instead, we need to focus on unlocking insights and understanding what drives behavior. Basically, instead of being in the business of data, the research world should be in the business of constructing and delivering meaning.
The problem is that market researchers are still nervous about losing their data duvet and going beyond data to make decisions. However, the fact is when it comes to communication, nothing beats truth, especially when it comes from a genuine, human source – and it is this level of in-depth insight that can allow researchers to unlock and share powerful customer stories that have a real impact.
The importance of storytelling in market research
Tasgal writes that if numbers numb us, then stories stir us. They translate information into emotion – and it is this storytelling in market research that makes your findings and conclusions infinitely more impactful and memorable. It’s all about finding the people behind your data and unlocking the story behind your statistics to engage and excite your executives and shake up your stakeholders.
Stories are patterns with meaning that the brain is hardwired to respond to. In fact, neurochemically there is even evidence that stories make us care, create empathy and build trust by producing oxytocin. Unfortunately, in today’s world researchers are often data rich but insight poor – however, all this needs to change. It’s the story that trumps information by turning information into emotion to create empathy, and empathy is the single biggest thing that can drive decisions. That means that in order to unlock real meaning, we need to start telling stories – and fast.
Using video to tell customer stories
And that’s exactly where video comes in. Agile video market research can help companies humanize their research and get to the bottom of what really drives customer decisions. It’s the single most powerful way to deliver real human feedback, giving researchers unbeatable access to how customers truly feel about different brands, products, and services. Basically, using video to uncover real human stories means you can give context to your data, get closer to what people think and make informed customer-centric decisions.
A typical, consumer-recorded video response is 6-8 times longer than a response to a text-based open-end. That means that in just a few clicks, video enables you to see what your customers truly think, feel and do by revealing real human responses. It goes beyond the information and data provided by scores and scales and instead unveils the true voice of the customer, unlocking real insights and telling the story behind the scores. The result? Multi-dimensional layers of insight and true customer stories that just aren’t achievable with scores and statistics alone.
Video cuts through the noise to obtain raw, unfiltered context so you can better understand your customers’ true feelings and build a deeper understanding of your consumers across your entire organization. Not only that, but it also increases customer closeness by allowing you to add depth, emotion, and authenticity to your data and build real human connections with your customers. That means instead of sharing yet more statistics with your stakeholders, you can humanize and add context to your data and deliver impactful, convincing and memorable stories that drive change.
Let’s take a look at how you can leverage storytelling in market research:
1. Bring your research to life
You can collect videos from consumers by adding open-ended video questions to your surveys or setting up projects with video at the heart of them through specialist apps and communities to bring a human voice to your data. Regardless of your approach, it takes just minutes to access rich, self-recorded video stories rather than bog-standard, five-word text responses.
2. See real human responses
Video feedback can help you see and understand genuine human interactions so you can get to the bottom of what your customers really think. And because they can share their thoughts in their own words and in their own environment, they’ll be empowered to share rich, insightful stories so you can make informed, customer-centric decisions.
3. Tell the story behind your results
Videos can be collected during or even after your research to boost the impact of your results. By supplementing your quant study with scalable video feedback you can add real weight to your research, find the golden thread and make everyone in your organization sit up and take notice by sharing impactful stories that drive change.
4. Find the insights you’re looking for
Once uploaded, videos can be searched by keyword or filtered by additional data such as age, gender or other customer tags so you can easily find the insights you need to tell customer stories. Additionally, because all videos are theme and sentiment coded you can easily discover what your customers are thinking and why – so in just a couple of clicks you can bring a topic to life and tell the story behind your scores.