Dost and 60 Decibels team up to listen to families
In the early days of Dost, we asked families what they thought of a Dost phonecast about sorting vegetables into different groups with their little ones. “But my son doesn’t even eat the vegetables! How do I get him to do that?” We realized that learning key early childhood concepts happens in tandem with those everyday parenting to-do’s - feeding, bathing, sleeping, and cuddling. This insight early in our product development became a fundamental ethos for Dost: Listen to caregivers and design for them.
It sounds obvious, but really listening to caregivers is the key, not-so-secret ingredient to our product design process. We wanted an outside perspective from experts to support our impact measurement strategy. We were primarily curious to understand how caregivers valued the Dost program and how it increased parental awareness and confidence in their ability to support their child’s early development.
For this particular piece of work, we partnered with the impact measurement organization 60 Decibels (named as such for the average volume of a human conversation). Their emphasis on customer-centric impact data as well as unprompted qualitative insights was a major motivation to work together: we wanted to hear directly from our users in their own words - at scale.
A first for us both - 60 Decibels’ study methodology for Dost
The 60dB team understood our goals immediately. Together, we decided the aim of this larger study was to understand priority impact areas by conducting in-depth interviews with a representative sample of Dost beneficiaries. This was the first time Dost was able to conduct a study with over 370 total respondents, of which 265 participated at both baseline and endline. We also wanted to develop an externally-verified benchmark of data (e.g. Net promoter score, % of caregivers feeling more confident in their parenting) to compare future performance against as a neutral measure of Dost’s current impact.
Building on their proprietary Lean Data method, 60dB co-designed two research approaches in partnership with Dost:
1. Baseline and Follow-up interviews
We interviewed caregivers when they first started the Dost course and then again 12 weeks later - an approach 60dB had never used with an education company / non-profit before. Typically, 60dB conducts one-time studies on the perception of a product’s impact in general. With a baseline and follow-up method, Dost and 60dB were able to collect robust and quantifiable impact data about how users' confidence changed over time.
2. Control and Treatment Groups
These two groups offered a comparison point for parenting techniques to understand the specific impact Dost’s course was having on participants versus non-participants. Dost had never done this before in a systematic manner.
Study insights and results
60dB found that those who had taken the Dost course were more likely to experience improvements in their parenting across several impact metrics. After 12 weeks, most course participants said they felt:
1. Dost’s course was a unique offering, providing access to something that they had not experienced before:
- 55% of participants live in poverty, according to the Poverty Probability Index
- 99% of participants reported that they had not had access to a parenting course before
- 88% highlighted there being no good alternatives to receiving the lessons provided by Dost’s course
Dost’s course improved participants parenting over time, leaving caregivers
- more confident (91% of treatment vs. 77% of control),
- less stressed (89% of treatment vs. 74% of control) and
- more knowledgeable (94% of treatment vs. 77% of control) as a parent after participating in the course.
Unprompted, open-ended questions provided powerful and surprising insights as well (see table below). When asked what the most meaningful area of improvement they’d seen in their relationship with their child after using Dost, caregivers commonly reported "keeping my child busy doing things that will benefit them."
Dost’s support was perfectly summarised by a 40 year old participant:
By the training of Dost Education, we now know the ways by which we can give our children more love as well as care about them better than before.”
The study provided us with some valuable feedback on how to improve our offering, one of which surprised us: Male caregivers were much more likely to report high levels of overall satisfaction compared with female participants, raising an important question for Dost around how caregivers of opposite genders experience the course.
Taking new insights into action
Dost was able to take action on some of these findings right away. Here are just two examples:
1. We adjusted our data collection process to include caregiver type and gender.
This will enable us to dig deeper into why and how different genders value and perceive the program and what we should do to respond to this. The 60dB results made us realize how well the content was received by fathers, and what an incredible opportunity that represents. Hot off the presses, we are now exploring options to directly reach out to fathers, like partnering with a Dairy co-op.
2. We now use the 60dB process on a small scale as part of our regular monitoring.
As soon as we launch in a new region or with a new partner, we do short customer surveys up front to understand how users in completely new geographies engage with Dost. In one state, a baseline survey helped us identify that 30% of the time, the person listening to the Dost phonecasts was not the primary caregiver due to phone sharing at home. We used that information to tweak our onboarding process to explicitly ask for the primary caregiver during sign up. Ongoing surveys will help us see if this tweak works, or if more experiments are needed.
A milestone in Dost's evaluation strategy
Measuring impact is multi-faceted when it comes to caregiver support and early learning, so Dost is building new tools and partnerships as part of its broader evaluation strategy. The 60dB study was a milestone to be celebrated: it allowed us to validate our overall product goals and value to users; and it gave the organization the greenlight to invest in other studies that will gather observational data about caregivers and children over time. Perhaps most importantly, it helped us embed a measurement process into every new partnership - a process that aligns with our approach to listen directly to caregivers.