By Josaya Wasonga
If a tree falls in the forest, and there’s no one around to hear it, does it make a sound? Sounds like a trick question, right? But it’s not. It’s the age-old question that pits reality versus perception.
And that’s what Collective Action Goal 4 at the 2023 Kenya EdTech Summit sought to shine a spotlight on. The Summit was co-created by several EdTech stakeholders organized by EdTech East Africa, and sponsored by Mastercard Foundation and other sponsors.
For context’s sake, let’s rephrase the question: If EdTech innovations make seismic waves, and there’s no one to measure the impact, does it cause a change or disturbance in the atmosphere? Sound causes change or disturbance in the airwaves. “We don’t have to continue measuring impact, the way we do if it’s not working,” the facilitator, Grace Muthoni said. “That’s why we’re here; to learn from each other.”
Digital divide
Muthoni works at Metis Collective. At Metis, they amplify and connect with innovators and help them to innovate for systemic change. They bridge the learning gaps with free programmes. One is the six-month fellowship, where they personally empower education leaders to turn their innovative ideas into practical solutions.
Metis piloted Innovate Ed two months ago. It’s training that helps teachers to solve problems. Currently, they are on what they call The Empathise Stage. This is where they go to their students, ask questions, and do a root cause analysis.
Metis learnt that those closest to the learners lack tools, resources, and a community of partners. These individuals in close proximity – teachers, pedagogical leaders, parents, and caregivers – need to have their capacity strengthened to lead data-driven, inclusive digital learning. These players may be physically closest to the learners, but as long as they are worlds apart as long as the digital divide exists between them.
Purpose-driven conversations
The purpose of Collective Action Goal 4 was to help participants think through how they can measure progress toward the end goal of Inclusive digital learning. In the first collective conversation – in the EdTech Workshop that happened in August – attendees talked about the enablers and barriers in strengthening the capacity of those closest to the learners, and developed indicators.
The other purpose was to build on that conversation that players in the EdTech field need, to develop our teachers. They also need inclusive curriculum integration, digital tools, collaboration, and engaging students, among others. That’s one end of the equation.
The other end is measuring progress toward that goal of being inclusive in digital learning. And while we’re at it, it’s not just about strengthening the capacity, but realising that all the players are data-driven and can measure and share findings with the communities of practice.
“Let’s not leave measuring to the government and the big institutions,” Muthoni advised. “We can contribute to that conversation from wherever we are.”
Measuring tools
“How do you measure progress toward your end goal?” Muthoni asked the sixty-four-thousand-dollar question. “Which tools are you using and what are you measuring? It could be as a teacher you are measuring student engagement or learning outcomes.”
Silence.
“Or you don’t measure?” Muthoni quipped. “That’s also an answer. If I hear someone saying they’ve come to the realisation that they’re not measuring anything, that would be great.”
That takes us back to the age-old question. If we EdTech players are not measuring their impact, how will they know what they are doing right? Or wrong?
When it comes to measuring, it’s different strokes for different subjects. There is measuring progress and learning outcomes based on the 8-4-4 system and the CBC, the latter being more descriptive. There are measurement tools provided by the government, like the Early Grade Reading Assessment (EGRA) and Early Grade Mathematics Assessment (EGMA).
There are also classroom measurements that educators generate according to how they have taught the syllabus. At the end of the term, a teacher sets an exam based on what they have taught. This exam is moderated by other teachers who will say whether the teacher has followed the syllabus and met the objectives. The pass will be percentage or descriptive.
Ideally, the goals of any measuring tool ought to be threefold. One, measure impact. Two, strengthen the capacity of learners. And three, strengthen the capacity of those closest to the learners. However, more often than not, most measuring tools accomplish the first two goals but barely scratch the surface of the third.
“The tool should be holistic,” Muthoni pointed out. “You need to include all stakeholders. As a teacher you don’t, for instance, look at data in isolation. The parents and caregivers also contribute to that goal.”
Actionable insights
“We should come up with actionable insights. What metrics will I start measuring?” Muthoni asked. “If you are an EdTech-preneur, what metrics are you going to use to measure your progress?”
“We measure revenue,” a participant said. “It’s important because we’re a for-profit organisation. We spend a lot of money to do content and so want to have a return on investment and see if it’s worth it.”
A player like Ntemata puts emphasis on the teacher who, in their opinion, is the driver of learning. They measure the quality and wellness of the teacher. They measure how a teacher enhances learning in a classroom. Their assessment tool is supposed to support the teacher to have 100% engagement with the learner. They are of the opinion that if they can strengthen the capacity of the teacher, they will have better learning outcomes.
Actionable insights are not just the preserve of some stakeholders in the ecosystem but for all and sundry. A pedagogical leader or EdTech-preneur can, for instance, look for the usage of apps and study the trends on the backend to arrive at actionable insights.
Learning metrics
There are different learning metrics for different learning scenarios. Let’s take the case study of Ziwani Primary School. Its metrics are professional development, inclusive pedagogy in curriculum integration, digital tools and infrastructure, collaboration and community building, student engagement, learning outcomes, and parental and caregiver empowerment.
Ziwani Primary School has implemented inclusive education for children with diverse needs. Emily is a fourth-grade student with autism and has shown remarkable progress in her social skills, and academic performance since joining the inclusive programme. She has made friends become more engaged in class activities and is meeting academic benchmarks. However, the school’s focus on Emily’s success has led to less attention given to other students with diverse needs.
“If you’re working in Ziwani Primary School – as a teacher, pedagogical leader, policymaker, EdTech-preneur – what metrics would you use to measure success?” Muthoni asked. “How do you ensure the metrics are effective in considering the needs of all students with diverse needs, not just Emily?”
“The key metric is inclusive pedagogy and curriculum integration,” one attendee opined. “This is because some of the learners are being neglected. They’re not fully engaged. The indicators in this case could be diversity in curriculum materials. This is because the other learners do not deal with the same challenges as Emily.”
“The other metric is learning outcomes,” another attendee chimed in. “Because if you’re feeling neglected, which means they’re not at the level they’re supposed to be at. They are not achieving the learning outcomes. One of the indicators would be competency in teaching.”
Learning metrics have a lot to do with measuring impacts to strengthen those closest to the learners. For starters, as Valamis states: “By gathering data throughout the process and carefully analyzing real-life results, learning metrics offer a way of accurately assessing the strengths and weaknesses of a particular course to determine whether it achieved its goal.”
Measuring the impact of a fallen tree
We’re back to para one. If a tree falls in the forest, and there’s no one around to hear it, does it make a sound? Well, there’s more to this than meets the eye. The tree falls on other flora and fauna. The impact of a fallen tree tremendously affects the surrounding habitat.
But then again, if there’s no one around to – in the context of EdTech – measure its impact, it would be akin to a lost opportunity. Why? The capacity of other forms of life that are closest to the tree – teachers, pedagogical leaders, parents, and caregivers – cannot be effectively strengthened.
