Words you will meet
- CSF
- Central Square Foundation, the Indian education nonprofit where I worked from 2021 to 2025. It works with state governments on foundational learning.
- Mentor
- A government teacher-educator who visits schools, watches lessons and advises teachers. Each mentor looks after many schools.
- Teaching steps
- The moves a lesson expects, set out in the teacher handbooks. An oral language lesson, for example, expects the teacher to discuss a picture or story, ask closed and open questions, and set a short writing task.
- Non-evaluative feedback
- Feedback that describes what happened in the lesson and what to try, with no score, grade or ranking.
- Speech-to-text
- Software that turns recorded speech into written text.
- Language model
- A program that reads and writes language. Here it reads the lesson's text and judges which teaching steps happened.
- False positive, false negative
- The two ways the coach can be wrong. A false positive says a teacher did a step she skipped. A false negative says she skipped a step she did.
- Accuracy, precision, recall, F1
- Standard scores for how often a model is right. Precision asks how many of its yes answers were correct; recall asks how many of the real yeses it found; F1 balances the two.
- Proof of concept, pilot
- A proof of concept is a small, short test of whether something works at all. A pilot runs the product with real users for weeks, to see whether anything changes.
- Foundational learning
- Reading, writing and basic maths in the first years of school. NIPUN Bharat is India's national mission for it.
Chapter 1 · The problem
A teacher got 3 to 7 mentor visits a year, so feedback on her teaching was rare
Picture a Class 1 teacher in a government school, teaching a Hindi lesson from her handbook. She was trained on the steps the lesson expects. Whether she used them today, nobody will tell her. In the programs I worked on, a mentor might not reach her classroom all year.
From 2021 at Central Square Foundation (CSF), I helped state governments run programs for foundational learning. They had mentors on paper, and the same three failures in practice.
Exhibit 1
Coaching failed in three places, and each one left teachers without useful feedback
| Where it failed | What I saw | What it meant for a teacher |
|---|---|---|
| Span | 3 to 7 visits per teacher a year; about 5 to 15 hours of coaching per school a year | Few repeat visits; some teachers had none |
| Judgment | Many mentors could not tell what was going well or badly in a lesson | A visit did not find the step that mattered |
| Feedback | When it came, it was rarely specific or shared in a way a teacher could use | She heard a verdict, with nothing to try next time |
Visits felt like inspection, so teachers closed up
The casebook that later published this work describes the same thing: classroom feedback that is "often delayed, generic, or focused on compliance", and teachers anxious about being observed. A teacher who expects to be judged hears even good advice as a mark against her. Any tool we built had to feel like a private coach, never another inspector.
Sources: CSF state programs, 2021 to 2025; IndiaAI casebook 2026, case 29, p. 57.
Chapter 2 · Choosing the problem
Asked to set up AI work from a blank page, I mapped the whole learning system before choosing teacher coaching
CSF asked me to decide what it should do in AI for education, and to found its AI practice to do it. Until then its tech team mostly funded other companies' products through an accelerator, rather than building its own. There was no brief beyond that.
I ranked every part of the system on the same three questions
I went through each part of foundational learning: teachers, students' learning, assessment, monitoring and mentoring. For each I wrote the problem, what already existed, and where products were missing. Then I ranked the problems on the impact they could have, whether the technology could do the job, and how few products already tried.
Teacher coaching came out on top. Teaching is where children's learning changes, and the failures in chapter 1 were ones AI could plausibly reach. A coach on the teacher's own phone does not wait for a mentor's visit.
Nothing close existed in India
To my knowledge, as of early 2025, there was no AI teacher coach in India. The closest tool I found was in the United States, and I used it myself to feel what the experience was like.
Exhibit 2
The closest tool measured talk time; our teachers needed feedback on the steps they were trained to teach
| Tool | What it does | What our teachers needed instead |
|---|---|---|
| A mentor's classroom visit | A trained person watches and advises | The same kind of advice, without waiting a year |
| TeachFX (US) | Measures how much the teacher talks and how much students talk | Feedback on the specific steps of the lesson in their handbook |
| What we set out to build | Records the lesson, checks the expected steps, says what was done and what to try |
Talk time is a useful signal. But a teacher told how much of the lesson she spent talking still does not know which step to add tomorrow.
CSF's first in-house product, built with a partner's engineers
Sakhee became CSF's first in-house product. A partner edtech organization's engineers and part-time designer built the app with us, and I led the product end to end, from user research to its Play Store launch.
Sources: CSF landscape analysis and benchmarking, early 2025; the partner's project notes, 2025.
Chapter 3 · Audio only
Recording sound, not video, made the coach usable in a real classroom and cost us the silent teaching steps
Video looked like the obvious choice, because a mentor watches a lesson. I chose audio. The first piece of design work was then to find out which teaching steps a microphone can actually hear.
Record sound, not video
- The call
- The teacher records the lesson's audio on her own phone.
- What I gave up
- Steps that make no sound, such as board work or children reading quietly.
- Why
- Video of a class full of children raised privacy questions, and a phone propped on a desk sees only part of the room. Audio works on any phone and needs little data.
Evidence · Chosen during technical scoping, early 2025.
We went through the handbooks step by step
With CSF's pedagogy team I went through the teacher handbooks lesson by lesson. For each expected step we agreed two things: whether it can be inferred from sound, and how deep the feedback on it should go, which we set at two levels.
Exhibit 3
Audio gave up the silent steps in exchange for privacy and a phone every teacher already owns
| Video | Audio | |
|---|---|---|
| Children's privacy | Faces of children on record | No images; the coach processes no student identity data |
| Where the phone goes | Must be placed to see the room, and sees half of it | Anywhere the teacher's voice reaches |
| Data | Large files on mobile data | Small files, uploaded when there is a signal |
| What it misses | Whatever is out of frame | Silent steps, such as board work and quiet reading |
Sources: CSF scoping work, early 2025; IndiaAI casebook 2026, case 29, p. 58.
Chapter 4 · What we built
A teacher picks the period, records it, and gets specific feedback on each step with a short video to try
The pilot app was for Hindi literacy in Classes 1 and 2. The lessons it covered come in three kinds of period: oral language, decoding and reading. The teacher picks her class and period, reads three short tips for it, and records.
Exhibit 4
Five steps from lesson to feedback, and the teacher does only the first three
- 01
Pick the class and period
- 02
Record the lesson on the phone
- 03
Submit; it uploads when online
- 04
The model checks the expected steps
- 05
Feedback on each step, with a short video
The feedback describes the lesson and never grades the teacher
The report opens with up to three things that went well and up to three that need work, each gap with a short video on how to do it, made with CSF's pedagogy team. Below that, every expected step is marked done or not done. A step marked done plays the moment from her own recording that shows it, so she can hear why the coach gave her credit.
Describe the lesson, never grade the teacher
- The call
- Describe what happened and link each gap to a short video. No scores.
- What I gave up
- A simple number that officials could report upwards.
- Why
- Teachers already felt observation as inspection. A score would have turned the coach into one more inspector, and she would stop recording.
Evidence · Chapter 1; casebook, p. 58.
After reading the report she gives it a thumbs up or down. A weekly streak and the share of steps she used over her last sessions sit on a separate screen.
It was built for a budget phone and a weak signal
The app works in Hindi and English. Recordings stay on the phone, uploads retry on their own when the signal returns, and a recording survives the app being closed by accident. The coach makes no automated decisions about teachers, processes no student identity data, and lets teachers delete their own data.
Sources: CSF product requirements, screens and prototypes, 2025; the partner's project notes on the pilot app, 2025; IndiaAI casebook 2026, case 29, p. 58.
Chapter 5 · The error that matters
Teachers quit over being told they missed a step they did, and forgave the opposite error
I set up the evaluation before choosing a model. But the most useful finding came from watching teachers read their feedback, first in the proofs of concept and then across the 50-teacher pilot, and it changed what we asked the model to do.
We compared models against recordings labeled by people
For every teaching step, the evaluation compared the model's answer with what a person heard in the same recording, and scored accuracy, precision, recall and F1. Against it we tried a two-stage pipeline, Whisper speech-to-text into GPT-4o mini; Gemini listening to the audio directly; several ways of prompting; RAG on lesson plans, so the model checks each step against the plan; and fine-tuning on our early data.
The two errors felt completely different to teachers
Exhibit 5
Most of the first test's errors were the kind teachers resented
| The coach said | What a person heard | Judgments | How teachers reacted |
|---|---|---|---|
| "You did it" | She did it | 18 | |
| "You missed it" (false negative) | She did it | 15 | Quit over it |
| "You did it" (false positive) | She skipped it | 2 | Forgave it |
| "You missed it" | She skipped it | 8 |
In this first test the coach missed 15 of the 33 steps teachers actually did. The pass mark for the next stage put recall first: at least 80% recall at 80% precision. Through model evaluation and RAG on lesson plans, I raised the coach's recall from 70% to 95%. On a partial check in the second proof of concept, 2 of 42 judgments were false negatives (my count from a working sheet).
A teacher told she failed at something she did well has no reason to believe the next point either. So we stopped chasing overall accuracy. We tuned the model and the prompts to avoid false negatives, and accepted some false credit in exchange.
Optimize for the error teachers feel
- The call
- Tune the model to avoid telling a teacher she missed a step she did.
- What I gave up
- Some overall accuracy, and more steps credited that did not happen.
- Why
- A false negative breaks trust in every other piece of feedback. Teachers forgave the opposite mistake.
Evidence · Teachers' reactions in the proofs of concept, which held across the 50-teacher pilot.
The trade was deliberate: a coach that leans toward giving credit will sometimes praise a step that never happened. Chapter 8 sets out how I would check its counts, with people scoring a sample of recordings.
Sources: CSF evaluation framework and model experiments, early 2025; first proof of concept, early 2025.
Chapter 6 · Stage gates
Each phase had to earn the next, and the first showed the coach must work offline
Usage and accuracy had to be proven before we could ask whether teaching changed, and only then which way to run it at scale. So I planned four phases, each with its own definition of success.
Exhibit 6
Four phases, each answering one question before the next could start
| Phase | Size | The question it had to answer | Status |
|---|---|---|---|
| Proof of concept 1 | Small | Do teachers use it? Is the model accurate enough? | Piloted |
| Proof of concept 2 | Small, with field visits | The same, plus: where do teachers get stuck? | Piloted |
| Pilot 1 | 50 teachers, 8 weeks | Does teaching practice change? | Piloted |
| Pilot 2 | 150 or more teachers | Which of three ways of running it works best? | Designed |
I also decided, feature by feature, what belonged in which phase. That included how the app reached phones: builds installed by hand for the proofs of concept, and the Play Store for the pilot.
Remote schools showed the coach had to work offline
In remote schools with no signal, a recording from the first proof of concept could not upload and no feedback came back. These were the schools that most needed a coach. So we added offline use for rural schools: the pilot app records offline. The audio stays on the phone, uploads when the teacher is back online, and the feedback arrives then.
In the field we watched where teachers got stuck, button by button
During the second proof of concept I went to schools with the team. I wrote the observation and interview guides and ran live usability sessions. I also set up event tracking, click heatmaps and session recordings, so we could see which screen and which button stopped a teacher, and fix it before the pilot.
Sources: CSF stage-gate plan and feature priorities; field research guides and usability sessions in the second proof of concept; the partner's project notes, 2025.
Chapter 7 · The pilot
In an eight-week pilot, 50 teachers used 53% of key teaching steps at first and 74% by their last recording
The pilot ran for eight weeks with 50 government school teachers in three blocks of one district, with approval from the district education authorities. Teachers recorded about one lesson a week, each about 14 minutes long.
Before it started, I had set its pass marks: at least 30% of teachers using more key steps from first recording to last, and at least 60% completing six or more feedback cycles. Both were met.
Exhibit 7
Teachers used 53% of key teaching steps in their first recording and 74% in their last
- First recording53%
- Last recording74%
Exhibit 8
Both pilot targets were met, and most teachers found the feedback accurate
| What was measured | Result | Kind of evidence |
|---|---|---|
| Pilot teachers who used it every week for eight weeks (target: 60% or more completing six or more cycles) | 45 of 50 | Met |
| Teachers who showed overall improvement (target 30% or more) | 74% of 50 | Met |
| Found the feedback accurate and helpful | 87% | Self-report |
| Felt comfortable recording their own lessons | 68% | Self-report |
| Would recommend it to another teacher | 72% | Self-report |
Field reports from teachers' phones set three product rules
When the pilot app reached teachers' phones, CSF's field team reported issues that had nothing to do with the model. On some Android phones a finished recording showed a length of 00:00 and could not be saved or analyzed. The app sometimes opened to a blank white screen after being closed. A warning said the phone was out of storage when it was not, uploads could hang with no way out, and going back and forth while setting up a class could create the same class twice.
The partner's engineers traced and fixed each one. The product rules that came out of it are simple. A recording is never lost. No upload stays stuck: anything that fails moves to a retry button. A class is created only once its details are complete.
Sources: IndiaAI casebook 2026, case study 29, pp. 57-58; CSF research framework sheet, 2025; the partner's project notes on the pilot app and field reports, 2025.
Chapter 8 · What comes next
The next test adds a comparison group, people scoring recordings, and three ways to run the coach
The pilot met both its targets, and the share of key steps teachers used rose from 53% to 74%. Scaling to 500,000 teachers was approved. The next step is to measure how much of that change the coach accounts for, and how best to run it.
A randomly chosen comparison group
I would run the next pilot with a randomly chosen group of teachers who do not use the coach. Comparing their first and last recordings with those of teachers who do shows how much of the change the coach accounts for.
People scoring a sample of recordings
People would score a sample of recordings step by step, alongside the model, as in the first evaluation (chapter 5). That checks the coach's own count of steps used, and shows whether the trade toward giving credit needs retuning.
Three ways of running the coach
Pilot 2, for 150 or more teachers, is designed to compare three ways of running the coach, such as teachers using it alone against teachers with a mentor's support (Exhibit 6). The casebook says larger pilots are planned over the next two years.
Sources: CSF Pilot 2 design, 2025; IndiaAI casebook 2026, case study 29, p. 58.
Reflections
Both of the biggest changes to the coach came from watching teachers use it in their own schools
What worked
Letting teachers, more than the model's scores, decide which error to avoid. The false-negative call came from watching people read their feedback, and it is the decision I would defend first.
What I learned
Put offline recording in the first version. The schools with no signal are the ones that most need a coach, and the first proof of concept showed it.
What I'd do next
Run the second pilot with a group of teachers who do not use the coach, chosen at random, and have people score a sample of recordings, so that a change in teaching can be traced to the coach and measured independently of it.