Case Study: Human-Centered AI
Enhancing Student Outcomes in AP Psychology
🎯 The Challenge: Bridging the Gap
In the 2024-25 school year, the AP Psychology course at DSST: College View faced two significant obstacles to student success.
1. Too Few Aligned Practice Opportunities ▶
Students needed extensive practice with Article Analysis Questions (AAQs), a key skill on the new AP Psychology exam. However, the official College Board curriculum offered few examples, and they often didn't match the specific module level content being taught.
2. Not Enough Timely Feedback ▶
While strategies like peer review and verbal coaching were used, it was impossible to give every student frequent, high-quality, personalized feedback without causing teacher burnout. The velocity of feedback couldn't keep up with student needs.
💡 The Human-Centered AI Solution
The hypothesis was simple: if we increase aligned at-bats and feedback velocity, student performance will rise. This wasn't about replacing good teaching; it was about unlocking new levels of support by intentionally leveraging AI.
Playlab Custom Bot
A custom AI bot was built using Playlab trained on AAQs and informed by the teacher's expertise in the curriculum to instantly generate AAQs tailored to current class content. This allowed the teacher to easily embed aligned practice into lessons.
Class Companion Feedback
Students submitted responses to Class Companion, a tool that delivered personalized, rigorous feedback aligned to AP rubrics and supported by custom teacher instructions, within seconds, augmenting teacher feedback.
“Before AI, this amount of feedback would’ve been completely unsustainable and would require sacrificing depth of planning or intellectual prep.”
📈 The Results: Significant & Equitable Growth
+50%
AP Pass Rate from 24 to 25
+26%
Avg. AP Exam Score from 24 to 25
+18%
Avg. AAQ Increase
+100%
PL1 AAQ Increase
AAQ Performance by Proficiency Level on Mock Exam
Hover over bars to see detailed scores.
A Note on Data Analysis with Manta AI ▶
Manta AI performed advanced analysis, normalizing data from practice exams that had different lengths and cut scores. This was powerful because in the past, practice exams have been highly predictive of AP Exam performance. However, this year students outperformed predictions because of inaccurate cut scores on the new test. Manta helped reveal the true growth despite these variables.
Detailed Performance Data
| Proficiency Band |
S1 Score | S2 Score | Gain | % Improve | Students |
|---|---|---|---|---|---|
| PL1 | 21.43% | 42.86% | +21.43 | +100.00% | 8 |
| PL2 | 40.82% | 51.43% | +10.61 | +26.00% | 15 |
| PL3 | 46.94% | 60.00% | +13.06 | +27.83% | 15 |
| PL4 | 65.55% | 68.07% | +2.52 | +3.85% | 17 |
| PL5 | 85.71% | 88.10% | +2.38 | +2.78% | 6 |
Only cohort in all of DSST with students at PL 2 (math PL) to pass the AP exam. Achieved a 27% pass rate with PL 2, compared to 0% across the network in AP Psychology.
Data analyzed with Manta AI
🔍 Key Insights & Implications
Lower-Proficiency Students Benefited Most
PL1-PL3 students showed dramatic improvements (26-100%), indicating the protocol was particularly effective for struggling learners and narrowed the achievement gap by nearly 30%.
Performance Variance Reduced
AAQ score variance decreased by 7.43%, indicating more uniform skill development. The gender performance gap also nearly disappeared by the second semester.
Success Despite Elevated Standards
Students improved across all components despite a 34% increase in the raw points required for passing in S2, validating the intervention's effectiveness.
🗣️ Student Reflection
“It felt like I could improve faster. I still wanted your feedback, but I didn’t have to wait.”
“It was really hard to get feedback right away. Sometimes I was scared to submit. Once I saw I was getting better I tried harder”
🤔 Human-Centered, AI-Leveraged
These results rest on several fundamentals:
-
1
We Value Human Thinking First
This work started with my expertise in prompting, the curriculum, and students' needs.
-
2
Leverage AI to Overcome Bottlenecks
Leveraging AI to overcome bottlenecks in challenges we have expertise in can be the fastest way to results. The challenge of multiple at-bats with immediate feedback isn't new; most solutions were just unsustainable, so few had viable solutions before AI.
Consider the following questions to explore the implications for your context.
Consider:
- How does this align with our DSST Position, Principles and Beliefs?
- How did this approach balance innovation with instructional priorities?
- Where could this go wrong if implemented poorly?
🔧 Investigate
Choose a solution to test drive (it will be quick!)
Playlab
Try generating an AAQ based on this unit:
Click Here: Playlab AAQ and School Teams AI Collaborative Explanation
- Unit: Cognition
- Topic: Spacing Effect
Trying this tool helps you see the idea of “scaling” writing aligned AAQs.
Class Companion
Check out feedback in action:
Click Here: Class Companion Demo and School Teams AI Collab Case Study
Reading this Case Study helps you see the process in action.