teacher_agent_dev/FIXES_SUMMARY.md

Summary of Fixes for Accuracy Drop Issues

Issues Identified

1. Accuracy Drops at End ❌

Root Causes:

1. Evaluation uses NEW tasks each iteration → Variance and inconsistency

   • Line 171-175: Generates new tasks on-the-fly for general_accuracy
   • Different tasks each time = different difficulty/variance

2. Forgetting rate too aggressive for 500 iterations

   • Forgetting rate = 0.05
   • After 500 time units: retention = exp(-0.05 * 500) ≈ 0.0
   • All skills completely forgotten by iteration 500!

3. Evaluation timing: Evaluation happens after time advance, but we log before - this is actually OK

Fix:

• ✅ Use FIXED eval sets generated once at start
• ✅ Reduce forgetting rate from 0.05 to 0.01 (5x slower forgetting)
• ✅ Evaluation happens BEFORE time advance (accurate snapshot)

2. Accuracy Calculation Method

Current Method:

• Uses student.evaluate(eval_tasks) which samples answers stochastically
• Accounts for forgetting correctly
• BUT: Uses different tasks each time

Problems:

• Stochastic variance (random sampling)
• Inconsistent eval sets (regenerated each time)
• Small eval sets (10-15 tasks) = high variance

Better Method:

• ✅ FIXED eval sets generated once
• ✅ Same tasks used throughout = consistent measurement
• ✅ Larger eval sets (15+ tasks) for stability

Alternative (for future):

• Use expected accuracy = mean(prob_correct) instead of sampling
• Removes stochastic variance

3. Mock vs Real Components

Current Mock Components:

• ✅ Mock Student: Captures learning + forgetting well
• ✅ Mock Task Generator: Simple but functional
• ❌ Simplified learning model
• ❌ Limited task diversity

Real Components (MentorFlow):

• Real Student: Full PPO with neural network
• Real Task Generator: Procedural generation, 15 families

Will Real Components Be Better? YES:

1. Real PPO Student:

   • Can learn complex patterns
   • Better generalization
   • More realistic learning curves
   • But: Slower to train

2. Real Task Generator:

   • More diverse tasks
   • Procedural generation = infinite variety
   • Better tests generalization

3. Teacher Agent Algorithm:

   • UCB algorithm will work the same
   • Should perform even better with real components
   • More realistic reward signals

Expected Improvement:

• Teacher should learn better curriculum
• Student should achieve higher accuracy
• More realistic forgetting patterns (if implemented)

Applied Fixes

✅ Fixed evaluation to use FIXED eval sets ✅ Reduced forgetting rate from 0.05 → 0.01 ✅ Evaluation happens BEFORE time advance ✅ All strategies use consistent eval sets

Remaining Considerations

1. Forgetting Model: Could use more sophisticated model (spaced repetition optimization)
2. Evaluation Method: Could use expected accuracy instead of sampling
3. Eval Set Size: Could increase for more stability (currently 15 tasks, could be 50-100)
4. Time Reset: Could periodically reset time to prevent complete forgetting in long training