I Built a Dual-Pool Adversarial Review System for AI Agents — And It Actually Works

AI code review has a problem: abstract roles produce generic feedback. "Saboteur" says "add error handling." "New Hire" says "this is confusing." Useful? Sometimes. Specific? Rarely.

I built something different: a review system that uses real engineers with searchable philosophies instead of abstract roles. Linus Torvalds doesn't say "consider error handling" — he says "eliminate the special case entirely." That's not a wording difference. That's a completely different action.

The Core Idea: Two Pools, Cross-Orchestrated

Fixed Pool (Convergence)          Random Pool (Divergence)
Digital-twin matched              Web-searched fresh each time
Stability & depth                 Surprise & blind-spot coverage
    │                                    │
    └────────── Cross-orchestrated ──────┘
              explore ←→ exploit

Fixed Pool

9 workers + 2 managers, curated to match the user's expertise, personality, and goals. Patty McCord (Netflix's former Chief Talent Officer) and Ed Catmull (Pixar's Braintrust creator) serve as managers who recruit teams per task instead of using a fixed template.

Random Pool

Fresh personas via web search each session. No preset list — the manager defines search keywords based on what the task needs. This is where the surprises come from.

How One Round Works

1. Manager drawn from the pool
2. Manager analyzes task → decides depth + required roles
3. Manager recruits specific workers (2 engineers + 1 product/designer)
4. Team reviews — each person MUST find at least 1 issue
5. Output: [Manager] picked [A,B,C]. Found N issues. Verdict: BLOCK/CONCERNS/CLEAN

Next round: new manager, keep at most 2 previous members.

Real Validation Data

I tested this on my own PR to alirezarezvani/claude-skills (18.7K stars):

• Round 1 (Fixed/McCord): 10 findings — structure, format, adoption gaps
• Round 2 (Fixed/Catmull): 8 findings — clarity, edge cases, UX
• Round 3 (Random/Spolsky+DuVander): 3 findings — positioning, first impression, output destination

The random pool found things both fixed-pool rounds completely missed. Fixed pool reviewers — who know me — were blind to how an outsider would perceive the skill.

Key Innovations vs Existing Systems

Open Source (MIT)

• github.com/YuhaoLin2005/dual-pool-review
• PR #866 — Installable skill

What I Learned

1. Real philosophers > abstract roles. "What would Torvalds say?" produces different code than "be more defensive."
2. Managers matter more than workers. McCord replacing one designer with another was the highest-leverage decision.
3. Random pools catch what fixed pools can't. If you only use people who know you, you have blind spots.
4. A system that reviews itself is essential. The review system found 14 issues with itself across 3 rounds.