From a Panel of Arguing AIs to a Working Behavior-Change Protocol: Notes on the Polemos Experiment
The setup
A few weeks ago I built Polemos, a small browser-based tool I've been calling a "Polymath Debate Bench." The idea is simple to state and more interesting to watch: you give it a case, a claim, or a question, pick a panel of archetypal personas, and let them argue. No moderator smoothing things over, no consensus-seeking — each persona reasons from its own logic and its own register, opening statements followed by continuous rebuttal, for as many rounds as you want. It runs entirely client-side; you bring your own model API key (Claude, Gemini, OpenRouter, or any OpenAI-compatible endpoint), and nothing passes through a server of mine.
I built it because I wanted something structurally different from the usual "ask one model, get one balanced-sounding answer" pattern. A single model asked to "consider both sides" tends to hedge toward the middle. A panel that's actually allowed to disagree — where a systems-thinker archetype and a behavioral-psychology archetype and a skeptic archetype are all pushing from different premises — surfaces friction that a single hedged answer smooths over. That friction is the point.
Turning debate into design
The specific run that produced the work I'm writing about here was a debate about designing a lifestyle/behavior-change program to run over WhatsApp — the kind of thing a lot of small health or fitness communities try to do with an open group chat and good intentions, and which usually decays into noise within a couple of weeks.
I ran the debate at polemos, let the panel go several rounds of rebuttal, and downloaded the transcript. That transcript — disagreements, half-conceded points, the places where one persona forced another to get more specific — was the raw material. I uploaded it to Claude and asked it to synthesize an actual, runnable protocol out of the argument, rather than just summarizing who "won."
What came out of it
What Claude produced wasn't a restatement of the debate — it was a structured protocol that resolved the tensions the panel had raised into concrete design decisions. A few things stood out to me:
- Small cohorts, split channels. Instead of one big group chat, the protocol calls for cohorts of 5–8 people split into a "Data Log" channel (structured prompts only) and a "Cheer" channel (peer encouragement). That separation of signal from noise was a direct answer to a tension the debate had surfaced about social comparison and diffusion of responsibility in large groups.
- If-then planning over open discussion. Commitments are collected privately, in a fill-in-the-blank format ("If it's 7am on a weekday, then I will walk for 10 minutes before checking my phone"), rather than announced publicly — avoiding the trap of performative commitments that look good in a group chat but don't predict behavior.
- Aggregate, not individual, feedback. Weekly updates report the group's numbers, never call out individuals — encouragement without shame-based comparison.
- Intention-to-treat measurement. This is the part I found most valuable. The protocol insists on reporting outcomes for everyone enrolled on Day 0, including people who stop responding, and it explicitly bans reporting results as "% of active participants." That's a direct fix for survivorship bias — the classic failure mode where a program looks like it works because only the people it worked for are still around to be counted.
- A real attrition plan. Specific triggers (3 days no reply, 14 days of low adherence, two missed weekly logs) map to specific, non-punitive re-engagement actions instead of silent dropout.
None of these ideas are individually novel — if-then planning implementation intentions and intention-to-treat analysis are both well established in behavioral science. What was interesting to me was watching them arrive as the resolution of an argument rather than as a list I'd have generated by prompting for "best practices." The adversarial structure forced trade-offs to be made explicit — cohort size versus intimacy, engagement metrics versus honest outcome metrics, encouragement versus honesty about who's dropping off — instead of getting flattened into a single unopinionated summary.
Why this is worth writing up
I think of this less as "I used AI to write a protocol" and more as a small case study in a different way of using these tools: instead of asking one model for an answer, stage a disagreement between several framings of a problem, and then have a separate synthesis step turn the disagreement into a decision. The debate does the work of stress-testing an idea; the synthesis step does the work of committing to specifics.
It's also, honestly, a fairly minimal pipeline — a static HTML page, a debate transcript, and one prompt to Claude — which is part of why I think it's worth sharing. The interesting part isn't the tooling, it's the workflow: disagreement as a design method.
If you want to see the tool itself, it's at polemos.html, and the running version I used for this is at polemos. The full protocol that came out of this particular debate — onboarding scripts, daily and weekly check-in templates, a moderator checklist, and the measurement plan described above — is attached alongside this post for anyone who wants to actually run something like it.
Protocol to a working app now?
