ARIS — Autonomous Research via Adversarial Multi-Agent Collaboration

Let Claude Code do research while you sleep. Wake up to find your paper scored, weaknesses identified, experiments run, and narrative rewritten — autonomously. Repo: github.com/wanshuiyin/Auto-claude-code-research-in-sleep.

TL;DR

ARIS is a collection of 74 composable Claude Code skills that orchestrate cross-model collaboration: Claude Code drives the research (reads files, writes code, deploys experiments) while an external LLM (GPT-5.4 / 5.5 via Codex MCP) acts as a critical reviewer. The two models disagree, debate, and force each other to do better — adversarial, not self-play.

Seven workflows (W1 / W1.5 / W2 / W3 / W4 / W5 / W6) compose into a full research lifecycle: idea discovery → experiment bridge → auto-review → paper writing → rebuttal → resubmit → conference talk. Tested end-to-end on real ICLR/NeurIPS submissions. Score progression on a real overnight run: 5/10 → 7.5/10 with 20+ GPU experiments.

The Problem

Every ML researcher reading this knows the rhythm:

1. Spend 3 weeks reading 80 papers.
2. Brainstorm 12 ideas, kill 9 in your head, fail to validate the remaining 3 quickly.
3. Pick one, lose a week to a bug, miss the GPU window.
4. Submit, get a 5/10 review with "lacks ablation against XYZ".
5. Rebuttal week is 72 hours; you have 2 days of teaching duty.

The bottleneck isn't ideas. It's the end-to-end orchestration between literature → ideation → experiments → writing → rebuttal. AI can compress every individual step, but the integration is fragile — and worse, a single model reviewing its own work falls into local minima.

Core Architecture

The system is, in one sentence:

$$ \text{Research} = \arg\max_{\theta}\; \mathbb{E}_{x \sim \mathcal{D}_{\text{ideas}}}\bigl[\, U_{\text{exec}}(\theta\,;\,x) - \lambda \cdot R_{\text{review}}(\theta\,;\,x) \,\bigr] $$

where $U_{\text{exec}}$ is the utility of an executor model writing code / running experiments, and $R_{\text{review}}$ is an adversarial regularizer from a cross-family reviewer that penalizes overclaims, fabricated citations, unjustified theorem extensions, and self-flattery. The regularizer is non-differentiable — it's a fresh LLM thread reading the artifact cold.

The reviewer-independence protocol

Every review round uses a fresh codex thread. We never use codex-reply to continue a previous review conversation. This is a hard rule, learned from a real NeurIPS run where codex-reply chains inflated scores from 3/10 → 8/10 through narrative accumulation (the reviewer started defending its earlier criticism instead of evaluating the current artifact). The protocol is codified at skills/shared-references/reviewer-independence.md.

┌──────────────────────┐
│  ARIS — execution    │     ┌───────────────────────┐
│  (Claude Code)       │────▶│  Codex MCP (GPT-5.5)  │
│  — reads files       │     │  — reads paper cold   │
│  — writes code       │     │  — fresh thread       │
│  — deploys to GPU    │     │  — scores 1-10        │
└──────────────────────┘     │  — suggests fixes     │
         ▲                   └───────────────────────┘
         │                              │
         │                              ▼
         │              ┌─────────────────────────┐
         └──────────────│  weakness list (.md)    │
                        │  fix list (with budget) │
                        └─────────────────────────┘

The Workflows

Workflow 1 — Idea Discovery & Method Refinement

"I have a research direction. What should I actually work on?"

/idea-discovery takes a vague research direction and outputs a ranked, pilot-validated proposal:

1. 📚 Survey — multi-source literature search (Zotero / Obsidian / arXiv / Semantic Scholar / DeepXiv / Exa) builds a landscape map
2. 🧠 Brainstorm — GPT-5.4 xhigh generates 8-12 concrete ideas anchored to identified gaps
3. 🔍 Novelty check — each top idea cross-checked against arXiv + DBLP for prior work; failed ideas killed early
4. 🧪 Pilot — 2-3 surviving ideas get 1-2-hour single-GPU pilot runs (a real signal, not just LLM opinion)
5. 🏆 Refine — top pilot result fed to /research-refine which anchors the problem, tightens the method, and emits an experiment plan ready for Workflow 1.5

/idea-discovery "factorized gap in discrete diffusion LMs" \
    --- effort: max \
    --- sources: zotero, web, deepxiv

Workflow 1.5 — Experiment Bridge

"I have a plan. Make it run."

/experiment-bridge closes the gap between paper plan and running code:

1. 📋 Read EXPERIMENT_PLAN.md (from W1 or hand-written)
2. 💻 Implement experiment scripts — reuses your existing codebase, adds argparse / logging / seeds where missing
3. 🔬 GPT-5.4 code review — fresh-thread cross-model review of the generated code BEFORE any GPU time is spent (catches ~80% of bugs that would otherwise burn 8-GPU-hour runs)
4. ✅ Sanity check — smallest config runs first; checks for OOM, NaN, runtime errors
5. 🚀 Deploy — SSH to your GPU server (per CLAUDE.md), launch in screen, capture stdout/stderr
6. 📊 Collect — /monitor-experiment polls until completion, fetches results, formats for downstream skills

/experiment-bridge --- base repo: https://github.com/anthropic-experimental/some-baseline

Workflow 2 — Auto Review Loop

"Review my paper, fix what's wrong, repeat until it passes."

/auto-review-loop is the most-cited workflow. Run it the night before a deadline; wake up to a polished paper.

1. 🔍 Deep review — GPT-5.4 xhigh reviews the paper, identifies weaknesses (severity tagged)
2. 🩹 Fix — Claude implements the fixes (rewrite, add baselines, run experiments); skips any experiment > 4 GPU-hours, flags for manual follow-up
3. 📊 Re-evaluate — collect results, update paper, feed back to the reviewer (fresh thread)
4. 🔁 Repeat — until score ≥ POSITIVE_THRESHOLD (default 6/10) or MAX_ROUNDS (default 4); if the context window fills mid-loop, auto-resume from REVIEW_STATE.json

/auto-review-loop "focus on Section 3-5, our CRF results are weak" \
    --- difficulty: nightmare \
    --- effort: max

The difficulty: nightmare flag lets GPT-5.4 read your repo directly via codex exec — Claude can't filter what it sees. Maximum stress test before submission.

Workflow 3 — Paper Writing Pipeline

"I have results. Turn them into a submission-ready PDF."

/paper-writing takes a NARRATIVE_REPORT.md (claims, experiments, key figures) and produces compiled LaTeX:

1. 📐 Plan — /paper-plan builds a claims-evidence matrix from the narrative, then a section-by-section outline
2. 📊 Figures — /paper-figure auto-generates plots (line / bar / heatmap) and comparison tables from JSON/CSV results. Architecture diagrams via /figure-spec (deterministic JSON → SVG) or Gemini illustration
3. ✍️ Write — /paper-write emits per-section LaTeX following the venue's style file. Citations pulled from DBLP / CrossRef (real BibTeX, never LLM-generated)
4. 🔧 Compile — /paper-compile runs latexmk until clean, fixes overfull \hbox, verifies the page limit via pdftotext
5. ✨ Polish — /auto-paper-improvement-loop runs 2 rounds of GPT-5.4 content review + 1 round of format check. Real ICLR run: 4/10 → 8.5/10 across 3 rounds

/paper-writing NARRATIVE_REPORT.md --- venue: ICLR --- effort: max

Workflow 4 — Rebuttal

"Reviews came back. Help me draft a safe rebuttal."

/rebuttal is structured around three hard safety gates that block fabrication:

1. 📥 Parse reviews — atomize into per-reviewer / per-concern entries; normalize against the venue's format (ICML char limit, NeurIPS per-reviewer threads, etc.)
2. 🗺️ Strategy — global themes + per-reviewer priority + character budget + a list of blocked claims (things the reviewer flagged but the paper can't actually back)
3. 🧪 Evidence sprint (optional) — if --- auto experiment: true and the reviewer asked for a missing experiment, hand off to /experiment-bridge, wait for results, fold into draft
4. ✍️ Draft — global opener + numbered per-reviewer responses + closing for the meta-reviewer
5. 🛡️ Safety check — 6 lints: coverage (no concern dropped), provenance (every claim cites paper/review/user-confirmed result), commitment (no overpromising), tone, internal consistency, character limit
6. 🔬 GPT-5.4 stress test — fresh-thread reviewer reads the draft cold, tries to break it
7. 📤 Finalize — two outputs: PASTE_READY.txt (exact character count, ready to drop into OpenReview) + REBUTTAL_DRAFT_rich.md (extended, for human editing)

/rebuttal "paper/ + reviews" --- venue: ICML --- character limit: 5000

Workflow 5 — Resubmit Pipeline

"Move a polished paper from venue A to venue B under hard constraints."

/resubmit-pipeline is for the ICML→NeurIPS / workshop→archival transition. Distinct from Workflow 3 (which writes from scratch) — here the paper exists, the goal is targeted edits only:

1. 📁 Physical isolation — copy into <NEW_VENUE_DIR>/; the original submission directory is never touched
2. 🛡️ 5-layer anonymity check — author names, affiliations, self-citations, GitHub / Overleaf URLs, in-text "we" patterns that break double-blind
3. 🔬 Soft-only audits — /proof-checker, /paper-claim-audit, /citation-audit --soft-only (the --soft-only mode rewrites instead of editing bib, since the bib is frozen)
4. ✏️ Microedit — /auto-paper-improvement-loop --edit-whitelist with a YAML schema (allowed_paths / forbidden_paths / forbidden_operations)
5. 🗡 Adversarial gate — /kill-argument writes the strongest 200-word rejection memo a senior area chair would write; an independent adjudicator scores each point
6. 📤 Compile + push — /paper-compile + optional /overleaf-sync push

/resubmit-pipeline "paper/" --- venue: NeurIPS

Workflow 6 — Conference Talk Pipeline

"Paper accepted. Prepare the talk."

/paper-talk is the post-acceptance sister workflow to /paper-writing:

1. 📋 Outline — extracted from paper/ (or NARRATIVE_REPORT.md if no compiled paper); one slide-cluster per contribution; section→talk-beat mapping
2. 🎨 Generate — /paper-slides emits Beamer source + PPTX + speaker notes + Q&A prep
3. 💎 Polish — /slides-polish does per-page Codex review against the reference PDF; applies a fix-pattern catalog (PPTX font scaling 1.5-1.8× for projector legibility, banner-as-tcolorbox, em-dash spacing, Chinese EA font hint via PingFang SC, anonymity placeholder discipline)
4. 🛡️ Audit (when assurance: conference-ready) — /paper-claim-audit + /citation-audit run against a synthetic paper directory at .aris/paper-talk/audit-input/sections/*.tex to verify slides don't introduce claims the paper can't back

/paper-talk "paper/" --- venue: ICLR --- assurance: conference-ready

Cross-cutting: Research Wiki & Meta-Optimize

Two skills that aren't bound to any single workflow but make the whole system smarter over time:

• 📚 /research-wiki — persistent knowledge base across all seven workflows. Stores every paper read, idea piloted (including failures), experiment run, claim verified. Failed ideas become anti-repetition memory: the next time /idea-creator runs, it sees what didn't work and steers around it.
• 🧬 /meta-optimize — the outer-loop SKILL.md optimizer. Reads accumulated event logs from your past runs (.aris/meta/events.jsonl), analyzes patterns (which skills failed often? which parameter overrides were common? where did scores plateau?), and proposes SKILL.md improvements. Reviewer-gated, user-approved.

Real Results

A real overnight 4-round run on an ML research project, from borderline reject to submission-ready:

Final: 8 pages main body (ICLR limit: 9), 0 overfull \hbox, ICLR-compliant. +2.5 points across 4 rounds.

The 74 Skills

Grouped by role (full catalog: docs/SKILLS_CATALOG.md).

The 3-layer audit chain

A core ARIS invariant: the executor must not judge its own integrity. Three layers of cross-model audit:

Plus /citation-audit (4th layer): every \cite{...} validated for existence, metadata, and context-appropriateness — the most diagnostic check ("does the cited paper actually establish this claim?"). And /kill-argument (5th layer): two fresh codex 5.5 + xhigh threads write the strongest 200-word rejection memo and an independent adjudicator pass before submission.

Cross-platform Support

ARIS skills are plain SKILL.md files. They run anywhere an agent reads markdown:

• 🤖 Claude Code — the default, most tested
• 🤖 Codex CLI — full skill mirror; spawn_agent instead of mcp__codex__codex
• 🖱️ Cursor — agent mode reads ARIS skills directly
• 🖥️ Trae — ByteDance AI IDE
• 🚀 Antigravity — Google's agent-first IDE, native SKILL.md
• 🐙 GitHub Copilot CLI — terminal agent, native SKILL.md
• 🐾 OpenClaw — without Claude Code slash skills

中文版速览

ARIS（Autonomous Research via Adversarial Multi-Agent Collaboration，梦中科研）是一组 74 个可组合的 Claude Code skills，编排跨模型对抗式协作：

• 执行：Claude Code 读文件、写代码、跑实验、改论文
• 审稿：GPT-5.4/5.5（via Codex MCP）以跨家族审稿人身份打分、找弱点、提建议
• 关键：每轮 review 用新 thread；执行者绝不审判自己的实验诚实度

七条工作流（W1 / W1.5 / W2 / W3 / W4 / W5 / W6）端到端贯通：找 idea → 实验桥接 → 自动审稿循环 → 写论文 → 写 rebuttal → 跨 venue 移植 → 会议演讲。在真实 ICLR/NeurIPS 投稿上验证过。

Get Started

# 1. Clone ARIS to a stable location (once)
git clone https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep.git ~/aris_repo

# 2. Attach to a project (creates project-local symlinks)
cd ~/your-paper-project
bash ~/aris_repo/tools/install_aris.sh

# 3. Configure the GPT-5.4 reviewer (Codex MCP)
npm install -g @openai/codex
codex setup                                    # pick gpt-5.5 when asked
claude mcp add codex -s user -- codex mcp-server

# 4. Use in Claude Code
claude
> /research-pipeline "factorized gap in discrete diffusion LMs"

Inspirations

• 🧪 AI Scientist (Sakana AI) — automated research pioneer
• 📖 AutoResearch (Karpathy) — end-to-end research automation
• 🔭 FARS (Analemma) — fully automated research system
• 🎨 PaperBanana (PKU) — multi-agent academic illustration framework

Community