PenguinHarness

Why PenguinHarness

PenguinHarness is an open-source framework for agent self-evolution (recursive self-improvement, RSI). The model stays fixed; the harness evolves: prompts, Skills, tools and hooks. It runs on your computer or server as a desktop app, a Web App, a CLI and an SDK.

• Every RSI algorithm in one framework. Default, OPRO, APE, ACE, AWM, SkillAdam, SkillOpt, OpenSkill and TRACE ship as plugins and run on the same built-in Benchmarks, scoreboard and snapshots.
• One sentence builds an agent app. Describe what you need, and an agent writes the app, its page and its run instructions.
• Cheap to run. A deliberately small toolset, tuned for open models: on our data-analysis suite, the best accuracy at 1/70 of Claude Code's cost.

Self-evolution

Pick an algorithm, pick a Benchmark, and an agent improves another agent round by round. Every candidate is a new version, snapshotted before it changes, and every score lands in the Evaluation Center.

Toolkit	Algorithm
default-rsi-toolkit	Trial, reflect, improve: one hypothesis and one change per round
opro-rsi-toolkit	OPRO: new instructions from the history of scored ones
ape-rsi-toolkit	APE: candidate instructions induced from examples
ace-rsi-toolkit	ACE: lessons from Traces curated into a playbook
awm-rsi-toolkit	AWM: workflows induced from successful runs
skilladam-rsi-toolkit	SkillAdam: evolving issue tracker and adaptive Skill edit budget
skillopt-rsi-toolkit	SkillOpt: bounded Skill edits, score-based selection and cross-epoch reflection
openskill-rsi-toolkit	OpenSkill: external research and independent virtual tests before frozen-State evaluation
trace-rsi-toolkit	TRACE: skill-bank evolution from contrasting trajectories, with consistency measurement

Every new Project includes seventeen Benchmarks, run as Harbor tasks in Docker: PenguinHarness Benchmark Sec A to Sec E (from rag-bench-essential, DeepSWE, AutomationBench, Terminal-Bench-Science and Terminal-Bench) and a Train/Test pair for each of GDPEVO Essential's six scenarios. More are in penguin-harness-benchmark.

Demo: a decision agent is evaluated, revises its working method once, and is retested on the same model. Its score goes from 84 to 100, and the cost of every run is on screen.

https://github.com/user-attachments/assets/7925b4b5-db78-4e00-b209-89deca394e34

One sentence, one app

Build a legal review agent with a simple workspace where users upload contracts, track the review, and download a report with risky clauses, explanations, and suggested revisions.

The result: a legal review app. Users upload contracts, follow each file's review, and download a report of risky clauses with explanations and suggested revisions. Five sample reviews cost $0.0423 in model calls.

https://github.com/user-attachments/assets/28c0ef95-1dfa-47be-bad0-7326ccdf4e7c

What agents can do

Replies can also carry UI: choices, forms, steps, Mermaid diagrams and live widgets (A2UI).

Results

Each harness on the model it is usually paired with, on the same tasks:

Built-in plugins

Agents can also write and optimize their own Skills (docs).

Category	Plugins
Agent Self-Evolution	default-rsi-toolkit, opro-rsi-toolkit, ape-rsi-toolkit, ace-rsi-toolkit, awm-rsi-toolkit, skilladam-rsi-toolkit, skillopt-rsi-toolkit, openskill-rsi-toolkit, trace-rsi-toolkit
Office Productivity	a2ui, data-analysis, use-firecrawl, browser-automation, use-bento-slides, humanizer, goal, continual-learning, agent-company, penguin-support
Software Development	software-development, use-claude-code
AI App Development	agent-development, agent-evaluation, model-development, skill-porting

Supported models

These are the model families we focus on supporting. The app's Models page lists each variant and its providers.

Closed-source models

Model	Providers
GPT 6.1	OpenAI, OpenRouter
Claude 5.5	Anthropic, OpenRouter, OpenCode Go
Gemini 4	Google Gemini, OpenRouter, Penguin Go, OpenCode Go
Grok 4.7	OpenRouter, OpenCode Go
Qwen3.8 Max	Qwen Token Plan, Qwen Pay-As-You-Go, OpenRouter, TokenDance, OpenCode Go

Open-source models

Model	Providers
DeepSeek V4.1	DeepSeek, OpenRouter, Fireworks AI, Atlas Cloud, TokenDance, Penguin Go, OpenCode Go, Qwen Token Plan, Qwen Pay-As-You-Go, ModelScope
Kimi K3	Moonshot AI, OpenRouter, Fireworks AI, TokenDance, OpenCode Go, Qwen Pay-As-You-Go
GLM 5.3	Z.AI, OpenRouter, TokenDance, OpenCode Go
Hunyuan 4	OpenRouter, TokenDance, OpenCode Go
MiMo v2.6	OpenRouter, TokenDance, OpenCode Go

OpenCode Go offers Claude Haiku 5.5; Fable 5.1, Opus 5.5 and Sonnet 5.5 are available through Anthropic and OpenRouter. Hunyuan 4 uses hy4-preview.

Custom OpenAI-protocol endpoints also work, covering 1000+ online and local models.

Install

Desktop app

Download the installer from penguin.ooo/download: macOS 11+ (dmg), Windows 10+ (exe) or Linux x64 (AppImage, deb). The app opens signed in and needs no terminal.

This is the beginning of the README. Read the rest at the original. Read the full README ↗