Build AI Agents to Automate Your Business — the Full Framework
this is my AI-OS thesis said out loud — markdown context + skills + MCP connectors, own-your-context, portable across harnesses. Remy independently arrived at the exact architecture I'm building toward with Hermes, and explains it non-technically.
Watch the original by Open Residency on YouTubeTL;DR
The jump from chat (question → answer) to agents (goal → result) is a 5–10× productivity divide, and most people are stuck on the wrong side. An agent is just a loop — observe → think → act — run by a "harness" (Claude Code, Codex, Manus, etc.). You don't build agents, you tune them with four levers: Context, Skills, Model, Tools. The mental model that makes it click: onboard the agent like a new employee — give it context (who you are, as markdown), tools (email/browser/etc. via MCP), and skills (your SOPs/IP as markdown). Keep all of it in a local "OS" folder you own, so it's portable across any harness. And it's not about the model — a well-tuned "Volvo" beats a stock "Lamborghini."
Key takeaways
- Chat vs agents: question→answer (chat) can't take real work off your plate; goal→result (agents) hands you a finished product. Use chat for trivia ("calories in a banana"), agents for meaningful work.
- The loop is the whole thing: observe → think → act, repeated until done. So define what "done" looks like explicitly or you get vague output.
- A "harness" runs the loop — Claude Code, Codex, Manus, Perplexity Computer, Open Claw, Hermes are all just harnesses. "I built an agent" almost always means "I tuned one."
- Four levers, mapped to the loop: Context + Skills (observe) · Model (think) · Tools (act).
- It's the tuning, not the model. Learn to "drive" the fundamentals and you can hop between any harness in minutes.
- Own your context. Markdown files you control > the ChatGPT "black box" memory you can't see or edit — and your files are portable across every harness.
- MCP is just a universal translator connecting your agent to tools. He now picks vendors by whether they have an MCP, and barely opens app front-ends anymore.
- Skills are where your IP lives. Generic agents write generic slop; a skill encodes your taste and process so it's repeatable.
- Security: start read-only, escalate as trust builds, and keep the last line (payments) human-gated.
Two stages: chat → agents
flowchart LR
subgraph CHAT["Stage 1 · Chat — question to answer"]
Q["You ask"] --> A["It answers"] --> W["you still do the work"]
end
subgraph AGENT["Stage 2 · Agents — goal to result"]
G["You give a goal"] --> L["agent works the loop"] --> R["finished product handed back"]
end
"Chat is a brilliant consultant; an agent is hiring that person in-house to actually do it."

The agent loop + the four levers
Every agent, given a task, just runs this until it decides the goal is met — and each step has a lever you can tune:
flowchart LR
O["OBSERVE<br/>tune with:<br/>Context + Skills"] --> T["THINK<br/>tune with:<br/>Model (LLM)"]
T --> A["ACT<br/>tune with:<br/>Tools (MCP)"]
A -->|"not done yet"| O
A -->|"done"| DONE["Output result"]

The "harness" is just the app that runs this loop. Key mindset: a well-tuned lesser model beats a stock frontier model — "a Volvo beats a Lamborghini if the context, skills, and tools are dialed."
Onboard the agent like an employee

Treat the agent as a "really capable stranger" you hired for ~$100/mo with infinite bandwidth — useless until properly onboarded on three things:
flowchart TD
E["Capable stranger<br/>(the agent)"] --> C["CONTEXT<br/>who you are, your business,<br/>ICP, offers, brand voice"]
E --> T["TOOLS<br/>email, browser, Slack, Stripe...<br/>connected via MCP"]
E --> S["SKILLS<br/>your SOPs and secret sauce<br/>(repeatable processes)"]
C --> B["= the best employee you've had"]
T --> B
S --> B
1 · Context — markdown, not PDFs
Give it the docs you'd give a new hire, but as markdown (easy for the model to ingest; PDFs are harder). Structure it in a folder you own:
flowchart TD
ROOT["your OS folder (local, yours)"] --> CM["CLAUDE.md / agents.md<br/>north-star, auto-loads every session"]
ROOT --> CTX["context/<br/>about-me, business, ICP,<br/>offer-catalog, brand-voice"]
ROOT --> SK["skills/<br/>your SOPs (progressive disclosure)"]
ROOT --> MEM["memory.md<br/>DIY self-improvement — writes<br/>your corrections back as rules"]
CM -.points to.-> CTX
CM -.points to.-> MEM

Keep files lean — every session pre-loads them, and a bloated context causes "context rot" (the agent starts forgetting/hallucinating). Owning these markdown files (vs ChatGPT's hidden memory) means you control exactly what it knows and you can carry it to any harness. In practice it's just a folder on your desktop:

2 · Tools — MCP connectors
MCP (Model Context Protocol) is a translator between your agent and third-party tools — so the agent doesn't need custom code to "speak Gmail / Slack / Stripe." In practice it's one-click connectors. His six must-haves:

| Tool | What it adds |
|---|---|
| Apify | Marketplace of scrapers (Instagram, YouTube, LinkedIn, Maps…) |
| Firecrawl | Actually reads a full page (CRO audits, branding) vs. plain web search |
| Composio | An "MCP of MCPs" — one bridge for all your tools, portable across harnesses & multiple accounts |
| Chrome DevTools | Agent spins up its own browser to check its own work |
| Playwright | Drives your logged-in browser (form-fills, actions) |
| Higgsfield | Image/video generation (the "Composio for image models") |
He now evaluates software by "does it have an MCP?" — if not, it's out. The agent becomes an intelligence layer on top of all your tools, and you stop opening front-ends.
3 · Skills — SOPs for AI (where your IP lives)

A skill is a folder with a skill.md: name + description + step-by-step contents — exactly the Google Doc you'd write for a VA, but for the agent. Two things make it powerful:
- Progressive disclosure: only the names + descriptions ("book spines") load at session start; the full skill loads only when triggered — so you can have hundreds without bloating context.
- Build process-first: do the task once with the agent, dial it in, then say "save this as a skill" — the built-in skill-creator skill packages it. Now it's repeatable forever.
Skills are where your taste and process get encoded — the difference between generic AI slop and output that sounds like you. In the demo, an "ad scraper" skill autonomously pulls a competitor's live ads and builds a full report:

The bigger point
The whole system is vendor-independent by design: your context and skills live in a local folder you own, so if Claude goes down you open the same folder in Codex and keep working (Composio carries your tools over). That's the same conviction behind a git-backed markdown "brain" — the models and harnesses are swappable UIs; the durable asset is your own well-built context and skills. Remy's framing of the future: 1–3 person teams doing billions, and the individual "100× employee" who learns to tune agents now.
Mitchell Miller