General vs Specialized AI Agents in the IDE
The question
Three months into building an AI IDE extension, we had to make a fundamental architecture choice: do we build one general-purpose AI assistant ("ask me anything") or six specialized agents (one for code generation, one for test generation, one for requirement analysis, etc.)?
This isn't a theoretical debate. The choice affects code complexity, UX simplicity, deployment surface, cost per user, and how safe users feel trusting the tool with different types of work. We spent weeks building prototypes of both approaches, talking to early users, and arguing about the tradeoffs. Here's the reasoning on both sides and why we landed where we did.
The case for one general assistant
The pitch: ChatGPT-style. One model, one interface, one conversation thread. You ask it anything, it answers anything. Simpler for the user. Simpler for us.
Why it's appealing
Simpler UX: one chat box. You talk to one assistant about code generation, then tests, then requirements, in the same conversation. Lower cognitive load.
Easier to ship: one model, one interaction pattern. You don't have to design six different interfaces.
Lower development cost: one thing to maintain. We spend that time on making it better instead of managing multiple surfaces.
Feels familiar: everyone knows how to talk to ChatGPT. No learning curve.
Single LLM context: one conversation thread means the assistant remembers what you asked it to generate for tests when you ask it to generate code. That context is valuable.
The real advantage
The honest advantage isn't the UX simplicity. It's cost and operational burden. One model means one API integration, one set of rate limits, one billing stream, one place where things can go wrong. Multiple specialized agents means multiple LLM calls, multiple vendor relationships (or multiple endpoints to one vendor), and more failure modes. That's not trivial.
The case for six specialized agents
The pitch: different jobs need different tools. Code generation has different safety requirements than test generation. They have different cost profiles. They need different levels of user trust. Give each its own agent.
Why it matters in practice
Different trust levels: When I ask an AI to generate tests, I want a high bar for safety. I'm going to run those tests in my CI. If they're wrong, they fail, and my pipeline breaks. When I ask it to generate code suggestions, I want it to be helpful and creative. Different trust profiles need different tuning.
Different cost optimization: Generating a test case from a requirement is a straightforward task. Could use a smaller, cheaper model. Generating code from a vague requirement is harder. Needs a bigger model. Don't pay for big-model latency on easy tasks.
Parallel execution safety: If one assistant is generating code while another generates tests, they should know they're not stepping on each other. A single general assistant serializes everything into one conversation. Multiple agents can coordinate more safely.
Different UX per task: Test generation should show you generated tests immediately and let you edit. Code generation should show you diffs. Requirement analysis should show you structure and extracted intents. Different tasks, different UI. A general assistant pushes all of it through the same text interface.
Specialization tuning: A specialized agent can be tuned for its specific task. The test-generation agent can be trained on thousands of test examples. The code-generation agent can be tuned for idiomatic code in your language. A general agent is a compromise across all tasks.
The real tradeoff
The honest downside: complexity. You have to manage six agents. Six contexts. Six potential failure modes. Six places where the user experience can be inconsistent. And you have to decide: do they all use the same model, or different models? If different, you're managing multiple LLM vendors or multiple endpoints. If the same, you lose the specialization benefit.
Three months of debate, and where we landed
We built both. Prototyped both. Then chose specialized.
Here's why:
Safety matters more than simplicity: an engineer who runs generated tests that are wrong once will never trust test generation again. Same with code generation. Different tasks need different safety bars, and you can't tune a single model across all of them equally well.
Cost doesn't matter if UX is wrong: yes, one model is cheaper per request. But if the UX is confusing (why is my code generation output in chat format? Why can't I edit the tests inline?), users won't adopt it. We optimized for adoption.
Context is overrated: yes, a general assistant remembers what you asked it earlier. But in a specialized system, the test agent already knows the code context because the IDE extension provides it. You don't need the assistant to remember; you feed it the right context upfront.
Different tasks, different people: code generation is used by every engineer. Test generation is used by QA and some devs. Requirement analysis is used by PMs and tech leads. They have different workflows. One interface can't fit all of them well.
The architecture we chose
Six specialized agents, all running the same foundational model (Claude Sonnet), coordinated through a single orchestrator. The orchestrator knows which agent to call based on what the user asked for. Each agent has its own:
Context window setup: test generation gets the requirement + code context; code generation gets the spec + existing code; requirement analysis gets the document.
Prompt tuning: each agent has a different system prompt optimized for its task.
Safety gates: test generation is more conservative; code generation is more aggressive.
Output format: tests render as a table; code renders as diff; requirements render as structured JSON.
The honest counterargument
Building six agents instead of one is more work. We could have shipped a general assistant in half the time. And maybe users would have liked the simplicity. Maybe the context-in-conversation would have been valuable. Maybe we overthought the trust and cost angles.
We won't know until we ship and see how people use it. That's the tradeoff: we chose architectural purity (the right tool for each job) over UX simplicity (one chat box for everything). Whether that was the right call depends on whether engineers actually want specialized agents or whether they want ChatGPT-in-their-IDE.
For builders making this choice
If you're building AI-powered developer tools and facing this same decision, here's what I'd watch:
Who is the user? Different users have different safety expectations. If your users are QA engineers, safety matters more than simplicity. If they're developers, they might want flexibility.
What's the most dangerous output? If the wrong output could break a production pipeline or introduce a security hole, specialize and be conservative. If it's a code suggestion the engineer will review, generalize.
How much context can you pre-load? If the IDE can give the agent rich context (repo structure, language, frameworks, existing code), a specialized agent doesn't need to remember conversation history. If context is scarce, a general assistant with conversation memory might be better.
What's your cost ceiling? If you can afford multiple API calls per request, specialize. If you need to minimize calls, generalize.
What we're watching
We shipped specialized agents. Now we're watching:
Do engineers actually use different agents for different tasks, or do they ask one agent for everything?
Does the specialized UX (inline test editing, code diffs) actually increase adoption, or would a simpler chat interface have been fine?
When an engineer wants to generate code and tests together, does coordinating between two agents work smoothly, or do they wish it was one conversation?
The closing honest take
Specialized agents are architecturally cleaner and operationally safer. General assistants are simpler to explain and use. We chose the former. We think we made the right call. But ask us again in six months when we have usage data.
See specialized agents in action. We built six of them into our VSCode extension. Try it.
