AutoGen vs CrewAI - two multi-agent idioms

Open in Colab ▶️ Run this notebook in Colab

AutoGen vs CrewAI - two multi-agent idioms#

Track 04 - Agents · Notebook 06 · Runtime: ≈30 s on CPU

Prerequisites: 04_agents/03 (LangGraph state machines).

References:

What#

Both frameworks orchestrate multiple LLM-backed agents toward a goal; they differ in the primitive:

  • AutoGen - agents communicate by sending messages. A group chat manager dispatches. Good for collaborative back-and-forth.

  • CrewAI - agents have named roles (planner, researcher, writer); each role is assigned a task. Execution follows a task DAG. Good for structured pipelines with clear hand-offs.

Both work; the choice is stylistic. We implement minimal clones of each and run them on the same toy task (draft + critique + revise a short article). With the same underlying LM stubs, they reach the same final answer - the frameworks just route the messages differently.

from llm_systems_cookbook.nb import bootstrap

from dataclasses import dataclass, field

s = bootstrap("04_agents_06_autogen_0_4_vs_crewai")

Shared task and LM stubs#

Three deterministic LM-like functions:

  • drafter(topic) - returns a short paragraph.

  • critic(text) - returns up to three issues.

  • reviser(text, issues) - edits the text addressing the issues.

TOPIC = "the importance of the mitochondrion in eukaryotic cells"


def drafter(topic: str) -> str:
    return ("Mitochondria are organelles in eukaryotic cells. They generate energy. "
            "They contain their own DNA. They evolved from bacteria. The End.")


def critic(text: str) -> list[str]:
    issues: list[str] = []
    if "The End" in text:
        issues.append("remove the closing 'The End' phrase")
    if "energy" in text and "ATP" not in text:
        issues.append("mention ATP when describing energy production")
    if "bacteria" in text and "endosymbiotic" not in text.lower():
        issues.append("reference the endosymbiotic theory")
    return issues


def reviser(text: str, issues: list[str]) -> str:
    revised = text.replace(" The End.", "")
    if any("ATP" in i for i in issues):
        revised = revised.replace("generate energy.", "generate ATP via oxidative phosphorylation.")
    if any("endosymbiotic" in i for i in issues):
        revised = revised.replace("evolved from bacteria.",
                                   "evolved from bacteria per the endosymbiotic theory.")
    return revised

AutoGen-style: conversation driven#

All agents write to a shared message log. The “group chat manager” decides who speaks next based on the last message. Termination is explicit - when the critic returns zero issues.

@dataclass
class Message:
    sender: str
    content: str


def run_autogen_style(topic: str, max_turns: int = 6) -> tuple[str, list[Message]]:
    log: list[Message] = []
    current_draft = drafter(topic)
    log.append(Message("drafter", current_draft))

    for _ in range(max_turns):
        issues = critic(current_draft)
        log.append(Message("critic", "issues: " + (", ".join(issues) or "none")))
        if not issues:
            break
        current_draft = reviser(current_draft, issues)
        log.append(Message("reviser", current_draft))

    return current_draft, log


autogen_final, autogen_log = run_autogen_style(TOPIC)
print(f"AutoGen-style: {len(autogen_log)} messages")
for m in autogen_log:
    print(f"  [{m.sender}]  {m.content[:80]}")
print(f"\nfinal: {autogen_final}")

CrewAI-style: roles + task DAG#

Each agent has a role and a task list; the crew executes tasks in order. The Crew.run method is a plain loop. Same functions as AutoGen, just routed by pre-declared role.

@dataclass
class Agent:
    role: str
    goal: str


@dataclass
class Task:
    description: str
    agent: Agent
    expects_input_from: list[str] = field(default_factory=list)


@dataclass
class Crew:
    agents: list[Agent]
    tasks: list[Task]

    def run(self, topic: str, max_revisions: int = 3) -> tuple[str, list[str]]:
        artefacts: dict[str, str] = {}
        traces: list[str] = []

        draft = drafter(topic)
        artefacts["draft"] = draft
        traces.append(f"drafter -> draft")

        for _ in range(max_revisions):
            issues = critic(artefacts["draft"])
            traces.append(f"critic -> issues[{len(issues)}]")
            if not issues:
                break
            artefacts["draft"] = reviser(artefacts["draft"], issues)
            traces.append(f"reviser -> draft")
        return artefacts["draft"], traces


crew = Crew(
    agents=[
        Agent("drafter", "write an initial paragraph"),
        Agent("critic", "identify issues in the paragraph"),
        Agent("reviser", "fix issues in the paragraph"),
    ],
    tasks=[
        Task("write a paragraph about the topic", Agent("drafter", "write")),
        Task("critique the paragraph", Agent("critic", "review"), expects_input_from=["drafter"]),
        Task("rewrite incorporating critiques", Agent("reviser", "fix"),
             expects_input_from=["drafter", "critic"]),
    ],
)
crewai_final, crewai_trace = crew.run(TOPIC)
print(f"CrewAI-style trace: {' -> '.join(crewai_trace)}")
print(f"\nfinal: {crewai_final}")

s.check(
    "autogen_final_removes_the_end",
    lambda: "The End" not in autogen_final,
    msg=f"final = {autogen_final!r}",
)
s.check(
    "autogen_final_mentions_atp",
    lambda: "ATP" in autogen_final,
    msg=f"final = {autogen_final!r}",
)
s.check(
    "autogen_final_references_endosymbiotic",
    lambda: "endosymbiotic" in autogen_final.lower(),
    msg=f"final = {autogen_final!r}",
)
s.check(
    "crewai_reaches_same_final_answer",
    lambda: crewai_final == autogen_final,
    msg=f"crewai={crewai_final!r}  autogen={autogen_final!r}",
)
s.check(
    "autogen_terminates_when_critic_returns_empty",
    lambda: autogen_log[-1].sender == "critic" and "none" in autogen_log[-1].content,
    msg=f"last message = {autogen_log[-1]}",
)
s.check(
    "crewai_trace_includes_all_three_roles",
    lambda: all(role in str(crewai_trace) for role in ("drafter", "critic", "reviser")),
    msg=f"trace = {crewai_trace}",
)

Same work, two routings#

Three bars per framework. Completion is scored against the critic’s checklist (no “The End”, mentions ATP, references endosymbiosis); both idioms reach the same final, so quality is identical by construction. Wall-clock is measured over 50 runs of each flow - the conversation-log bookkeeping costs more than the role-DAG here. Message/trace-step count quantifies routing overhead: AutoGen pays an explicit critic turn per iteration, CrewAI logs one artefact per role.

import time
import matplotlib.pyplot as plt


def quality(text: str) -> float:
    hits = sum(bool(x) for x in ("The End" not in text, "ATP" in text, "endosymbiotic" in text.lower()))
    return hits / 3.0


N = 50
ag_times: list[float] = []; cw_times: list[float] = []
for _ in range(N):
    t0 = time.perf_counter(); run_autogen_style(TOPIC); ag_times.append((time.perf_counter() - t0) * 1e3)
    t0 = time.perf_counter(); crew.run(TOPIC);          cw_times.append((time.perf_counter() - t0) * 1e3)

frameworks = ["AutoGen-style", "CrewAI-style"]
completion = [quality(autogen_final), quality(crewai_final)]
med_ms = [sorted(ag_times)[N // 2], sorted(cw_times)[N // 2]]
messages = [len(autogen_log), len(crewai_trace)]

fig, axes = plt.subplots(1, 3, figsize=(10, 3.2))
axes[0].bar(frameworks, completion, color=["tab:blue", "tab:orange"])
axes[0].set_ylim(0, 1.05); axes[0].set_title("task completion (critic-checklist)")
for i, v in enumerate(completion):
    axes[0].text(i, v + 0.02, f"{v:.2f}", ha="center")

axes[1].bar(frameworks, med_ms, color=["tab:blue", "tab:orange"])
axes[1].set_ylabel("median wall-clock (ms)"); axes[1].set_title(f"latency over {N} runs")
for i, v in enumerate(med_ms):
    axes[1].text(i, v, f"{v:.2f}", ha="center", va="bottom", fontsize=9)

axes[2].bar(frameworks, messages, color=["tab:blue", "tab:orange"])
axes[2].set_ylabel("messages / trace steps"); axes[2].set_title("routing overhead")
for i, v in enumerate(messages):
    axes[2].text(i, v, str(v), ha="center", va="bottom")
fig.tight_layout(); plt.show()

Exercises#

  1. Cyclic task DAG. Real CrewAI supports manager processes that can loop. Extend the Crew class with a conditional edge: “if critic finds issues, reschedule reviser + critic”. This converges to the AutoGen behaviour.

  2. Multi-critic voting. Add three critics, each focusing on a different dimension (clarity, accuracy, style). The reviser only acts on issues flagged by at least two critics.

  3. Real AutoGen. pip install autogen-agentchat and run the same flow with RoundRobinGroupChat. The message routing is identical.

References#

  • AutoGen 0.4 docs - the new event-driven model supersedes the older conversable-agent API.

  • CrewAI’s Process abstraction (sequential, hierarchical) for the production routing semantics.

s.summary()
s.save()