Agentic AI: Comparing New Open-Source Frameworks

We’ve all heard of CrewAI and AutoGen, but did you know there are dozens of open-source agentic frameworks out there — and many of them have been released in the last year.

I’ve briefly tested some of the more popular ones to get a feel for how they work and how easy they are to get started with. So follow along as I go through what each one brings to the table.

The focus will be on LangGraph, Agno, SmolAgents, Mastra, PydanticAI, and Atomic Agents. We’ll compare them to the slightly older but more mainstream CrewAI and AutoGen.

We’ll look at what a framework actually does, the different design choices, how they differ from one another, and a bit about the schools of thought behind them.

Agentic AI

Agentic AI is basically about building systems around LLMs so they can have accurate knowledge, access to data, and the ability to act. You can think of it as using natural language to automate processes and tasks.

Using natural language processing in automation isn’t new — we’ve used NLP for years to extract and process data. What’s new is the amount of freedom we can now give language models, allowing them to handle ambiguity and make decisions dynamically.

But just because LLMs can understand language doesn’t mean they have agency — or even understand the task you’re trying to automate. That’s why there’s a lot of engineering involved in building reliable systems.

I go through Agentic AI in more detail here and here, if you’re keen to get a beginner friendly overview.

What does a framework do?

At their core, agentic frameworks help you with prompt engineering and routing data to and from the LLMs— but they also offer additional abstractions that make it easier to get started.

If you were to build a system from scratch where an LLM should use different APIs — tools — you’d define that in the system prompt. Then you’d request that the LLM returns its response along with the tool it wants to call, so the system can parse and execute the API call.

So basically, we’re talking about prompt engineering — which forms the foundation of any framework.

The framework usually helps in two ways: it structures the prompt properly to make sure the LLM responds in the right format, and then parses the response to route it to the correct tool - or API, document or what have you.

When we set up knowledge, a framework might help with chunking documents, embedding them and storing them. This gets added to the prompt as context, similar to how we build standard RAG systems.

A framework can also help with things like error handling, structured outputs, validation, observability, deployment — and generally help you organize your code so you can build more complex systems, like multi-agent setups.

Still, a lot of people feel that using a full framework is overkill.

The issue is: if the LLM doesn’t use the tool correctly or something breaks, the abstraction becomes a pain because you can’t debug it easily. This can also be a problem if you switch models — the system prompt might have been tailored for one and not transfer well to others.

That’s why some developers end up rewriting parts of a framework —such as create_react_agent in LangGraph — to get better control.

Some frameworks are lighter, some heavier and offer additional features, but there’s community around them to help you get started. And once you learn one (including how it works under the hood), it becomes easier to pick up others.

Different open source frameworks

We do look to the community to understand how well a framework performs in real cases. However, the most popular frameworks may not always be the ideal choice.

The ones we’ve all heard about are CrewAI and AutoGen.

CrewAI is a very high-abstraction framework that lets you build agent systems quickly by hiding low-level details. AutoGen focuses on autonomous, asynchronous agent collaboration, where agents have the freedom to collaborate as they see fit - which may make it more suited for testing and research.

LangGraph is still a fairly well-known system but deserves to be highlighted as one of the main frameworks for developers. It uses a graph-based approach where you build nodes and connect them via agents. Compared to the other two, it gives you stricter engineering control over workflows and doesn’t assume agents should have much agency.

It should be noted that many feel that LangGraph is overly complicated in its abstractions and difficult to debug. The idea is that it has a steep learning curve but once you learn the fundamentals it should get easier.

Now, there are a few newer frameworks I also want to include here.

The next one is Agno (previously Phi-Data) which focuses on providing a very good developer experience. It also has one of the cleanest documentations I’ve seen. It’s very plug-and-play, helping you get started quickly with a lot of built-in features, organized into logical, clean abstractions that make sense.

SmolAgents is a very bare-bones framework that introduces an agent — CodingAgent — which route data via code rather than JSON. It also gives you direct access to the entire Hugging Face model library out of the box.

As for the open-source frameworks that aren’t as often mentioned:

PydanticAI builds on Pydantic with minimal abstraction, offering a bare-bones framework that’s highly transparent. It’s great when you need strict type safety and predictable, validated outputs, for fine-grained control, making it easier to debug.

Atomic Agents is developed by an individual agent builder and uses schema-driven building blocks you connect like Lego, with a strong focus on structure and control. It was built as a response to the lack of alternatives that worked well in practice.

Both of PydanticAI’s and Atomic Agent’s goal is to move away from black-box AI that acts independently.

Mastra, created by the team behind Gatsby, is a JavaScript framework built for frontend developers to easily build agents within their own ecosystem.

We’ll go through what each of these have, and what makes them different.

What they all have

Most frameworks come with the same core building blocks: support for different models, tools, memory, and RAG.

Most open-source frameworks are more or less model agnostic. This means they’re built to support various providers. However, as mentioned earlier, each framework has its own structure for system prompts — and that structure may work better with some models than others.

That’s also why you ideally want access to the system prompt and the ability to tweak it if needed.

All agentic frameworks support tooling, since tools are essential for building systems that can act. They also make it easy to define your own custom tools through simple abstractions. Today, most frameworks support MCP, either officially or through community solutions.

It’s important to understand that not all models are built for function calling, which is necessary for using tools. To figure out which models are best suited as the base LLM, you can check Hugging Face’s agent leaderboard.

To enable agents to retain short-term memory between LLM calls, all frameworks make use of state. State helps the LLM remember what was said in earlier steps or parts of the conversation.

Most frameworks also offer simple options to set up RAG with different databases to provide the agent with knowledge.

Finally, nearly all frameworks support asynchronous calls, structured outputs, streaming, and the ability to add observability.

I have gathered a lot of the research I have done in this Google sheet, you’ll also find a table in this repository if you need a better overview.

What some don’t have

Frameworks will differ in some areas, such as supporting multimodal input, memory, and multi-agent systems. Some handle it for you, while others leave the wiring to you.

First, some frameworks have built-in solutions for handling multimodality — i.e. text, image, and voice. It’s all possible to implement this yourself, as long as the model supports it.

As said previously, short-term memory (state) is always included — without it, you can’t build a system that uses tools. However, long-term memory is trickier to implement, and this is where frameworks differ. Some offer built-in solutions, while others you’ll have to connect other solutions on your own.

Frameworks also vary in how they handle multi-agent capabilities. Multi-agent systems allow you to build collaborative or hierarchical setups with teams of agents connected via supervisors.

Most frameworks recommend keeping agents focused — a narrow scope with a limited set of tools. That means you’ll likely need to build out teams of agents to handle complex workflows. All frameworks let you build one team, but some get complicated when scaling into multi-hierarchical systems with multiple layers.

This is where LangGraph stands out — you can build out nodes, connect them to various supervisors, and visualize how different teams interact. It’s clearly the most flexible when building multi-agent systems at scale.

Agno recently added support for teams, both collaborative and hierarchical, but there aren’t many examples yet for more complex, multi-hierarchical setups.

SmolAgents lets you connect agents to a supervisor but can get complex as the system grows. It reminds me of CrewAI in how it structures agent teams. Mastra is similar in that sense.

With PydanticAI and Atomic Agents, you’ll need to manually chain your agent teams, so orchestration falls on you.

How they’re all different

Frameworks differ in how much they abstract away, how much control they give agents, and how much coding you’ll need to do to get something working.

First, some frameworks make it a point to include a lot of built‑in features, making it easy to get started quickly.

I’d say Mastra, CrewAI, and to some extent Agno are built to be plug and play.

LangGraph also has a good amount of abstraction, but it uses a graph-based system where you manually connect nodes. That gives you more control but also means you have to set up and manage every connection yourself, which comes with a steeper learning curve.

Then we have the low-level abstraction frameworks like PydanticAI, SmolAgents, and Atomic Agents.

These make it a point to be transparent, but you often have to build out the orchestration yourself. This gives you full control and helps with debugging — but it also increases time to build.

Another point of difference is how much agency the framework assumes the agent should have. Some are built on the idea that LLMs should be smart enough to figure out how to complete the task on their own. Others lean toward tight control — giving agents one job and guiding them step by step.

AutoGen and SmolAgents fall into the first camp. The rest lean more towards control.

There’s something to consider here: when developers build frameworks that focus on tight control, it’s often because they haven’t found a way to let agents work on their own yet — at least not reliably.

This space is also starting to look more and more like engineering.

If you’re going to build these systems, you do need to understand how to code. The real question is how much the frameworks differ in terms of how technical you need to be.

If you’re less experienced, going with CrewAI, Agno, or Mastra might be a good idea.

SmolAgents is also pretty straightforward for simple use cases.

As for PydanticAI, Atomic Agents, and LangGraph — you’ll be writing a lot more of the logic yourself. Though to be fair, it’s always possible to build an agent to help you structure your code correctly as well.

If you’re completely new to programming then you can check out Flowise or Dify.

Lastly, it’s worth mentioning the developer experience across these frameworks.

From what I’ve seen, most developers find CrewAI and AutoGen tricky to debug. SmolAgents’ CodeAgent introduces a novel approach where agents output code to route data — a cool idea, but it doesn’t always work as intended.

LangGraph, especially when paired with LangChain, comes with a steep learning curve and some confusing abstractions that you may end up having to break apart and rebuild.

PydanticAI and Atomic Agents are generally liked by developers, but they do require you to build the orchestration yourself.

Agno and Mastra are solid choices, but you might run into issues like looping calls that can be hard to debug.

Some notes

The best way to get started is just to jump in and try something out. But I hope this gave you a decent overview of the open-source frameworks out there — and what might work for you.

This was a fairly shallow look at each one though, as I haven’t gone into things like enterprise-grade scalability or operational robustness. You’ll need to research those parts separately if that’s what you’re building for.

Some devs say AI agent frameworks are some of the worst forms of abstraction — that they often make things more complicated than just using the official LLM provider’s SDK directly.

I’ll leave that one up to you to decide.

Remember the full list of features for each of these frameworks you can find here, and the repository with the full list of agentic frameworks you can find here.

I hope you liked it, and if so be sure to share it.

❤