Are AI Development Frameworks The Foundation Of The Agentic World?

Over the past few months we have heard quite a bit about AI agents and agentic development. Last September, I predicted that agents would be a big deal, and to be honest, I am not sure we have even seen peak agentic hype yet. So, everyone should expect the term to be stretched, pulled, and completely overdone by springtime. That said, I firmly believe that agentic is a simultaneously powerful and pragmatic way to generate business results from AI technology.

So far, we have seen a great deal of momentum from platform vendors such as Salesforce, which is able to deliver agentic capabilities in the context of its Agentforce platform. We have also seen agentic capabilities injected into developer tools such as Amazon Q Developer and Google Agentspace. But what if I want a more flexible approach to developing AI applications, whether they are agentic or not? That specific programming need is where we are starting to see the emergence of AI development frameworks.

What Is An AI Development Framework?

Conceptually, ADF is a type of middleware technology. To be more specific, it’s a layer of shared services that provides a set of APIs and integrations for practitioners (not just developers) to build AI applications, particularly agents. The benefit of this is twofold. First, developers don’t have to sacrifice too much flexibility while gaining the potential to work more efficiently. Second, the enterprise also gets a more uniform set of standards to drive better governance and sustainability. The emergence of ADFs is very important right now in the world of enterprise AI, for several reasons:

1. The idea of one LLM suited and trained to most or all tasks is unlikely in the near term. This is giving way to companies adopting multiple smaller and less expensive models distributed across multiple locations.
2. Agentic development is more complex than development for other types of AI apps and requires a mix of capabilities including natural language, visual processing, machine learning, proprietary data, business logic, workflows, security, robotic actions, deterministic programming . . . and the list goes on and on.
3. Enterprise concerns about scaling generative AI, including governance, security and monitoring, are slowing adoption and raising questions about ROI.
4. AI efforts are evolving and growing. So it’s no longer about individuals and small teams “in the lab.” Instead, the work is shifting to larger teams with diverse and necessary enterprise skills (including developers, data scientists, line-of-business analysts and so on).
5. While we have seen great progress in the embedding of agentic capabilities in SaaS offerings (such as Agentforce and ServiceNow), those are more traditionally directed at providing agentic capabilities to existing data, workflows and tools — versus the more horizontal approach in an ADF.

Why AI Development Frameworks Matter To Enterprises

More Flexibility To Support Broader Use Cases

Ultimately, when an enterprise makes a strategic technology decision, the company will want something that provides as much flexibility as it can reasonably manage with the resources it has — human, financial and technical. And so far, AI development has been a bit lacking in this arena. Initially, there was a thought that if you chose a large AI model from a trusted vendor, you could just build simple applications (bots) and let the model do the work. And for some use cases, that is still a pretty useful approach.

But it turns out that it is not the only way to interact with models. For instance, we started to see some models respond in ways that businesses didn’t want. So different approaches and types of interaction emerged. One that was very interesting in the responsible-AI world was the filtered interaction. IBM recently announced that one of its Granite AI models could be inserted into the language flow to provide guardrails for the output of a large generic model. My colleague Paul Smith-Goodson recently wrote about Granite and this new capability. We could also see other instances such as language translation where this filtered interaction could work. But while filtering has led to better responses, it still lacks the flexibility needed for fleets of hundreds or thousands of task-based agents.

This is where we have started to see the middleware interaction emerge. This type of interaction enables more of a mesh, in which many agents can work together with many different models. The shared services and coordination among those agents and models is provided by the AI development framework, which allows for more flexibility on both the agentic and model layers.

Application Of Consistent Standards

Approaches like this also enable the consistent application of, and compliance with, industry or internally developed standards. For instance, AWS has externalized its Guardrails service so that the same responsible-AI approaches and rules are applied no matter which model you choose for your agent. Again, this is in contrast to what we saw in early implementation of generative AI, where the model itself was where the logic was managed.

This is a big deal because today the typical IT user may know about only a handful of the general-purpose models out there. To be fair, those very large and very expensive models are useful for many apps and agents. But by now there are many thousands of useful models available, some of which have very specific industry or machine understanding that will add extreme value — but only to very few agents. So much so, that without the rise of these smaller niche models, the project never would have been viable (economically or otherwise). So, if you are going to be supporting potentially dozens of models, you will need a consistent way to manage them. (More on that later.)

Scalable Integration

Enterprises also require a high degree of integration with their applications and data. That alone represents a very sizable effort within an AI project. So, providing a common set of integration services and pathways through a framework can be critical to agentic applications. Not only will these common paths be easier to manage and scale, but they should also be easier to diagnose and repair if there are performance problems.

This will become a bigger issue than we have seen in the past with other integration movements such as microservices and service-oriented architectures. Agents by their nature are autonomous and will stop working if there is too much latency in the data supply chain. However, by providing the agents with a prescriptive way to access and utilize that data supply chain, you also reduce the potential for agents to color outside the lines and create chaos.

AI Development Frameworks Components

I believe that ADFs are a critical part of an enterprise AI strategy. That said, this is still nascent technology and in some ways not fully realized. Yet the industry has reached a certain level of AI maturity, which gives us a sharper focus on the key components an ADF needs. This is a very helpful milestone, as we can now more fully consider the relative strengths and weaknesses of different ADF solutions. This will also help illustrate where ADFs need to further evolve.

So what are these components?

• Practitioner Services — It was a very deliberate decision to call this product category a “development” versus “developer” framework. That is because, unlike other types of middleware, there are many different roles that will interact with an ADF. For instance, Google’s Vertex supports many different types of tools, including an IDE, Python notebooks and the recently announced Agentspace no-code toolset. But it’s a lot more than just support for tooling. There are also testing spaces and features, workflow designer tools, orchestration tools and advanced prompt engineering tools.
• Data and Integration Services — Data and integration services drive not only how data gets into a model using techniques like RAG, but also how data can be shared between models in an agentic application. Additionally, as we see more machine-based integrations with agents (much like we saw in Nvidia’s CES keynote), this layer will also cover interactions with edge and hybrid deployments.
• Model Services — Model services are in many ways the heart of the ADF. Model services are what enables ADFs to support integrating applications with multiple models. This also includes across-the-board support for evaluating and comparing models to ensure that developers pick (and buy time with) the right model based upon the application’s needs. Beyond these common features, this area has received the most recent innovation, including richer fine-tuning, multimodel support and model distillation.
• Management Security and Governance — As is typically the case, this part of the model is a bit more abstract. I do not expect to see most of these functions built into the framework. But I do think there needs to be a high degree of integration with ADFs and the IT control plane. So while I believe an ADF may require some degree of role-based access control, the authorizations should be stored in other solutions. As of early 2025, this is the area of ADFs that could use the most improvement.

The ADF Inflection Point

The evolution of generative AI and the recent rise of agentic development places AI practitioners at an inflection point. While AI has provided a great catalyst in terms of how digital transformation and automation can be scaled, there are still some foundational IT principles that need to be considered. This is where AI development frameworks have emerged and are showing potential in delivering value to less bleeding-edge enterprises.

However, this is still a new product category, and the level of maturity and capability varies from solution to solution. In part two of this study, I will examine the current state of ADF, the players and where I think the category needs to go next.