AI agents conquered the 2024: Unlocking data value
Learn More
Join us for our daily and weekly emails to get the latest updates on AI. If 2023 was the era of AI-powered chatbots, then 2024 will be all about AI agents. What started from Devin earlier this year grew into a full-blown phenomenon, offering enterprises and individuals a way to transform how they work at different levels, from programming and development to personal tasks such as planning and booking tickets for a holiday.
Among these wide-ranging applications, we also saw the rise of data agents this year — AI-powered agents that handle different types of tasks across the data infrastructure stack. The data agents were able to handle downstream tasks such as analysis and managing the pipeline while some handled basic data integration. This made things easier and simpler for enterprise users. Gen AI Agents have taken over data tasks
While agentsic capabilities were around for a while, they allowed enterprises to automate some basic tasks. However, the rise in generative AI took things to the next step. They also learn to improve their performance over time.
Cognition AI’s Devin was the first major agentic offering, enabling engineering operations at scale. After that, larger players started offering more tailored enterprise and personal agents powered with their models. In a recent interview with VentureBeat, Gerrit Kazmaier, the Google Cloud’s Gerrit Kazmaier, said that he had heard from many customers about their constant challenges, including automating manual data team work, reducing cycle times of data pipelines, and analysis, and simplifying data administration. Kazmaier said that Google’s core data infrastructure, BigQuery, was revamped with Gemini AI to fix this problem. Agentic capabilities allow enterprises to not only discover, cleanse, and prepare data for downstream apps — breaking down silos and ensuring consistency — but to also manage pipelines and perform analysis. This allows teams to focus their efforts on more valuable tasks. Gemini agentic capabilities are used by many enterprises, including Julo, a fintech company that relied on Gemini to understand complex data structures and automate the query generation process. Unerry, a Japanese IT company, also uses Gemini SQL generation in BigQuery. This helps its data teams to deliver insights more quickly. As the underlying models evolved, even granular data operations — pioneered by startups specializing in their respective domains — were targeted with deeper agent-driven automation.
For instance, AirByte and Fastn made headlines in the data integration category. AirByte launched an assistant which created data connectors in seconds from an API documentation. The latter, meanwhile, enhanced its application development offerings with agents that generate enterprise-grade APIs – whether it’s to read or write information on any subject – using only a natural language description. Altimate AI in San Francisco, on the other hand, focused on different data operations, including documentation, tests and transformations. Its new DataMates technology used agentic AI for context extraction from all data layers. Redbird, RapidCanvas and other startups also took the same approach, claiming that AI agents can handle 90% of data tasks in AI and analytics pipelines. Agents powering RAG
Agents are also used to automate downstream workflows. For instance, the team behind vector database Weaviate recently discussed the idea of agentic RAG, a process allowing AI agents to access a wide range of tools — like web search, calculator or a software API (like Slack/Gmail/CRM) — to retrieve and validate data from multiple sources to enhance the accuracy of answers.
Further, towards the end of the year, Snowflake Intelligence appeared, giving enterprises the option to set up data agents that could tap not only business intelligence data stored in their Snowflake instance, but also structured and unstructured data across siloed third-party tools — such as sales transactions in a database, documents in knowledge bases like SharePoint and information in productivity tools like Slack, Salesforce and Google Workspace. With this context, agents can surface relevant insights to answer natural language questions. They also take specific actions based on the insights generated. Users could, for example, ask their data agents to upload their surfaced insights to Google Drive in an editable format. Much more to come
While it is possible that we have not covered all the applications of data agents announced or seen this year, there is no doubt: This technology is here for good. AI agents are expected to be adopted at full speed as gen AI models evolve. Organizations of all sizes and sectors will delegate repetitive tasks, regardless of sector, to AI agents. This will directly translate into efficiencies.
As evidence of this, in a recent survey of 1,100 tech executives conducted by Capgemini, 82% of the respondents said they intend to integrate AI-based agents across their stacks within the next 3 years — up from a current 10%. As many as 70-75% of respondents stated that they would trust an AI to synthesize and analyze data for them, and handle tasks like generating and improving code. The results of agents are currently not up to production standards, so a human must take over to adjust the work to their needs. This gap is likely to disappear in the next few years with the help of AI agents who are faster, more accurate, and less prone than humans to make mistakes.
So, to sum up, the roles of data scientists and analysts that we see today are likely to change, with users possibly moving to the AI oversight domain (where they could keep an eye on AI’s actions) or higher-value tasks that the system could struggle to perform.
Daily insights on business use cases with VB Daily
If you want to impress your boss, VB Daily has you covered. We provide you with the latest information on generative AI from regulatory changes to practical implementations so that you can maximize your ROI.
Thank you for subscribing. Click here to view more VB Newsletters.
An error occured.
