How much does it cost build a conversational artificial intelligence
Join our daily and weekly emails to receive the latest AI updates Learn More
More that 40% of Conversational AI is the fastest-growing gen AI technology Many marketing leaders that I have spoken to are stuck at They’re worried about spending too much money on a They’re worried about spending too much money on a new and uncharted technology.
Companies can certainly buy off-the-shelf conversational AI tools, but if they’re going to be a core part of the business, they can build their own in-house.
To help lower the fear factor for those opting to build, I wanted to share some of the internal research my team and I have done in our own search for the best LLM to build our conversational AI. We looked at different LLMs and what you can expect We consider these two models to be of the highest quality. How do you calculate LLM cost for conversational AI? Set up costs include all expenses required to set up the L When it comes to setting up, your cost-to value You may pay a premium to get a model with It may take weeks to get Llama 3 set up, during which time you could already have been fine-tuning a GPT product for the market.
However, if you’re managing a large number of clients, or want more control over your LLM, you may want to swallow the greater set up costs early to get greater benefits down the line.
When it comes to conversation processing costs, we will be looking at token usage, as this allows the most direct comparison. LLMs such as GPT-4o or Ll Some LLMs calculate tokens per word, sub words Some calculate tokens per word, per sub words, per character or other variations.
Because of all these factors, it’s hard to have an apples-to-apples comparison of LLMs, but we approximated this by simplifying the inherent costs of each model as much as possible. We found that GPT-4o was cheaper initially, but Let’s get into why, starting with the setup considerations.
What are the foundational costs of each LLM?
Before we can dive into the cost per conversation of each LLM, we need to understand how much it will cost us to get there.
GPT-4o is a closed source model hosted by OpenAI. Due to this, you only need to set up your tool The setup is minimal. The model components can be downloaded for free by your business. Unless you’re purchasing your own servers, which is relatively uncommon to start, you have to pay a cloud provider a fee for using their infrastructure — and each different provider might have a different way of tailoring the pricing structure.
Most of the hosting providers will “rent” an instance to you, and charge you for the compute capacity by the hour or second. AWS charges for its ml.g5.12 Amazon Bedrock calculates the cost based upon the number Bedrock is a managed, serverless platform by AWS that also simplifies the deployment of the LLM by handling the underlying infrastructure. Beyond the direct costs, to get your conversational AI operating on Llama 3 you also need to allocate far more time and money towards operations, including the initial selection and setting up a server or serverless option and running maintenance. The time it takes to deploy the LLM, as well
What are the costs per conversation for major LLMs?
Now we can explore the basic cost of every unit of conversation.
For our modeling, we used the heuristic: 1,000 words = 7,515 characters = 1,870 tokens.
We assumed the average consumer conversation to total 16 messages between the AI and the human. The input is higher due to prompt rules and logic. (The input is a lot higher due to prompt rules and logic).
On GPT-4o, the price per 1,000 input tokens is $0.005, and per 1,000 output tokens $0.015, which results in the “benchmark” conversation costing approximately $0.16.
GPT-4o input / output
Number of tokens
Price per 1,000 tokens
Cost
Input tokens
29,920
$0.00500
$0.14960
Output tokens
470
$0.01500
$0.00705
Total cost per conversation
| $0.15665 | For Llama 3-70B on AWS Bedrock, the price per 1,000 input tokens is $0.00265, and per 1,000 output tokens $0.00350, which results in the “benchmark” conversation costing approximately $0.08. | Llama 3-70B input / output | Number of tokens |
| Price per 1,000 tokens | Cost | Input tokens | 29,920 |
| $0.00265 | $0.07929 | Output tokens | 470 |
| $0.00350 | $0.00165 | ||
Total cost per conversation
| $0.08093 | In summary, once the two models have been fully set up, the cost of a conversation run on Llama 3 would cost almost 50% less than an equivalent conversation run on GPT-4o. However, any server costs would have to be added to the Llama 3 calculation. | Keep in mind that this is only a snapshot of the full cost of each LLM. Many other variables come into play as you build out the product for your unique needs, such as whether you’re using a multi-prompt approach or single-prompt approach. | For companies that plan to leverage conversational AI as a core service, but not a fundamental element of their brand, it may well be that the investment of building the AI in-house simply isn’t worth the time and effort compared to the quality you can get from off-the-shelf products. |
| Whatever path you choose, integrating a conversational AI can be incredibly useful. Just make sure you’re always guided by what makes sense for your company’s context, and the needs of your customers. | Sam Oliver is a Scottish tech entrepreneur and serial startup founder. | DataDecisionMakers | Welcome to the VentureBeat community! |
| DataDecisionMakers is where experts, including the technical people doing data work, can share data-related insights and innovation. | If you want to read about cutting-edge ideas and up-to-date information, best practices, and the future of data and data tech, join us at DataDecisionMakers. | You might even consider contributing an article of your own! | Read More From DataDecisionMakers |
 | |||