I recently wrote an article exploring how AI impacts startups’ unit economics. It’s a hot topic - covered by a16z’s recent post as well. Inspired by the buzz, I decided to dive deeper and do some real-life math on what those costs imply if someone were to interact with AI continuously - 24 hours a day for a month.

Breaking down the numbers

I started with the cheapest model out there - Together AI’s lowest pricing tier is $0.06 per 1 million tokens. Here’s how the numbers add up:

1. Together AI’s Pricing

•Assumptions:

   •Conversation speed: 60 words per minute. 

   •Word-to-token ratio: 1 word equals 1.3 tokens.

   •Continuous interaction: 24 hours a day, every day.

•Daily token count: 60 words/min × 60 min/hr × 24 hr / day × 1.3 tokens / word = 112 320 tokens/day

•Monthly token count: 112 320  tokens/day × 30 days = 3 369 600 tokens / month

•Monthly cost:   (3 369 600/1 000 000) × 0.06 = 0.202 $/month

For $0.20 per month, you could talk to Together AI’s model non-stop, 24/7. Pretty cheap, isn’t it?

2. OpenAI’s most expensive model

Now, let’s contrast that with OpenAI’s most expensive pricing tier: $120 per 1 million tokens.

•Monthly Token Count (Same as Above): 3,369,600 tokens.

•Monthly Cost: 3 369 600 / 1 000 000 × 120 = 404.35 $/month

 At OpenAI’s rates, a similar 24/7 usage would cost $404.35 per month. That’s 2,021 times more expensive than Together AI’s lowest pricing tier.

                                             Most of the other vendors fall somewhere in between these final costs.

Costs improvement from data curation

Just before posting this I saw this post from Datology (startup developing data curation technology for optimizing the use of AI). RO founder btw!

A new frontier in AI efficiency lies in data curation. By curating high-quality training data, they’ve shown remarkable gains in training efficiency, performance, and inference costs.

Highlights from Datology’s results:

   1. Train faster: By curating the training data, they achieved the same baseline performance 7.7x faster. This drastically reduces the cost of obtaining results and speeds up iteration cycles;

   2.Train better: DatologyAI’s curated dataset improved performance by 8.5 absolute percentage points over the baseline (60.5% vs. 52.0%);

   3.Train smaller: With better data, smaller models could be trained to outperform larger ones, cutting inference costs by 2.1x while still improving performance over the baseline by 5.7%.

These optimizations reduce the operational costs of AI significantly. For businesses training or customizing their own models, these savings in time, cost, and performance are game-changing.

What this means for the AI landscape

This comparison highlights the race to the bottom in AI pricing. While Together AI charges a fraction of a cent for tokens, OpenAI’s pricing—while justified by model quality and features—caters to a different market tier. 

Key implications:

   1.Cost vs. Quality: OpenAI’s higher pricing reflects the advanced capabilities of its models, which may be indispensable for enterprises with high-stakes use cases. Together AI, meanwhile, offers a highly competitive option for more basic or volume-based use cases.

   2.Accessibility: Together AI’s low pricing opens doors for startups, small businesses, and hobbyists to explore AI without worrying about costs.

Reflecting on unit economics

This exercise ties directly back to my earlier article on startups’ unit economics. If AI becomes this cheap to operate at scale, it’s a game-changer for businesses relying on AI-driven services. Customer service, product recommendations, and even personal assistants could become standard offerings, as costs would no longer be a barrier.

Yet, this also poses questions:

•Will providers like Together AI rely on massive adoption to sustain such low pricing?

•Can premium providers like OpenAI justify their pricing over time, or will they also need to adapt to the downward pricing pressure?

Closing thoughts

The math here is pretty interesting in my opinion. On Together AI’s pricing, 24/7 interaction costs just $0.20 per month, compared to $404.35 per month with OpenAI’s most expensive model. This underscores how competitive the AI space has become.