[EXPERIMENT 2] I built a chatGPT replacement so that I can see how much AI really costs

Yes AI costs a lot. Token maxxing is really a thing. Companies are spending massive amounts in AI token and do they really get the value they expect?

I can’t answer all of those questions today but I wanted to share my second experiment, which is in fact already a few weeks old but I use it consistently.

This is kind of a mix between an educational article and an experiment. Educational because I want to learn about how cost is related to tokens and what type of tokens that we are charged for when we send a request to OpenAI or Anthropic or any of the other labs that provide LLM models with API service.

The experiment because I wanted to set up a chatGPT replacement at the same time. To be honest, I set up this solution but as it operates as a local host, I still pay 20 or so euros per month for a chatGPT subscription. Probably I will get around to change this experiment to an IOS app (I built and published a few in the past) or to transfer it into Telegram like a Hermes agent or something.

The Educational Part

So first some learning. There are three type of tokens that we are charged for when we communicate with OpenAI/Anthropic:

Input tokens

Cache tokens

Output tokens

Input tokens are everything that goes into a request. In a simple chat, that means only the first question (and maybe the system/context prompt). If you ask a follow up question in the same chat dialog, all the previous information becomes input tokens.

Cache tokens. If the question is identical to another question all the tokens in the beginning that is identical, are considered cache tokens. Cache tokens are way cheaper. This is why it is important (especially in an API chat conversation), not to go back to change the past in some chat conversation before resending. In this case the price will be input token price for everything.

Output tokens is everything that comes out from the model. Interestingly, the newer models like chatGPT 5.5 or Opus 4.8 got reasoning tokens. Those tokens are apparently more demanding tokens but they are still charged under the output token pricing.

So the obvious ways to reduce cost are:

There are obviously many more strategies to reduce AI but already by being aware of the cost and how it is structured around the different types of tokens, will be a big improvement

Generally speaking, the real measurement is cost per valuable output. What I mean is basically how much do you pay in average for an output with value. If you have lots of token spend but you don’t get any tangible output, you are probably on a dangerous path as an enterprise.

I suspect this is the case for lots of businesses today

Anyway, enough of learning. The real reason you are here is probably because of my chatgpt replacement. As often, I did build it with Cursor AI.

The chatGPT Replacement

I have this very thing that I build everything with the same technology. I guess it makes me understand it better. I do know that it is probably not optimal but as I used it a lot before AI became popular, I feel like I understand it the best. And I am not ready yet to have AI create things that I don’t understand

So Django (Python) as a backend and html/javascript/css is the front. Because the incredibly easy way to manage data in django through its model structure, it just feels so good to build around it. Html/js/css is kind of a no-brainer for the frontend.

What was important for me as well, was to also include to get the token breakdown (input, cache, reasoning, output) for every conversation so that I can include that information throughout the tool.

You can see here below that I opted for a weird basic dark colour scheme. Basically just the standard Claude Sonnet choice but I didn’t bother to play around with it as it works well.

We have model choice at the top, conversation history at the right (just to be a little different from chatGPT :) and we have a hidden system prompt and the actual conversation

You can’t see the input question above but you can see many things related to cost. First of all, you can see cost per conversation on the right. The question to Fable was about 0.0842 dollars with a total of 1712 tokens. As we can see the Fable 5 conversation is selected, we can see at the bottom that it had 36 input tokens and 1676 output tokens. We can also see that it took 29.1 seconds to get the answer.

So this tool is pretty similar to the basic conversational agent that we got with chatGPT. Probably, at this point chatGPT is way more powerful because it visualises tables etc which is something I didn’t activate here.

Comparing Models Tokens and Cost

Some token and cost information just for fun on the basic question:

“Please explain the top 10 recommendations from MrBeast on how to build a viral youtube video”

As we can see below, Claude Fable 5 is by far the most expensive model out there at the moment. Secondly we have gpt 5.5. The popular glm 5.2 generates a massive amount of tokens but still comes across as cheap as it is really cheap per token. Claude Haiku is incredibly cheap considering what it can do.

Some additional comments. In a few days, the Fable 5 numbers were reduced quite some. You can see the screenshot earlier that the cost was 0.842. I am not sure if the reduction is because it was nerfed/updated to generate less output tokens or if it is just LLM output variance. I did test it on many other questions and the numbers now seems to be consistent across many conversations.

The most obvious cost reduction

I also wanted to share how much we can reduce cost just by asking the model to be concise. Same question as before, but one more sentence:

“Please explain the top 10 recommendations from MrBeast on how to build a viral youtube video. Please be concise and minimalist in your output”

We can see how the output tokens and cost changed for GPT 5.5 to about 60% less and cheaper. This is a massive change.

And the same schema applies for GLM 5.2 that was reduced with about 40%

Conclusion

We can easily produce a chatGPT replacement today for basic conversations. If you have cursor AI or Claude Code you can probably get it right in one or two prompts.

The API providers also share token details in the responses which means that we can calculate cost per question and conversation. Cost and token spend per conversation is really good as it introduces better understanding and you can clearly get a feeling of what you are ready to pay for a request and its answer.

I believe that the biggest learning is that we can drastically reduce cost by asking for concise minimalist answer to a question

Thanks for reading

Andreas