[Experiment 8] Can a Hermes AI Agent improve his retail category manager skills by playing an online game?

I ended Experiment 7 by saying I was moving on to build agents again instead of another game. And I did, kind of. Because this time I built a game, but not for a human. I built a game for an agent to learn by playing.

The idea goes back to something I have been circling around since some time. You play, you get the feelings, the concepts stick in a way reading docs or watching Youtube videos never does.

So my question for this experiment was a simple one: if playing a game teaches a human how a process or a subject works, can playing the same game teach an AI agent how a real job works?

And especially I want to see if I can move an AI LLM from providing information into making decisions.

Before I did any work, I asked my basic Hermes agent to play my data center game (experiment 3) until it reached 100$ in score. Something it did relatively easy. At this point I knew that it was possible to have an agent to play an online game.

So I let Fable 5 build me a game (in Cursor AI) of a real business process and then I let an agent play it. The job: retail category manager. The category: tomatoes.

Why the tomato retail category manager business process?

A few years ago, I was doing a consulting freelance project and the project was to look into the profitability of a supermarket. After getting all the data, I concluded quickly (with the owner) that most of this supermarkets profitability came from fruits and vegetables category. And within that category, tomatoes were by far the most impactful product.

The supermarket could literally double the profit if the staff managed well the tomato category. Unfortunately for them, it was not the case.

There are multiple parameters to consider for tomato management in a supermarket. Purchase prices fluctuates drastically from day to day. Suppliers don’t have capability to deliver the full request every time. Tomatoes rot after a few days. Sales price to customers in the store can make customers leave the store if it is too expensive, but setting the sales price must consider the multiple recent purchase prices.

As you can see, on a simple category like that, there are quite a few possibilities to mess up. On top of that, in real life there is not one person managing a tomato category at full time. Typically there is one person managing the whole fruit and vegetables section. And this is most likely done by a temp worker as tomatoes are mainly fresh during a certain part of the season when some regular employees are on holidays.

I didn’t select tomatoes because of the complexity though. I select tomatoes because I have already looked into it before AND, I have access to years of real transactional data so I am planning at some point to put through the agent on a real test (but not in this experiment).

The Technical Stuff

Same stack as always, so I can be quick. Django backend, HTML/JS/CSS frontend. However, this time a big difference. To avoid cheating by the agent, the game logic is now in the backend. Game is built with Cursor AI and Fable 5 in about an hour. It is quick because I have already built game logic through backend in the past.

The game is free to play for anyone (including humans) at https://www.buildlooplabs.com/tomatoes/

For the purpose of the game, there was no API endpoint, instead the agent had to surf the webpage to navigate and make decisions. Game is not representing reality because it is highly simplified but I do believe that in most cases over the next year, if I want to deploy this agent somewhere else, an agent should learn to act by seeing what the human sees and not be requiring some special API.

As a comparison, I am thinking of how Elon Musk always pushes for his Teslas to have only camera visions and no Lidar. This feels the same. Let the agent be like a human and he can train in the same environment instead of giving him special equipment

The Gameplay

The game itself is a simple business. You run one supermarket tomato category for a 60-day season. Every day two suppliers quote you a cost per kilogram. You set your sales price and you place an evening order. The delivery arrives the next morning and you pay the price from the day you ordered, not the delivery day. Your score is the simplest possible business metric: all the revenue you take in, minus all the money you paid to suppliers. That’s it. Profit.

The fun part happens when you take the pricing and the stock together, because you get the two constraints that make this job actually hard:

- Demand follows your price, not your shelf. Roughly, the more you charge above what the cheap supplier quotes, the fewer people buy. Charge too much and nobody buys at all. You cannot sell more by stocking more — once your price caps demand, a full shelf does not help you.

- Tomatoes rot. Three days including arrival day, then they get binned. Customers take the oldest first (a game thing for simplicity but also in real life, the good employees keep the freshest products out of reach from customers in some temporary storage). Anything you over-ordered that does not sell is not just unsold — the whole cost of it is pure loss. You paid for it and then you threw it away.

So as a player you fight the classic retail war between two ways to lose money: over-stock and rot the margin, or under-stock and miss sales. Miss in sales is much more brutal in real life because in real life a customer can leave the store and you therefore miss the whole basket. While in this game you only miss the tomato revenue.

The game differs from the other learning games because this time I did set up a high score by the end of the game. And the end of the game is defined after 60 turns (days) so there is really a proper comparison that can be done. In the other game there was no time limit. In this game, an AI business agent therefore knows well in the end if he played through it well or not.

The Business Side

Let me mention two words about why I think this matters and why it is the direction I want to push.

For the last seven experiments I kept building games “to learn something myself.” That is a nice pattern and I am not done with it, but it put the human in the center of the loop. The flip side is much more interesting for a business: the day the ai agent is the one being trained on the exact same conditions as humans do work today, you have a way to add value and to automate real processes.

Walk through a modern company. The demand planner gets rough forecasts and messy supplier emails. The category manager gets capacity changes as text in a slack channel. The procurement person reads quotes out of an old system that nobody has touched in ten years and then they speak on the phone to the suppliers with no official traceability.

So the AI agent learning idea is: build a game that re-creates one of those real jobs, in the messy way the job actually feels — prose over structure, no clean API — and let the agent play it until it reproduces the work. The tool the company already has becomes the playfield, and the agent learns the process through the same clicks a human does. That is automation you can actually deploy on the software that exists, not the software we wish existed.

Tomatoes turned out to be a perfect first case for this because its simplicity in understanding and its non calculated edge cases. Perishable retail, price decisions, forecast noise, a hard capacity constraint that punishes you if you ignore it. If an agent can learn the perishable tradeoff in a browser, learning a less cruel version of the same tradeoff in a real category is a smaller step than it sounds.

Now in the tomato game, I have excluded the slack and email discussions. Mainly because I wanted to try a simple scenario first and I want to nail a few things before I push the boundaries for the agent.

How the agent did

The agent was a Hermes agent and I didn’t try to tailor it before the game. It was running on the DeepSeek Flash model 0731 which is a cheap model but capable.

I asked it to run 3 times in total for the experiment

In run 1, the agent spent lots of time to navigate the website. It clicked on things but it didn’t really understand much how the game, the decisions and the score were connected. It also lost its save a few times because it didn’t save the game. In the end of game 1, the agent achieved a score of 3001$ profit but it failed to save it.

In run 1, It achieved the score by doing like a dumb IT software which means, it applied a few simple business rules like always order 40Kg per day and set the sales price as 2 x the cheapest price of the day. It kept an inventory of around 90kgs as a strategy towards the end of the game

After run 1, the Hermes agent saved everything in a web browser markdown file skill which was not really satisfactory from my side as tomato and retail learnings were grouped with browser navigation

In run 2, the agent was not given any instructions and it decided to begin with the same inventory and price setting strategy as in run 1. This strategy was adjusted half way through and made it to 3564$ in final profit. That was quite an improvement.

In run 2 it adjusted some of its strategies along the way. The adjustment was that it reduced the inventory to around 40kg-60kg. It also adjusted the pricing strategy to make it 75% based on cheapest quote and 25% based on oldest stock and then the price was a 2 times multiplier of that weighted cost.

After the second run, I asked it to divide the learning skill into two parts, technical skills (basically navigation) and business skills (like the rules it created for reordering, sales price etc)

In the run 3, I really don’t know why but it decided to go for a low inventory approach and it tanked the score so it achieved 2445$. It was out of inventory lots of times and as a consequence it missed some sales

To summarize the three runs, it found a strategy in run 1. It improved it in round 2 and got a better score. It tried a different approach in run 3 to see if it could go further, and failed.

The total of the 3 runs costed 1.61$ in API cost which is still a lot for being Deepseek model. Mainly because the agent had problems to connect with the browser

What I actually learned doing it:

- It validates my hypothesis that An AI agent can genuinely learn the business rules. Playing the game well means internalizing the price-cap and the rot/perishable constraint, and there is a clear, learnable, optimal strategy in there.

- It can play a game like a human. The agent worked through the same UI a human does, so all the “how does it access the game disappeared. It just had to read the page and act. That felt more real.

- I have a lot to learn about memory structure for the agent. I feel that only a real expert in a field can create a memory structure that supports an agent to learn and become better than a human

- Build games humans and agents can both play. The minute the same game teaches a human and trains an agent, your learning tool and your training data are the same artifact. That feels like the overlap I want to live in and explore further. I can also play the game and test how it works.

Improvements for next version

There are two aspects to be considered and maybe improved for a next version: Improve the game simulation, Improve the AI agent’s way of operating

Game simulation

• There could be more rough discussions and realistic communications and inputs for the agent. Like for example, we could simulate slack software or email communication with suppliers
• The agent should be trained in an efficient API environment, and then learn the online navigation separately. This would increase the speed probably
• Add multiple products in the same category, then add multiple categories. In short, the agent will in the end manage the whole supermarket
• Suppliers can have capacity constraints like if they get too much order over a week, they will not be able to deliver
• Player can be asked to deliver a forecast to the supplier and therefore negotiate longer volumes. This could lead to penalties if the forecast is not delivered
• There can be surprising supplier disruptions. And if a supplier didn’t get lots of orders, he might not deliver
• Cost of missing sales can be more expensive (due to missing customers full basket)
• Place the orders more realistic like in the morning for the following day. Which means there is more leadtime to be considered
• In a multi product environment, we can have limited total space so certain products get more visible space and volume in the store than others

AI Agents way of operating

• Define memory into a clear tailored structure for the agent. This should cover both core experience and edge cases. It should also cover historical memories that should be known and just surfaced at the right moment. Human power is also that we know history of a subject and decision which provides a better holistic view (context) to a subject. This needs to be transferred to the agent
• I should dive deeper into how agents navigate online and look into various frameworks around potentially improving this
• Basic supply chain topics like forecasting, re-ordering formulas, inventory safety stock calculations can be loaded into the agent before
• I could do like in AI and do human reinforcement learning. Maybe in this context it will mean that I with some experience in retail category management play the game a few times and then AI can observe from how I approach it. (This reminds of the approach Deepmind took with AlphaGo and then with Alphazero. Maybe I can look into that again).

Reflections and conclusion

Seven experiments ago I built a job-search agent that is still running on my Hetzner VPS. A lot of games later the loop finally closed: I built a game not to teach me, but to train an agent on the same interface a human does. It is the same funnel I always believed in — play it, learn and improve the skill — just with the agent on the other side of the game now.

Now the question is, could I have asked Fable 5 to create the skills for an Hermes agent instead of letting the agent play, I am not sure. It is difficult to create skills including edge cases compared to when simulating the environment and how they arrive like in real life

I could really build lots of products around this. Understand the business task, build the simulation, set up the agent and its learning needs, train the agent by letting him play, transfer into real life tasks

And honestly? A whole real (still simplified) job, reduced to a 60-day season of tomato pricing, is more fun to watch an agent struggle through than any of the tycoon games so far. If an agent can learn and improve his retail category skills not to let the tomatoes rot, there is hope for the rest of us.

Now let’s see if any humans can beat the agents high score?

Cheers

Andreas