[Experiment 10] Can an AI Agent learn to allocate carrier capacity like a transport volume manager?

Introduction

My experiments are slowly moving from generic AI Agent experiments and learn AI through games, towards more supply chain specific tasks for agents to handle.

This is of course a niche that suits me well, as I have 20 years experience in the supply chain field.

In experiment 8 and experiment 9 the job was retail category manager. Tomatoes. First one product, then three. The agent had to buy, keep inventory, set a price, and not let the tomatoes rot.

Experiment 9 showed that a Hermes AI agent can learn that job. So I felt it was time to leave the supermarket.

The next job is a transport volume manager. This sits close to multiple roles that I have seen during my years of working.

What this means is simple. Every week you have parcels. You have multiple truck companies. You must split the parcels between those companies, every week, for a full year.

From experiment 9 I also felt the setup had to be pushed forward. There are two ways to do that:

• Build a better and more realistic simulation
• Improve the agent

I have opted for improving both. The new simulation is about giving parcels to carriers. Regarding the agent, I have once again guided the agent to write down what he learns in one document per run. There is now also a living strategy (guidance) file. The last important document is a hypothesis log that the agent is adding thoughts to as needed.

The game is easy to explain but difficult to play. Six delivery regions. Around 30 carriers. 52 weeks. Each week is one game turn. Basically there are two numbers that fight each other: how much each parcel costs, and how often the carrier makes a mistake. Volumes are allocated to the carriers and the volume can sit on a contract or can simply be bought on the spot market.

So the question is: can an AI agent run a whole year of this job well, and keep improving when he plays it again?

The Problem

I believe that transport volume management is one of the more complete problems in supply chain. It is kind of like same importance as the retail category management job from the previous experiments

Here is why.

In the game, PackDel earns $6.58 for every parcel that is delivered. That number does not change if the parcel went with a cheap carrier or an expensive one.

So you only make money if you pay the carriers as little as possible, and still get every parcel delivered, without too many mistakes.

That last part is the tricky. You push the prices too much and providers will start taking shortcuts which can result in delivery problems.

If you cannot find a normal carrier for some parcels, those leftover parcels go to a last-resort carrier. In the game he is called SwiftRelay Spot Network. He is expensive. A normal parcel that costs about $4.00 to carry still makes about $2.58. A leftover parcel that goes to SwiftRelay at $9.34 already loses about $2.76, before you even count the mistakes. He also breaks more parcels.

That is how a week that looks "saved" can still destroy the profits of the year.

The two numbers also pull against each other. The cheapest carriers usually make the most mistakes. Late parcels and damaged parcels are a hidden cost. A late parcel costs $4.14 back. A damaged one costs $26.80. You think you bought cheap. Then you pay it back in customer complaints.

There are a few other ways to reduce the profit:

• Leftover parcels going to that expensive last-resort carrier, because you did not cover the week
• Contract penalties, when you promised a weekly volume and then you did not send it
• Error and damage costs, sitting under the cheap rate
• Capacity that is half used, or fully loaded on one region while another region is starving

The six regions also cost differently. Midtown Core is about ×0.95. Westfield Plains is about ×1.28. Westfield is rural. Cheap trucks are thin there. The same carrier is not the same money depending on where he drives.

So we are answering this: can an agent learn to run a whole year of this well, and keep improving when the next year looks different?

The Hypothesis

I wanted to see two things.

First, that we can build a more-real life-like simulation than the tomato game.

Second, that the Hermes AI agent plays better if he writes things down.

Success is judged by me (Andreas) and the key number is profit; however, similar to in real life, if the reasoning or explanation is good, I can still find it a success.

The reasoning is a judgement call. Did he understand why the score moved, or did he just get a lucky year.

The writing is the extra bet. I asked him for several files.

One markdown file for every year he plays that includes Strategy at the top and the decision for every week. Then the lessons learned at the end.

A living best-strategy document. He may update it only when there is evidence. Not because the last year felt bad.

And a hypothesis log. This one matters, so I will explain it.

The agent starts with a list of ideas that he brainstormed himself. H1, H2, H3, and so on. Each idea is a small change he could try. He is not allowed to delete an idea. He may only change the status line: NOT TESTED, TESTED, or VALIDATED. When a year is over, new ideas get added at the top. The old ones stay.

The point is simple. Any idea get the status line added to it and then if it is validated, it is also added to the strategy document

I believe that if the agent writes it down properly, the next run is not a reset but a continuation.

The Technical Setup

In order to have the best outcome, we need a game which simulates well the real life, and we need an agent which learns and captures well all decisions.

The Game

The game is free to play at https://www.buildlooplabs.com/3plvolume/

You are the transport volume manager of PackDel. Every Monday for 52 weeks you split the week's parcel volume in six regions across roughly 30 third party carriers. One goal: end the year with as much profit as possible.

Technically it is the same stack as the other experiments. Django backend, html forms on the front. A web-form game. No special API for the agent. The game rules and game state lives on the backend server, so there is no possibility to cheat from the page.

On top of the six regions and the carriers, there are a lot of parameters:

• Weekly capacity that is real. Give a carrier more than he has, and the extra parcels come back
• Half-region caps. Some carriers can only reach half a region in a week
• Spot rates and contracts. A contract is a promise. You send a certain number of parcels every week, and you get a cheaper rate. Miss the promise and you pay a penalty
• A fuel index that moves the rates
• Seasonal demand, campaigns, and a year-end peak
• Disruptions: strikes, driver shortages, weather, depot problems
• Hidden error rates per carrier, shown week by week
• Relationships. Steady volume gets better quotes. Ignore a carrier and he serves someone else first
• The last-resort carrier, SwiftRelay, who takes whatever you failed to place

All rates, error rates, capacities, demand and events are new for every game. There is nothing to memorise between runs.

I think this is closer to a real transport job than the tomato game was. Still a simulation and still simplified. But overall this is a massively complex job simulation to play for a human or an agent.

The AI Agent

As often, I used Hermes agent for this experiment. I set it up on a Hetzner VPS and I activate a Telegram bot for my communication with it. It is quick and it works well.

I typically ask it to run 5 games at the time so I can monitor the cost of the experiment.

I have used DeepSeek model through OpenRouter. Cheap enough that I can let him play a lot of years for a few dollars, which this experiment clearly needed.

My guidance is tighter than in experiment 9.

I asked the agent to create a clean folder for the whole project. Technical navigation in one place. Business and play skills in another. If those two are mixed, the agent will learn the website and the job together in the same file, and then it is difficult to reuse the job skills in a similar scenario but different setup.

I asked him to write the three files I already described. After 35 runs we have 35 documents, one strategy document, and one hypothesis log. There is a massive amount of text to analyse. Luckily that is something an LLM is actually good at.

On top of the writing, our lovely ai agent built tooling to support himself. A small solver that picks the week's mix under capacity and coverage. A cost model that puts error cost into the price, because the cheap carrier is not cheap if he damages and delays. And a few tools he can use: how many contracts, how much extra cover, a seasonal boost for the rural regions.

The way he plays is the same idea as tomatoes. He drives the public HTML forms with Python urllib and cookies, plus the CSRF token. Not a browser window. Same page a human sees. Same forms.

The Outcome

Short Summary

Overall the experiment is a success.

The agent learned across 35 runs. Run 1 ended at $4,152,821.71. The best year, run 8, ended at $8,320,937.59.

There is an explanation and is therefore acceptable why the best run happened already in game 8 even though there were 35 runs.

When I asked him where the money went in year 1, he wrote it like this:

• CONTRACT PENALTIES $1,389,438 (by far the worst leak). Auto-taking almost every offer (active contracts climbed to >20, some with large minima) created huge shortfall exposure when regional demand dipped below my commitments. This single line ate ~$1.39M of profit - the #1 thing to fix.
• BACKUP cost $875,952 / 94,042 parcels (3.5%). Concentrated in Westfield (rural, very few cheap carriers); my cover buffer helped but the rural region still spilled.
• ERROR cost $842,365.

That is the same kind of summary he produced on tomatoes. It might sound foreign when you don't know the game but it is still better than what most humans can say.

The limit is this in the simulation. The score depends a lot on the year he was given, not only on how well he played.

On an expensive year, even a good run sits around $5.5M to $6.9M. The record years only happen when the year is cheap and big. Same way of playing, different year, different ceiling.

Detailed Outcome

The hypothesis log is the easiest way to tell the year.

He started with a catalog H1 to H10 that he created himself. A cost-min solver. Concentrate volume so relationships get cheaper. Leave headroom. Take many contracts. Take almost no contracts. Weight contracts by error. Never plan to use the last-resort carrier. Size for the peak. And a few others. Most of those first ideas were still sitting there as NOT TESTED.

Then he played year 1 with what he thought was the safest simple setup.

It was not safe. He auto-took almost every contract offer. Active contracts climbed above 20. When demand dropped, he could not fill the promises and he took a $1.39M penalty.

He also thought leftover parcels were growing every week. They were not. The number on the page was year-to-date. Weekly leftovers were small, about 260 parcels a week, mostly in Westfield. He wrote that down as his first real miss-understanding.

So of the money that was not normal carrier cost, penalties were 44%, last-resort leftovers 28%, errors 27%. Net $4.15M.

Run 2 isolated one thing which was the strict contract discipline. Hypothesis H12. He stopped auto-taking the offers.

Penalties went from $1.39M to $31k. That is minus 98%. The year landed at $5,760,247.47.

That is the first big lesson. A contract looks like a discount. It is only a discount if you actually send the volume every week. If you promise too much, you pay a second time.

Then he went after leftovers and errors.

Covering a bit above the forecast cut the weekly leftovers. The last-resort carrier is never the plan. Over-allocating is free. Under-allocating sends parcels to SwiftRelay.

Error was the third leak. He put a coefficient on error inside the solver. Run 6 used 1.8 and landed at $7.39M. Run 8 used 1.4 and became the record, $8,320,937.59.

On that record year the year-over card looked like this:

• Revenue $19.44M. Carrier $10.07M. Last-resort leftovers $0.043M. Errors $-0.99M. Penalties $-16k. Net $8.32M.

Leftover parcels were $43,285 / 4,725 parcels which was the lowest yet. Cost per parcel $3.78. Error rate 2.69%.

He also wrote, later, that 1.4 should stay the default. Never start at 2.0.

Run 13 is why. Same idea, coefficient 2.0, on an expensive year. The solver clung to the scarce cheap-and-clean carriers and starved volume. Final profit $4,332,479.03. Worst of that block of runs. He wrote: error_coef=2.0 is toxic when the cheap-low-error carriers are scarce/pricey.

So the three big tests in the log are really these.

1. Do not plan to use the last-resort carrier. Run 1 leftovers $875,952. Run 8 leftovers $43,285.
2. Do not auto-take 20 contracts. Run 1 penalties $1,389,438. Run 2 penalties $31k. Later years got down to a few thousand.
3. Put error into the price, but do not move the coefficient to 2.0 by default. Run 8 at 1.4 is the record. Run 13 at 2.0 on a hard year crashed.

After the record, he learned a fourth thing. The year itself caps the score. The year sets the ceiling, and he also changed how he played once he saw that.

On an expensive or small year he kept cover low and the error coefficient at 1.4. On a big year that was not too expensive, he pushed volume. Later he also started the rural boost at week 40.

Some years the rates are just high. Cost per parcel around $4.2 to $5.2. On those years you cannot reach 8M, no matter how clean the contracts are. Cheap and high-volume years are the only path to the record.

Run 24 is the proof that run 8 was not luck. He promoted volume on a big year. Final profit $8,216,923.02. Rank 2. Only $104k off the record. Highest revenue of any run, $20.43M.

Run 12 was a smaller, quieter test. He lowered the extra cover a bit, so he would not over-promise. Penalties $13,765. Final $6,932,729.63. The log marked that as VALIDATED.

Run 34 was the rural one. He started the Westfield and Lakeshore boost at week 40, not week 44. Final $8,042,902.71. Rank 3. Also VALIDATED.

The scoreboard of the best years looks like this:

• Run 8 — $8,320,937.59 ← BEST
• Run 24 — $8,216,923.02
• Run 34 — $8,042,902.71
• Run 29 — $7,520,160.48
• Run 6 — $7,390,137.57
• Run 5 — $7,271,744.49
• Run 31 — $7,270,877.83
• Run 10 — $7,155,875.90
• Run 32 — $7,073,256.48
• Run 12 — $6,932,729.63

And then the other end: run 1 at $4,152,821.71, and run 13 at $4,332,479.03.

One more thing about the log, because it is easy to misread. The first catalog, H1 to H10, is still sitting at the bottom of the file with some old NOT TESTED labels.

I also asked him to go experimental on some later years, same as in experiment 9.

Further Improvements

In terms of in the game, there are some ideas that were not tested. It would be interesting to play on to test them. The problem is that every time the agent plays, he also identifies new things to test so it never stops. A few things not tested:

• Concentrate volume on 2 or 3 carriers per region, and let the relationship discounts build (H2)
• A full year with zero contracts (H5). He only went down to about four
• Deliberately send some parcels to the last-resort carrier as a buffer (H9). The log already says this is probably a bad idea. It is still there so it can be proven wrong
• A cheaper set of low-error carriers (H16)
• Push the rural boost plus volume-promotion further, on a year that is both big and cheap, and try to beat $8.32M (H32)
• A fully automatic read of the year in week 1, so he switches style without me asking (H25 is only partly done)

I believe the next jump is not another 20 runs of the same style. It is isolating one of those leftover ideas per run, the way he did with contracts, then leftovers, then error.

In terms of general improvements, the game could include more regions, more complex situations and maybe more brutal change in market conditions

For the agent improvement, I believe we have now found a good structure. I do feel that the decision data should be classified in some kind of database so he can distinguish decisions related to edge cases and to every day cases.

I would also want to test this with different LLM providers like maybe change from DeepSeek to OpenAI or Anthropic model and maybe a really small model as well.

Closing Thoughts

Can an AI Agent learn to allocate carrier capacity like a transport volume manager?

The answer is YES.

With the limit I already wrote. He can learn the job. He can play a year. He can keep improving and he can provide clear explanations and reasoning why he acts like he acts.

We can imagine in an intermediate first step in real life that the agent does the job, but instead of deciding it tells the human what to do, and then the human validates.

In a second step, when the human will feel confident, the agent will act alone as trust is improving between humans and agents.

I think it is also important to mention that maybe these agents do mistakes, but humans do mistakes as well. Anyone that is working in supply chain or in any role knows that people make mistake all the time. Some are, of course, more costly.

Cheers

Andreas