Distributed training across browser peers.

The experiment

The distributed-training workflow explores how participating browsers can coordinate a neural training task. The host application defines the model, task format and validation, while the session keeps participants and progress in context.

This note collects what happened when we ran it on real models: the choices that mattered, the results we measured and a few things that went wrong.

Merging the results: three methods, one model

Every round, each participant trains on its own part of the data and sends back new weights, and the coordinator merges them into the next checkpoint. The default merge is federated averaging (FedAvg). Two alternatives, FedAvgM and FedAdam, treat the change over a round as a step and continue it with momentum or with an adaptive step size.

We compared them on one model: a 2.3 MB byte-level transformer with 55 data chunks, five participants per round and three passes over the data (33 rounds). Every method started from the same weights, saw the same chunk order and was validated after every round.

The best settings reached FedAvg's quality in 20 rounds instead of 31, and after 33 rounds their validation loss was up to 38% lower. FedAdam started more slowly than FedAvg, and a step that was too small (0.003) never caught up. This is one model and one run, not a promise: measure on your own.

Teaching it arithmetic

Arithmetic is a good test for a small model: every answer is right or wrong, and anyone can check it. Our first attempt produced something else. After ten epochs the answers looked like this:

1+1=5
1+2=5
1+3=5
1+4=5
1+5=6
1+6=10

The answers stayed constant for stretches and then jumped, as if the model remembered a few phrases. The cause was in the tokenizer, not in the training. A byte-pair tokenizer trained on arithmetic learns to merge frequent neighbours, including digits with operators. This is how it cut one training example:

37+48=85<EOS>  ->  "3" | "7+4" | "8=8" | "5<EOS>\n"

The model never saw numbers as digits. It saw fragments such as 7+4 and 8=8 and had to memorize them like words, and the end-of-answer marker was glued to the last digit as well.

The fix needed no new code, only generator settings: a vocabulary without merges, so that every digit, operator and equals sign is its own token, and a real end-of-answer token. With that, the same architecture and the same data file reached the following, at about four seconds per epoch on a CPU (exact answers on 200 random problems from 0 to 99):

One caveat: that range holds almost every possible problem, so part of this may be memorization. Checking real generalization needs problems the model has not seen, for example three-digit numbers.

A momentum trap

The next surprise came from the merge method. We trained the corrected arithmetic model in the widget with FedAvgM and a package made of a single data chunk. By round 85 the model answered every problem with dashes, and its validation loss was 27.

We reproduced the rounds outside Weei with a small script. With FedAvg the same package reached a validation loss of 0.94 by round 20. With FedAvgM it climbed to 11 by round 30. With one chunk, a round is a whole pass over the data. FedAvgM treats that change as one step and adds momentum 0.9 on top, so a large step grows larger every round. The method is meant for many small rounds, like the 55 chunks in the comparison above. For a package with a single chunk, FedAvg is the safe choice.

Letting the model play

Distributed training no longer has to start from a dataset. A model can declare that it generates its own experience, as in reinforcement learning. The coordinator then hands each participant a number of episodes and a seed instead of a data chunk, participants play and train on their own episodes, and the weights are merged as usual. The same seed replays the same episodes, so a retried assignment repeats exactly.

The first model was the classic CartPole task. With two participants playing 32 episodes each per round, the greedy return went from 9 to the maximum of 500 within 25 to 30 rounds. Reinforcement learning of this kind is noisy: in some runs the evaluated return dipped to between 300 and 400 before it recovered.

We also tried the GPU in a hybrid form: the model plays its episodes on the CPU and the GPU runs the update. It works, and one update matches the CPU's to about 1.5 × 10⁻⁸. It is also about twelve times slower for this model. The network has only 114 weights, and sending data to the GPU costs far more than computing the update. Small networks belong on the CPU; the GPU has to earn its place on models large enough to need it.

A longer reinforcement learning story, with a game, is in How we trained our first RL AI.

What we would check first

• Look at how your tokenizer cuts the data before you tune anything else.
• Match the merge method to the size of a round: FedAvgM and FedAdam like many small rounds.
• Validate on data the model has not seen, and read some of its answers, not only the loss.
• Do not trust a single run, including ours.
• Keep small networks on the CPU.

Start small and verifiable

This is an experimental workflow, with results dependent on browser capabilities, network reliability and the workload. Begin with a small task and a controlled set of participants, then validate outputs before accepting them.

A peer session does not itself provide federated-learning privacy guarantees or trusted execution. Define which data crosses peer boundaries before running an experiment.

Explore the blueprint

Read the collaborative AI training solution and the distributed training widget documentation to plan a first experiment.