Table of Contents
Const has announced Teutonic-II-110B, a new permissionless training run for a 110-billion-parameter language model on Bittensor Subnet 3, expanding the subnet’s loss-based training market less than two weeks after the release of Teutonic-I 10B.
The new model is initialized from a modified, compute-efficient sparse Mixture-of-Experts architecture based on MiMoV2. Teutonic has published the genesis checkpoint on Hugging Face, and Const said the run will be open source at every checkpoint, including the final dataset.
In his post, Const noted that the project is scaling “10x beyond our previous model and 50% larger than any prior decentralized run.”
From Teutonic-I to a 110B Training Run
Teutonic-I 10B was produced during a 70-day open competition on Bittensor Subnet 3 from June 2 to August 10. Instead of requiring participants to train a model synchronously or contribute gradients to a single shared optimizer, the subnet used a sequential “king-of-the-hill” process.
At any point, the network maintained an incumbent model checkpoint, known as the king. Miners could download the current checkpoint, train a challenger using their own data, hardware, algorithms, and infrastructure, then submit that challenger for evaluation. If the challenger showed a statistically robust loss reduction against the incumbent, it became the new king. If not, the incumbent remained in place.
The Teutonic-I run completed 2,163 duels and accepted 203 coronations, an acceptance rate of 9.3%. The best checkpoint, king #191, reached a 62.28% unweighted mean across 11 reported benchmarks. The report said that score exceeded Quasar-Preview 18B at 59.78% and Covenant 72B at 57.55%, while leading on 8 of the 11 tasks, including an MMLU score of 75.29%.
Earlier this year, we covered Teutonic’s 80B model training effort on Bittensor, which helped position Subnet 3 as one of the clearest experiments in large-scale decentralized training.
Bart Hillerich
Teutonic-I then gave the subnet a benchmarked 10B release. Teutonic-II now attempts to combine that mechanism with a significantly larger sparse model and a fresh open competition.
How Teutonic’s Loss-Based Market Works
Teutonic-II is a “loss based market” that pays for model updates produced by “any compute, any individual, any algorithm.” The subnet does not reward participants simply for renting GPUs, running a prescribed script, or claiming to have trained a model. It rewards submitted checkpoints that measurably improve the current model under the subnet’s evaluation process.
That makes Teutonic different from conventional distributed training. In a normal large-model training run, one organization typically controls the architecture, data mixture, optimizer, training schedule, infrastructure, and checkpoint cadence. Even when computation is distributed across many GPUs, the system usually remains centrally planned.
Teutonic decentralizes a different part of the problem: the search for improvements. Participants can compete by procuring compute, tuning algorithms, changing data strategies, merging models, injecting weights, or using other training approaches, as long as the submitted checkpoint improves the shared incumbent.
The technical report describes Teutonic-I’s selection rule as a paired evaluation against the current king. Both the incumbent and challenger are scored on the same sequences, and the challenger must clear a conservative bootstrap lower-confidence-bound test. During the 10B run, promotion required a statistically robust loss reduction above a minimum advantage threshold of 0.0015 nats per token, using a bootstrap lower confidence bound at α=0.001.
That strict gate helps prevent noisy or marginal wins from replacing the current checkpoint. It also means most submissions fail. In Teutonic-I, only about one in 11 completed duels became a coronation.
The reward design also matters. Teutonic-I paid subnet incentives to a five-checkpoint window: the current king and up to four previous kings. When all five remained registered, each received 20% of the allocated weight. That structure softened the winner-take-all dynamic while still requiring miners to earn their way into the reward window by producing a successful challenger.
For Teutonic-II, Const said the network can subsidize contributions with TAO rewards “north of 6k USD per day for updates.” In this design, crypto-economic incentives are the coordination layer rather than an add-on to the training process.
“We use a digital currency reward because we believe that we cannot rely on altruistic computational resources alone,” Const wrote in the announcement. “Only by coordinating compute and talent with the power of digital markets, like Bitcoin did, can we hope to out-compete centralized labs.”
Open Checkpoints, Open Competition
The Teutonic-I report stated that future competitions would target substantially larger models, potentially around 100B parameters, and that mechanisms developed during the 10B run would be documented separately. The August 24 announcement now turns that future direction into an active training effort.
Teutonic-II will be open source at every checkpoint, and the final dataset will also be released. And that's essentially what's at stake here: proving a network can with centralized labs by stitching together stranded compute and talent would fundamentally reshape today's AI landscape.
Const, seemingly taking a subtle jab at decentralization naysayers (can't blame him), put it this way:
“Decentralization is not a gimmick: The only thing larger than a closed cluster is a network joined together… the only thing better than a closed lab is the network which pays for innovation.”
The approach still raises open questions. Loss-based selection can reward real improvements, but it can also create pressure to optimize for the evaluation mixture. Financial rewards can attract compute and talent, but they can also incentivize checkpoint copying, benchmark gaming, duplicated effort, or hidden data use. The Teutonic-I report acknowledges several of these limitations and calls for stronger audits around provenance, contamination, compute reporting, safety evaluation, and checkpoint integrity.
Teutonic-II-110B is therefore both a model launch and an institutional test for Bittensor. If the run produces meaningful improvements at 110B scale, it would give Subnet 3 a stronger claim that loss-based markets can coordinate serious pretraining work. If it struggles, the result will still provide useful evidence about where decentralized training mechanisms encounter practical limits.
For now, Const’s announcement gives Bittensor’s training ecosystem a new benchmark to pursue.
Participants can review the Teutonic technical paper, inspect the genesis model, and follow the live project through Teutonic’s website.
