Const has announced Teutonic-II-110B, a new permissionless training run for a 110-billion-parameter language model on Bittensor Subnet 3, expanding the subnet’s loss-based training market less than two weeks after the release of Teutonic-I 10B.
The new model is initialized from a modified, compute-efficient sparse Mixture-of-Experts architecture based on MiMoV2. Teutonic has published the genesis checkpoint on Hugging Face, and Const said the run will be open source at every checkpoint, including the final dataset.
In his post, Const noted that the project is scaling “10x beyond our previous model and 50% larger than any prior decentralized run.”
