Nous Research announces Psyche decentralised training network

Nous Research, the open-model group known for its Hermes fine-tunes, announced Psyche, infrastructure intended to let large models be trained across geographically distributed, otherwise-idle GPUs rather than inside a single company’s data centre. The pitch was about who could afford to train frontier-scale models at all: Nous described increasingly centralised compute requirements as a barrier only large corporations could clear, with Psyche as a way to pool smaller, scattered contributions instead.

The system combined a peer-to-peer networking layer, built on the Iroh library using UDP hole-punching for direct node connections, with the Solana blockchain for coordinating state across participants without a central operator, and bloom filters to verify contributed work cheaply. Communication overhead — the usual reason distributed training across the open internet has been impractical compared with a tightly networked cluster — was addressed with DisTrO, a gradient-compression method Nous had developed with a related project called DeMo, alongside training pipelines designed to overlap computation and communication.

Nous said the network’s first run, named Consilience, would train a 40-billion-parameter model using DeepSeek’s multi-head latent attention architecture on 20 trillion tokens, which it described as the largest pretraining run yet attempted over the public internet.