Decentralized training of foundation models is closer than widely believed despite concerns about low-bandwidth connections between nodes.
Naysayers doubt the feasibility of decentralized training due to slow internet speeds impacting communication during model building, but recent research shows promising results in altering distributed training methods.
Cost-effectiveness and model quality outweigh concerns about coordination and price in decentralized training environments.