Distributed training over consumer internet hits a hard wall: gradient sync becomes the bottleneck. In private datacenters you get fat pipes and sub-millisecond latency so gradients fly around fast. But on regular internet? The gradients from large models are so massive that transmission time exceeds actual compute time. Your entire cluster sits idle waiting for network I/O. Classic case where bandwidth and latency destroy parallelism efficiency. This is why decentralized training on consumer hardware is way harder than it sounds—network topology matters more than raw FLOPS when you're shuffling gigabytes of gradient data every step.