The 8B LLaCO model takes a somewhat different approach: instead of generating text one word at a time from left to right, it performs diffusion in continuous embedding space, denoising step by step. In their arXiv preprint, the team points out a longstanding problem with continuous diffusion: embedding vector norms decay, and measuring token competition with Euclidean distance can produce “false overtaking”—making it look as though a candidate token has surpassed the source token when the distance is simply being miscalculated. They formalize these pairwise competitive relationships as Competition Threshold Theory (CTT), and use it to propose Semantic Retention Scheduling (SRS), which allocates time across denoising stages according to the rate at which token identity decays. On the model side, they use an embedding-direction adapter and timestep-conditioned modulation to connect the pretrained backbone to continuous representations. They also change the noise process to spherical geodesic interpolation toward a mask-biased endpoint, avoiding norm decay. According to the abstract, LLaCO outperforms previous continuous-diffusion baselines and matches discrete-diffusion models of the same size. The preprint has not yet undergone peer review, so its experimental findings should be treated as preliminary.

$BTC