Filtering fine-tuning data for large language models—this time, by looking at gradients. SimpleLoRIF is a method proposed in a paper that simplifies gradient attribution by backpropagating only through the Transformer’s final four layers. This makes it possible for gradient analysis, previously practical only at small scale, to handle fine-tuning datasets with millions of examples.
The research team used it to remove the 20% of samples in Tulu 3 whose gradients were most closely associated with unsafe answers, creating SafeTulu. According to the abstract, training on SafeTulu improved GCG attack success rates by up to 22% on Qwen, Llama, and Olmo. It outperformed LLM judges and earlier gradient-filtering methods, while largely preserving—and even slightly improving—model capabilities.
One thing worth noting is what the removed data looked like: the abstract specifically mentions failed tool executions and narrative arithmetic problems. These may not appear harmful on the surface, making them difficult for standard content filters to catch.
This is an arXiv preprint and has not been peer-reviewed; the conclusions above are limited to the scope of the experiments described in the abstract. SafeTulu and a Hugging Face Space that lets users customize the removal rate are now available.
$BTC
The research team used it to remove the 20% of samples in Tulu 3 whose gradients were most closely associated with unsafe answers, creating SafeTulu. According to the abstract, training on SafeTulu improved GCG attack success rates by up to 22% on Qwen, Llama, and Olmo. It outperformed LLM judges and earlier gradient-filtering methods, while largely preserving—and even slightly improving—model capabilities.
One thing worth noting is what the removed data looked like: the abstract specifically mentions failed tool executions and narrative arithmetic problems. These may not appear harmful on the surface, making them difficult for standard content filters to catch.
This is an arXiv preprint and has not been peer-reviewed; the conclusions above are limited to the scope of the experiments described in the abstract. SafeTulu and a Hugging Face Space that lets users customize the removal rate are now available.
$BTC
