Joyce.
Post
橙子Joyce
·
--
An Apple research team ran a reinforcement learning experiment with 9 models and 11 languages to find out: if the training data is only in one language, can the model’s problem-solving abilities be applied to questions in other languages?
The answer is yes, and the effect is quite noticeable. Training with only one language still makes the model stronger across many other languages as well.
For example, on a French test, training directly in French improved the average score by 25.6 percentage points. If you don’t train in French at all and only use Spanish questions for practice, you can still improve the score on the French test by 24.6 percentage points—just a 1-point difference.
What the model learns isn’t only problem-solving for a single language; it also learns some problem-solving methods that can be transferred and continued to be used in other languages. So in the future, if you want to strengthen the model’s Chinese reasoning, you may not need to remake all reinforcement learning data into Chinese.
For example, on a French test, training directly in French improved the average score by 25.6 percentage points. If you don’t train in French at all and only use Spanish questions for practice, you can still improve the score on the French test by 24.6 percentage points—just a 1-point difference.
What the model learns isn’t only problem-solving for a single language; it also learns some problem-solving methods that can be transferred and continued to be used in other languages. So in the future, if you want to strengthen the model’s Chinese reasoning, you may not need to remake all reinforcement learning data into Chinese.
Disclaimer: Includes third-party opinions. No advice. Binance AI may be used without guarantee. See T&Cs.