Came across the statement @OpenLedger about OpenLoRA and got stuck on one number. The project claims thousands of fine-tuned models running on a single GPU and up to 90% savings on deployment costs.
The whole idea here seems logical. LoRA has long been used for fine-tuning models without changing the entire weight structure. Instead of separate heavy models, lightweight adapters are created that can be plugged on top of the base model. So, keeping many of these adapters on one GPU is a totally feasible scenario.
But it's important to split two things here.
First - the mechanics itself. It's understandable and looks technically sound.
Second - the specific figure of “up to 90% savings.” And here’s where I have a question. I couldn't find independent tests or open benchmarks that confirm this exact result.
Because the difference between “can save up to 90%” and “consistently saves 90%” is huge.
And I’d be interested to hear from those who have actually worked with LLM inference: when you see such numbers, do you take them as an argument or do you wait for independent verification?
@OpenLedger #OpenLedger $OPEN
The whole idea here seems logical. LoRA has long been used for fine-tuning models without changing the entire weight structure. Instead of separate heavy models, lightweight adapters are created that can be plugged on top of the base model. So, keeping many of these adapters on one GPU is a totally feasible scenario.
But it's important to split two things here.
First - the mechanics itself. It's understandable and looks technically sound.
Second - the specific figure of “up to 90% savings.” And here’s where I have a question. I couldn't find independent tests or open benchmarks that confirm this exact result.
Because the difference between “can save up to 90%” and “consistently saves 90%” is huge.
And I’d be interested to hear from those who have actually worked with LLM inference: when you see such numbers, do you take them as an argument or do you wait for independent verification?
@OpenLedger #OpenLedger $OPEN