Union's CEO Samvit Ramadurgam is running a multi-model routing strategy for their dealmaking agents. Core approach: treat tokens as a commodity and optimize for cost-adjusted performance per task.
Architecture breakdown: 95% of high-volume work gets routed to lightweight models (likely GPT-3.5-turbo or Claude Haiku tier), while only ~5% requiring deeper reasoning hits the heavy models (GPT-4/Claude Opus tier). They're continuously benchmarking model performance against cost per task type, allowing dynamic model swapping based on workload characteristics.
This is basically inference cost arbitrage at scale. Instead of defaulting everything to the most capable model, they're profiling task complexity and routing accordingly. Smart move for production AI systems where token costs compound fast. The 95/5 split suggests most dealmaking workflows are pattern-matching tasks that don't need frontier model reasoning.
Architecture breakdown: 95% of high-volume work gets routed to lightweight models (likely GPT-3.5-turbo or Claude Haiku tier), while only ~5% requiring deeper reasoning hits the heavy models (GPT-4/Claude Opus tier). They're continuously benchmarking model performance against cost per task type, allowing dynamic model swapping based on workload characteristics.
This is basically inference cost arbitrage at scale. Instead of defaulting everything to the most capable model, they're profiling task complexity and routing accordingly. Smart move for production AI systems where token costs compound fast. The 95/5 split suggests most dealmaking workflows are pattern-matching tasks that don't need frontier model reasoning.