Over the past couple of days, I’ve been testing an open-source inference framework. Running the same model on a consumer-grade GPU, quantizing it to 4-bit tripled the speed, while quality dropped less than I expected. This has a very practical implication: many tasks that previously required the cloud can now be handled by a single machine. When the cost structure changes, the product form changes too.