Louis Kirsch (co-founder, Inherent Labs) pushes back on the pure autonomous RSI (recursive self-improvement) narrative. His take: RSI won't be a lone AI grinding itself to superintelligence—it'll be a hybrid org where humans and machines co-evolve.
The architecture he's proposing is essentially a closed-loop feedback system: humans guide optimization targets, machines execute and propose improvements, then both iterate together at increasing velocity. Think less "AI in a box improving itself" and more "human-AI symbiotic optimization loop."
Core insight: scientific problem-solving acceleration comes from tightly coupled human intuition + machine compute, not from removing humans from the loop. This frames AGI development as an organizational design problem, not just a model scaling problem.
Makes you wonder: if RSI requires human oversight to stay aligned and effective, does that cap the improvement rate at human cognitive bandwidth? Or does the human role shift from "decision maker" to "constraint setter" as the system scales?
The architecture he's proposing is essentially a closed-loop feedback system: humans guide optimization targets, machines execute and propose improvements, then both iterate together at increasing velocity. Think less "AI in a box improving itself" and more "human-AI symbiotic optimization loop."
Core insight: scientific problem-solving acceleration comes from tightly coupled human intuition + machine compute, not from removing humans from the loop. This frames AGI development as an organizational design problem, not just a model scaling problem.
Makes you wonder: if RSI requires human oversight to stay aligned and effective, does that cap the improvement rate at human cognitive bandwidth? Or does the human role shift from "decision maker" to "constraint setter" as the system scales?