You think that the more you remember, the more you’re improving—but in reality, you’re digging a pit.
Today I read Google’s WikiSkill paper, and combining it with my experience building a quantitative system, I was deeply moved.
━━━ The paper’s core in one sentence ━━━
Turn failure into a permanent asset, but the execution strategy must be revisable.
Most people (including me) make the same mistake: mixing up “why the failure happened” with “what to do next” when taking notes. You write today’s failure reasons into your notes, tomorrow you adjust your playbook—but you don’t update the notes. After a month, you basically can’t remember what you were thinking back then.
There’s a data point in the paper that shocked me: a 9B small model with a structured manual directly outperforms a 27B plain run. When you add the manual to a 27B model, some tasks jump from 40% to 81%. The parameter gap isn’t nearly as big as the gap created by “how knowledge is organized.”
━━━ An even harsher point ━━━
To prevent going off the rails, a small model will use conservative patches that slam a top-tier model from 50% down to 18%.
I verified this in my own trading system. In the early stage, when the sample size was only 4, I banned a strategy direction—and it turned out that this “protective rule” may have intercepted every potentially valuable opportunity. With 4 samples, statistically it means nothing, but it survived as a “rule” and is still running in the system.
━━━ What I changed ━━━
Split all records into three layers:
Original trajectories → read-only archive; never modify
Lessons learned from pitfalls → only add, never remove; stored permanently
Execution manuals → patch anytime; can roll back
At the same time, tag every rule with a sample-size label. n<30 gets a warning, n<10 gets a forced downgrade—not deleted, but prevented from independently triggering decisions.
━━━ Remember this ━━━
Your playbook can fail, but your understanding of why you failed must not disappear along with it.
#量化交易 #交易系统 #knowledge management
Today I read Google’s WikiSkill paper, and combining it with my experience building a quantitative system, I was deeply moved.
━━━ The paper’s core in one sentence ━━━
Turn failure into a permanent asset, but the execution strategy must be revisable.
Most people (including me) make the same mistake: mixing up “why the failure happened” with “what to do next” when taking notes. You write today’s failure reasons into your notes, tomorrow you adjust your playbook—but you don’t update the notes. After a month, you basically can’t remember what you were thinking back then.
There’s a data point in the paper that shocked me: a 9B small model with a structured manual directly outperforms a 27B plain run. When you add the manual to a 27B model, some tasks jump from 40% to 81%. The parameter gap isn’t nearly as big as the gap created by “how knowledge is organized.”
━━━ An even harsher point ━━━
To prevent going off the rails, a small model will use conservative patches that slam a top-tier model from 50% down to 18%.
I verified this in my own trading system. In the early stage, when the sample size was only 4, I banned a strategy direction—and it turned out that this “protective rule” may have intercepted every potentially valuable opportunity. With 4 samples, statistically it means nothing, but it survived as a “rule” and is still running in the system.
━━━ What I changed ━━━
Split all records into three layers:
Original trajectories → read-only archive; never modify
Lessons learned from pitfalls → only add, never remove; stored permanently
Execution manuals → patch anytime; can roll back
At the same time, tag every rule with a sample-size label. n<30 gets a warning, n<10 gets a forced downgrade—not deleted, but prevented from independently triggering decisions.
━━━ Remember this ━━━
Your playbook can fail, but your understanding of why you failed must not disappear along with it.
#量化交易 #交易系统 #knowledge management