💡 Our sixth paper is out!
FIDES-Live: Evaluating Behavioral Concordance in Sequential Trading Agents A
n AI trading agent can submit a valid order, have it rejected, and still report that the trade was completed. A correct account balance alone would not reveal this discrepancy.
In this paper, we examine whether an agent’s decisions, proposed orders, execution outcomes, and subsequent reports agree with one another. We introduce FIDES-Live to audit this behavioral concordance through the records produced during each trading round.
We audited 10 requested inference routes across 7,609 account-rounds in cryptocurrency and an equity pilot, using real market observations and local paper execution.
Key findings from the cryptocurrency traces:
🔹 Decision validity reached 97.35%, while structured reporting accuracy was 90.81%, among scorable responses.
🔹 Commitment adherence was 94.36% overall, but only 62.63% for constraints declared with nonempty order proposals.
🔹 Only 25.13% of valid decisions declared a nontrivial commitment.
Coverage and activity matter when interpreting adherence. Across both environments, every retained round’s cash and positions reconciled with recorded fills. Yet all 10 routes had negative marked returns in the settled cryptocurrency session.
✅ Valid decisions do not guarantee accurate reports. ✅ Ledger reconciliation does not establish profitable trading.
Trading agents need to be evaluated across decisions, commitments, execution, and reporting, with financial outcomes measured separately.
Full Paper:
https://x.com/dgrid_ai/status/2118595864004563368
FIDES-Live: Evaluating Behavioral Concordance in Sequential Trading Agents A
n AI trading agent can submit a valid order, have it rejected, and still report that the trade was completed. A correct account balance alone would not reveal this discrepancy.
In this paper, we examine whether an agent’s decisions, proposed orders, execution outcomes, and subsequent reports agree with one another. We introduce FIDES-Live to audit this behavioral concordance through the records produced during each trading round.
We audited 10 requested inference routes across 7,609 account-rounds in cryptocurrency and an equity pilot, using real market observations and local paper execution.
Key findings from the cryptocurrency traces:
🔹 Decision validity reached 97.35%, while structured reporting accuracy was 90.81%, among scorable responses.
🔹 Commitment adherence was 94.36% overall, but only 62.63% for constraints declared with nonempty order proposals.
🔹 Only 25.13% of valid decisions declared a nontrivial commitment.
Coverage and activity matter when interpreting adherence. Across both environments, every retained round’s cash and positions reconciled with recorded fills. Yet all 10 routes had negative marked returns in the settled cryptocurrency session.
✅ Valid decisions do not guarantee accurate reports. ✅ Ledger reconciliation does not establish profitable trading.
Trading agents need to be evaluated across decisions, commitments, execution, and reporting, with financial outcomes measured separately.
Full Paper:
https://x.com/dgrid_ai/status/2118595864004563368
