Rule sandbox & simulation
Rule simulation is the process of running a newly created scenario against historical data to measure what it would generate before it goes live. TruvaLI performs this test before the rule is published and reports the expected alert volume together with the estimated false positive rate.
The Live System Is Not a Testing Ground
A single poorly configured rule can keep a compliance team busy for days.
Thousands of false positives can fill the investigation queue. While the team spends time clearing them, regulatory investigation deadlines continue to run and genuinely risky transactions may get lost in the noise.
This can create a greater operational risk than writing the rule itself. For this reason, many institutions hesitate to introduce new scenarios and fail to update their defenses as quickly as they should.
What Does Simulation Do?
Instead of publishing the rule immediately, you run it against your historical data.
The system provides a concrete report showing:
- How many alerts would the rule have generated during the selected period?
- What would the average daily alert volume be?
- Based on how similar historical cases were resolved, what proportion of those alerts would likely have resulted in false positives?
- Which customer segments and transaction types would be affected the most?
A statement such as:
> "If you activate this rule today, it is expected to generate an average of 1,000 alerts per day, and approximately 85% of them may be false positives."
is one of the most valuable insights you can have before moving a rule into production.
Finding the Right Threshold Through Simulation
Simulation is not a one-time approval step.
You can run the same rule with different thresholds and compare the results. When you raise the threshold, you can see how many alerts may be missed. When you lower it, you can measure how much the investigation queue may grow.
This allows thresholds to be based on measurement rather than assumptions.
Protecting Investigation Timelines
Under a risk-based approach, suspicious activity must be reviewed within defined timeframes.
These timelines are directly related to the capacity of your compliance team. Publishing a rule without knowing the volume it will generate may put your ability to meet those timelines at risk.
Simulation helps show in advance whether a new scenario can fit within the team’s operational capacity.
Rule History and Comparison
The actual performance of rules already in production can also be monitored.
Rules with consistently high false positive rates can be flagged, and threshold improvement recommendations can be provided.
This helps the rule set evolve over time instead of becoming less effective.
What It Provides to the Institution
- Development of new scenarios without using the live operation as a testing ground.
- Threshold configuration based on measurement rather than assumptions.
- More predictable investigation workloads and timelines.
- Faster response to new threats with greater visibility into the expected outcome.
Common questions
- How much historical data is required for simulation?
- Historical data can be transferred into the system during implementation, allowing meaningful testing from the beginning. The institution can choose the test period, and longer periods may be preferred when seasonal effects need to be evaluated.
- How is the false positive rate calculated?
- The alerts the rule would have generated historically are compared with how similar cases were resolved during the same period. The result is an estimate, but it provides a more reliable basis than publishing a rule without measuring its likely impact.
- Can we test the same rule with different thresholds?
- Yes. Running the same rule with multiple thresholds and comparing the results is one of the main uses of simulation. This allows the threshold to be determined through measurement rather than guesswork.
- Is the performance of rules already in production also monitored?
- Yes. Rules with consistently high false positive rates can be flagged and threshold improvement recommendations can be provided, helping the rule set remain effective over time.