Stop guessing when your split test is “done.”
With
Winner Recommender
, Vidalytics now tells you who’s ahead, how confident the data is, and when it’s safe to make a decision
— all built directly into your experiment.No spreadsheets. No statistical expertise required. Just clear guidance so you can move faster with confidence.
🔍
Overview
Running A/B tests is easy. Knowing
when to stop them
is the hard part.Winner Recommender solves that by analyzing your test data in real time and giving you a
clear, simple recommendation
— whether you should keep running or confidently call a winner.It removes uncertainty, reduces hesitation, and helps you make better decisions based on real data.
🆕
What’s New
Automatic Winner Guidance
- See which variant is leading, how much lift it has, and how confident the data is
- Clear recommendation shown directly inside your test
Smart Test Status
Every experiment now shows a clear status:
- Too Early– Not enough data yet
- Too Close– No meaningful difference yet
- Gaining Confidence– A leader is emerging
- Ready to Finish– Enough data to safely decide
Metrics Timeline View
- See how your test is evolving over time
- Track trends for each variant across days or weeks
Custom Guardrails
Adjust thresholds based on your goals:
- Minimum duration
- Minimum viewers
- Minimum conversions/events
- Confidence thresholds
- Minimum meaningful lift (MDE)
🛠️
How to Use It
- Go to Experiments → Create Experiment(or open an existing one).
- Set up your test: Select Control and Variant; Choose your primary goal metric;
- Start the test.
- Open the Experiment Profileto view results.
- Monitor the status indicator:
- Too Early → keep running
- Too Close → wait for clearer separation
- Gaining Confidence → trending toward a leader
- Ready to Finish → safe to decide
- When ready, click Finish Testand review the recommendation before confirming.
- Optionally apply the winner to 100% of traffic.
💡
Benefits
- No More Guesswork: Know exactly when your test has enough data to make a decision.
- Faster Decisions: Stop waiting too long — or ending tests too early.
- Higher Confidence Results: Make calls based on real statistical confidence, not intuition.
- Better Test Completion Rates: Especially helpful for teams who struggle to confidently finish experiments.
- Deeper Insight Over Time: Understand how performance evolves, not just final results.
⚠️
Restrictions
- Available on Pro Plans and up.
- The system recommends a leader, but the final decision is always yours
- Data updates may take up to 2 hoursafter new activity
If you have any additional questions please reach out to our Customer Happiness Team at hi@vidalytics.com