How to Backtest Stock Signals Properly — A Step-by-Step Guide

"I backtested it and it made 340% in 2023." That sentence has ended more trading accounts than bad stock picks. The problem isn't the strategy — it's the backtest methodology. A signal that looks amazing in-sample can be pure curve-fitting.

Here's the process that actually works.

Step 1: Define the Signal Hypothesis First

Before touching any data, write down why the signal should work. "Momentum persists because of slow information diffusion" is a hypothesis. "The RSI crosses 30 on a Tuesday" is a pattern-matching exercise. Only the former survives out-of-sample testing.

Step 2: Split Your Data Correctly

The golden rule: you must have data the model has never seen.

• In-sample (train): 70% of your data, used to develop and tune the signal.
• Out-of-sample (test): 30% of your data, never touched during development. This is your final verdict.

If you look at the test data even once to "adjust the parameters slightly," you've contaminated it. The test is now in-sample, and your confidence is an illusion.

Step 3: Walk-Forward Validation

A single train/test split is fragile. The professional approach is walk-forward analysis:

1. Train on months 1–12, test on month 13.
2. Train on months 2–13, test on month 14.
3. Train on months 3–14, test on month 15.
4. Repeat, rolling forward one month at a time.

This gives you a distribution of out-of-sample results rather than a single lucky/unlucky test window. If the signal works across 20 different test windows, you can trust it. If it works on 3 out of 20, it's noise.

Step 4: Model Transaction Costs

The #1 killer of "profitable" backtests. You must include:

• Spread: You buy at the ask, sell at the bid. For liquid large-caps, this is ~1bp. For mid-caps, 5–15bp.
• Slippage: Your order moves the price against you. Model at least 2× the spread for anything under $500M market cap.
• Commissions: Trivial for most brokers now, but still add $1/side as a floor.

Step 5: Stress-Test for Regime Change

A signal that works in a trending 2023 and breaks in a mean-reverting 2024 isn't a signal — it's a bet on the regime continuing. Test your signal across:

• Bull market (e.g., 2019–2021)
• Crash (e.g., March 2020, Feb 2022)
• Choppy sideways (e.g., 2015, 2017 H2)
• High-vol grind (e.g., 2018 Q4)

If the signal only works in one regime, it's not robust — it's conditional. And conditions change.

Step 6: Check for Look-Ahead Bias

The most common (and most embarrassing) backtest error: using data that wasn't available at signal time.

• Using today's close for a signal that triggers on today's close? That's fine if you execute at the close. But if you're modeling a mid-day signal, you can't use the close.
• Using earnings that were released 3 days after the signal date? That's look-ahead bias.
• Using a "corrected" price that accounts for a split that happened later? Also look-ahead.

Always use point-in-time data — exactly what was visible at the decision moment.

Step 7: The Significance Test

Even a properly conducted backtest can be a fluke. Run a Monte Carlo permutation test:

1. Take your signal's returns.
2. Randomly shuffle 10,000 times.
3. Count how often the shuffled version beats your actual result.

If your "edge" is beaten by 5% or more of random shuffles, it's not statistically significant. You need p < 0.05 minimum, ideally p < 0.01.

How GemStox Handles All of This

GemStox doesn't just run one backtest. Each signal must pass 10 independent strategy classes — stochastic simulation, momentum, mean-reversion, volatility targeting, DCF cross-check, and more. A signal only reaches your screen when the strategies agree. That agreement across independent methods is the closest thing to walk-forward validation in real-time.