I introduce below some principles of quantitative trading strategies design. The below article is by no means exhaustive and must be tailored to the strategy studied. It does not guarantee good performance either but it gives some clues to avoid the main pitfalls. Any comments welcome….
Step 1: know our dataset
Knowing the environment/data you’re dealing with is the very first step toward a proper strategy design. A good example is company reporting dates. Usually companies report earnings for a given quarter (in the US) a few weeks after the end of the quarter. However the data might appear in the database with a time stamp indicating the exact calendar end of the quarter. If you test a basic strategy where earnings predict future returns you’ll probably find amazing results. Fantastic yes but it has nothing to do with real life….To make the analysis meaningful, earnings should be lagged by a few weeks because this is the date where companies earnings were made publicly available.
Step 2: Only sensible assumptions should be tested
From personal experience any good trading strategy/idea starts with a solid fundamental background or a market observation. I’ve seen many people (usually junior or inexperienced quant) testing random ideas. Usually that does not take you very far, even worst; meaningful results might be found without being able to explain why. The danger is to start trading and all of a sudden the pattern breaks for no apparent reason.
Step 3: Make realistic assumption about transactions costs
Transaction costs and slippage can turn a profitable strategy into a losing one. This needs to be addressed with accuracy and it’s good practice to estimate realistic Bid/Ask spreads before starting the number crunching step.
Step 4: Run Out of sample / Walk-Forward analysis
A back test is performed on a given test period then the same backtest is run on a new test period (a different data sample). If the parameters were over-optimized or the market regime has changed in the first back test, it’s unlikely that they will perform well in the second time period (out of sample). To be completely effective, the out-of-sample data should only be used once otherwise it becomes to easily in-sample. You can even go a step further and reproduce the above process in a sliding or expanding window. This is commonly called Walk-Forward Analysis. This is the only way to produce results that are completely out of sample. This is also a very computational intensive task so my advice would be to consider such an analysis when the analyst has good reasons to think the strategy will produce tradable results (i.e. profitable after transaction costs).
Step 5: Run parameters optimization only when necessary
This is a very sensible topic as any optimization can turn into curve fitting before you know it. My advice is to avoid parameters optimization and use common sense instead whenever possible. Obviously this is not always possible and optimization can add value if used properly. When it is needed, I tend to minimize the number of parameters to optimize and optimize over a broad range of values with large increments. This is a good way of minimizing curve fitting risk.
Step 6: Use appropriate performance/risk metrics
There are many performance metrics and I will introduce below only the main ones.
Expectation (or performance) is obviously very important but it has to be looked at in the light of risk.
Sharpe ratio is a good starting point to incorporate risk in strategy design. It is calculated by subtracting the risk-free rate from the rate of return for a portfolio and dividing the result by the standard deviation of the portfolio return. Main drawback: it does not tell anything about extreme risk.
Maximum Drawdown is the one to use for extreme risk. It’s the peak-to-trough decline during a specific record period of an investment. It is often associated with the Time to recover which tells how long it takes to reach a new high from a valley obviously the shorter the better.
I also like looking at Profit Factor which is the value of profit generated by profitable trades divided by the losses generated by losing trades. For example, a value of 2 indicates that twice as much money was made from winning trades than was lost from losing trades. It is a good way to compare strategies with very different return profiles i.e. trend following strategies (low hit ratio and high expectation) vs. Mean reverting strategies (high hit ratio and low expectation).
Step 7: Run paper portfolio long enough to validate back test assumptions
Once you’re happy with the back tested performance/risk metrics you’re ready to start a paper portfolio. Everything should be set up as it was a live portfolio except that trades are not executed. This can be done by recording entry and exit prices and creating a return stream or running a strategy on a “demo account” with your broker. An important point to consider is how long to run a paper portfolio? Three aspects should be considered
- Get enough trades to make performance metrics meaningful
- The trader must get very familiar with the strategy. Entry and exit points should look familiar.
- Verify that the assumptions made in the back test match the observations in the paper portfolio. From experience there are always a few things to fix here.
Step 8: Start small and increase trades size gradually
Once all the steps above have been passed successfully, the strategy is ready to go live. However I would never go live with full size trades on day 1. Usually you should start with very small sizes and increase them gradually. All along the process you should make sure that live results match back test results.