TradeStation Strategy Testing - Violating These Steps Will Damage Your Account

One of the most rewarding experiences for a TradeStation trader is to pick up a performance report that proves their great strategy idea is indeed a profitable strategy. Strategy testing done properly, as is outlined in this article, can verify the efficacy of your trading strategy and give you confidence to start trading it. But be forewarned, strategy testing done improperly can lead you toward financial destruction.

Strategy testing done incorrectly can result in false hope in a losing strategy.A trader recently shared his experience of getting great results from strategy testing his idea, but after trading it live in the market, he was losing money every day. He was baffled about what he did wrong. Having gotten excellent results on his back-testing performance report, he wondered why his promising strategy was draining his trading account. The problem was he violated several of the proper steps necessary for reliable strategy testing.

With the knowledge of how to get an accurate performance report you will be able to trust your strategy in live trading and protect your trading account. In order to properly test a strategy, there are 5 main steps that are vital to follow; configure TradeStation, "in-sample data" testing, "out-of-sample data" testing, live forward testing on the simulator account, and real live trading execution.

Step 1: Configure TradeStation

Before you begin testing your data, you must configure TradeStation so that the data it pulls onto your performance report will be accurate. Follow these 3 critical items to properly configure TradeStation.

(a) In your TradeStation platform menu, go to "format symbol" and give TradeStation a starting and ending date to test. This historical date range is called the "in-sample data." Do not include the most recent six months in this "in-sample data." The most recent six months is called the "out-of-sample data," and it will be used later during your "out-of-sample data" testing step.

(b) Next, in your TradeStation platform menu, go to "format strategy" and select "properties for all." Now select the "general" tab and enter the commissions and slippage (be as realistic as possible, or estimate too high if you are not sure). If this step is skipped, then the strategy testing performance report will be meaningless. If this is not done you might have a good looking performance report equity curve, but as soon as you enter the commissions and slippage figures the equity curve can reverse into an underwater equity curve.

(c) The last configuration step is under "properties for all" under the "general" tab. Look in the bottom left section called "strategy testing resolution." Check the "look-inside-bar back-testing" option and then select the smallest time frame available for your chart style to make the strategy testing more closely resemble live data. When strategy testing, TradeStation uses the open, high, low, and closing data, thus the larger the time frame bar, the more distorted the strategy testing performance report can be. This "look-inside bar back-testing" option will make the computer do a lot more strategy testing calculations. This may really slow down your performance report generation, so please be patient. For an accurate performance report you must use the "look-inside bar back-testing" option.

These configuration steps are critical to getting an accurate performance report, so be sure this is completed precisely before continuing. Once TradeStation has been configured correctly, you can begin testing your strategy.

Step 2: In-Sample Data Testing (also called "Back testing")

You are now ready to start testing your strategy idea. We will begin with testing the "in-sample data" that you set up for testing during the configuration steps. Begin with bringing up a TradeStation performance report. Right now I have a performance report in front of me that I will refer to, but you will be looking at your own performance report to analyze your own numbers. This is what we will be referring to in the steps below. There are 7 sub-steps to "in-sample data" testing, as follows:

First, look at how many trades the strategy made. To reduce strategy testing errors, where error is defined by [ error = 1 / Square Root (Number Trades In Test) ], you want at least 400 trades to reduce the margin for error to 5% in your strategy testing results. At 100 trades you have a 10% margin for error. The greater the number of inputs in your strategy that you optimize, the greater the number of trades you need to keep from over optimizing your strategy.

Also look at how many times the strategy traded on average per day. The more often a strategy trades the more profit it can generate.

In the performance report that I am looking at, it traded 397 trades in the last 3 1/2 months, averaging 5.3 trades per day.

Second, look at the "Average Trade Amount." It needs to be large enough that slow order fills and/or larger than normal slippage does not kill the profitability of the strategy.

In my report the "Average Trade Amount" is $162.32. The commissions and slippage amount as defined in the set up steps is already subtracted in this performance report.

65% of the time this strategy trades 1 contract.

35% of the time this strategy trades 3 contracts.

10% of the time this strategy trades 5 contracts.

Third, look to see if the "Profit Factor" and "Ratio Average Win-Average Loss" are both above 1.5 and the percentage of winning trades around 45% or better

This strategy had a "Profit Factor" of 1.83.

This strategy had a "Ratio Average Win-Average Loss" of 2.28 (2.28 means breakeven is around 28% "Percent Winning Trades")

On this strategy the "Percent Winning Trades" was 44.58%.

 

 

https://canvas.rice.edu/eportfolios/769/Hayley/Ultimate_PSEPrismaCloud_Exam__PDF_Questions_for_Prep

https://canvas.rice.edu/eportfolios/769/Hayley/Unique_PSEStrataDC_Exam__PDF_Questions_for_Good_Result

https://canvas.rice.edu/eportfolios/769/Hayley/Preferred_PSMII_Exam__PDF_Questions_for_Passing_in_the_First_Attempt

https://canvas.rice.edu/eportfolios/769/Hayley/Tested_PT0001_Exam__PDF_Questions_for_Good_Result

https://canvas.rice.edu/eportfolios/769/Hayley/Unique_P_C4HCD_1905_Exam__PDF_Questions_for_Good_Result

Fourth, look at the trade list page and assess the profit run ups and draw downs column. Notice how many trades made money and how much money they made before the trade exit occurred. Looking at what amount of money was made in relation to the profit run up and draw down, you want to know if managing the trades could generate more profits. The example used here shows that a good percentage of trades made much higher profits than where the automated exit points occurred.

Fifth, look at the three draw down (DD) numbers. I like to see the largest number at 15% or less of the "Total Net Profit" and the "Max DD" at 5% or less of the "Total Net Profit" (these numbers tell about the draw down risk level during your trades).

Total Profit - $64,440

Peak to Valley DD - $8,960 is 13% of Total Profit

Close to Close DD - $7,120 is 11% of Total Profit

Max DD - $3,420 - 5% of Total Profit

Sixth, Looking at the "Largest Losing Trade" on the report, I like to see 5% or less of the "Total Net Profit." In my report the "Largest Losing Trade" that occurred was $2,580 which is 4% of "Total Net Profit."

Seventh, I review the length of time in the average trade. Does the average time in a trade comply with the golden rule of trading; "cut your losses quickly and let your profits run?" You will also want to see if the strategy is built using only profit exits (no real stop loss exits). It might have a nice looking report, but it could show a messed up ratio between average bars per winning trade verses average bars per losing trade if there are no stop loss exits. Here are my average bars:

Average bars per winning trade 7.24

Average bars per losing trade 3.51 bars

This strategy complies with the golden rule of trading. Notice how it cuts losses quickly, at an average of 3.51 bars, and lets the profits run for an average of 7.24 bars.

So what does this all mean? It means this strategy has passed the historical strategy testing phase of strategy testing.

Step 3: Out-of-Sample Data (also called "Walk Forward Testing")

Once you have tested your "in-sample data" and have determined that your strategy is worthy of continued testing, you can now test your strategy against the "out-of-sample data." If you have not yet tested your "in-sample data, do that before proceeding.

To test the "out-of-sample data" we use the most recent 6 months of data available that you reserved in step 1(a). In step 1(b) of this article, we talked about configuring TradeStation and covered entering commissions and slippage and using the "look-inside-bar back-testing" option, which must be used to run any performance report used in your strategy testing. Be sure you have configured TradeStation correctly before moving on.

Go into "format symbol" and change the date range to include ONLY the "out-of-sample data" date range that was NOT used during the strategy testing on "in-sample data." This is referred to as testing on the "out-of-sample" data.

Begin with bringing up a TradeStation performance report on the "out-of-sample" data and review all the items that we discussed in step 2 above on this "out-of-sample" performance report. The closer it performs to the Step 2 "in-sample data" performance report, the more robust the strategy is. This suggests that the results were not from curve fitting and you have a good chance of having a viable strategy. This "out-of-sample" date range test is much more important than the strategy testing step on the "in-sample-data" for finding a successful strategy. It is a good idea to test multiple different "out-of-sample" date ranges, which is called "Walk Forward Analysis."

Robustness: Perry J. Kaufman stated, "Practically speaking, a robust trading strategy is one that produces consistently good results across a broad set of parameter (input) values applied to many different markets tested for many years."

If the strategy fails during this "out-of-sample data" test, do NOT optimize using your reserved "out-of-sample data." This would defeat this vitally important step in strategy development. You can go back to your strategy and fix it, or else drop it and develop a new strategy idea.