I finally found a source for free forex data that is complete, and FREE!
Here is the link to the file - Metatrader Free Forex Data
And the recommended set-up to back test your EA on MT4, follow this link
Set up for back test
There are other helpfull articles on that website also. I took a look at the two systems they are offering for sale and they don't fit my style. I like smooth equity curves. One of the systems has a 30% win rate, I like small gains with high win rates.
Sunday, June 27, 2010
Metatrader Forex Data and back testing
Saturday, February 13, 2010
Bracket Trader Stats
Here is a handy tool if you use Bracket Trader (BT). This excel file will extract information from your BT log files to give you trading stats. Take a look at thew different stats it tracks.
Bracket Trader Spreadsheet
The instructions are good, so no need for me to explain how to use it.
Tuesday, December 1, 2009
HSI System
- allowable trading times
- shorting and buying switches to allow me to turn ON/OFF shorting or buying
The System
A brief description of each;
Sunday, April 19, 2009
Tipster Trendlines v2.1
This page is out of date, click "Tipster Trendlines" at the top of the page.
Trade directly from your Amibroker charts. This AFL code allows you to draw three lines on the Amibroker chart and place an error free trade. You must have Amibroker 5 or better, IBController (from Amibroker website), Trader WorkStation (TWS) and an Interactive Brokers (IB) account. I have not posted the code at this time, but for a small $20 donation I will send it to you.
Tipster Trendlines v2.1 from Another Brian on Vimeo.
Saturday, April 11, 2009
Tipster Trendlines - version 2.1
This page is out of date, click "Tipster Trendlines" at the top of the page.
The "free" version is no longer available. Go to the Tipster Trendlines 2.1 post for additional information.I've modified the code for my Amibroker interface to Interactive Brokers (IB) Trader Workstation (TWS). All the functionality is now on one pane.
The user puts three horizontal lines on the chart called "BS", "TA", and "ST". Bracket orders can be placed and the code will check to ensure the bracket is correctly setup. You can also disable this bracket order feature and place only a "BS" (Buy Short) with or without a stop or target.
The Risk display is only an approximation for forex, and I have not tested it for futures.
The menu's are all drop down now. Here are some sample screen shots. I'll make a video when IB is online, their server are shut down until Sunday evening.
Saturday, August 9, 2008
Indicators
DAX, QQQQ, SPY, IWM, TSX Stocks, Forex, whatever you trade, price patterns and understanding why price moves is probably better than memorizing patterns. Take a look at a MACD divergence and a price chart that uses the same MA's as the MACD. Find a divergence. Look at the patterns of higher highs and lowers lows. Notice that you can see these if you look. I find the most use for indicators is to filter my scan results.
Here is an interesting concept that I have looked at, it comes from some forum somewhere. I believe it has merit.
Crossover systems will generally do well when traded intraday, not so good when traded of daily prices. By substituting different filters based on the most recent action of the chosen market, you can use their basic rules (idea) forever. You can program this to be self adjusting (auto optimize).
As part of my exploration scan for trending stocks, I use the double stochastic, it filters out stocks that aren't pulling back quite to where I would like to see. This is a good use for an indicator, not as sole selection criteria. Support and resistance, and a dose of anticipation.
Friday, August 8, 2008
Using Multiple Trading Systems
Some info I found while surfing. I don't use neaural nets but I can relate to using multiple systems to increase trading frequency. The system should be non-correlated. It's a good read.
First find out what is going on in the market you want to trade in the timeframe you plan on holding a position. If you want to day trade with one trade per-day then find out all the different ways the day has played out in the past. Ex. trend day, two-way day, reversal day, etc.
Once you've done this you should have an idea of which type of day is most common and which is most profitable. Then define something which could be of value to trade one of the market types. An example might be in a reversal day to find out how often the market makes a low of the day in the first 15 min. of the session. If it happens often enough to be of interest then you go on to the next step.
Take every period for which the target is found and create a table of outputs with 1 for the target and 0 for non-targets. Then pre-process all the inputs into the target and convert them to binary inputs. (A common mistake is to take open, high, low, and close data -- analog and assume you can find relationships with the target). For ex. yesterday close > day before yesterday close. If found mark the input as a 1 if not present mark it as a 0. Do this for as many identifies as you can. This may present a hundred or more binary inputs leading to the target for each day of the data.
Then you'd pass the data into a backprop neural net and have it train on the data. (You’ll need to set aside some data for out of sample testing). Once it's trained to hit at least 90% correctly test the NN on the out-of-sample data. If you hit at least 85% correctly then you can do one of two things. If you're a discretionary trader, setup the NN and pre-process the inputs every day and use the net to predict whether tomorrow has the target (in this example the low of the day is within 15 min. of the start of the session). If so use it to trade to the upside as long the net remains 85% correct. If you're a systems trader then go back to the net and look at the weights of the net to see which of the binary inputs were most important in hitting the target. Use the inputs to create a back testable system based on the patterns. A system might be when xyz pattern exists then buy next bar above the lowest bar as long as the time is within the first 15 min. of the day. Set the stop to one tick below the low.
If the system tests profitable enough to be of interest then move on to the next step.
Next, take the trades and test them against random trades pulled from the same year (the edge test). Rank the trades versus random for each year of the back test. If the trades score consistently above the 70th percentile then you can guess you've found an edge-based system. If not, then you have to assume you've found a temporal characteristic in the data that can be exploited for some period of time.
If its edge based then all you need to do is adjust the trades for market volatility and apply a money management strategy. Check the trades on a periodic basis to ensure the edge continues and plan what to do with your next million. If it's not edge based you can still trade it but you need to setup an objective bailout method such as running a monte carlo sim and determining the bailout point to be say the 95% level of the predicted max drawdown point. Your trading would be more defensive using a non-edge based method as well. Maybe you'd split the trade size in half and have a 15 min. or 10% of daily range as a filter to adding the second position (letting the position prove itself) as long as the volatility was large enough to justify the scaled entry.
Every model I've worked on has gone through the same process. Look at the behaviour’s present in a market; characterize them by creating a rule and checking the fit until all behaviour’s are noted. Then start looking to see if there is a component to the behaviour that is non-random. If so, develop a system to mine it and create a way to monitor the behaviour to ensure it's persistent over time. For example, one of the behaviour’s widely known is the trend day in the SP market. It can be identified just by visually inspecting a chart. I characterized it as a low/high within 10% of the low/high of the day and the close within 20% of the high/low of the day. With the definition I can see how many of these days have persisted over the years (averages about 25 day’s per-year). Then I can see if there is a way to identify these days in advance (realizing I'm going to also be capturing some false days as well).
The algorithm used to adjust the number of contracts traded to volatility is calculated by the range (high - low), then average it for the past ten days. I use ten because I want my model to cut back on size pretty quickly if the volatility jumps. Then I divide the highest historical 10 day volatility (approx. 48 pts.) by the current volatility (ex. 8 pts) to come up with a multiplier (ex. 6). The model would then apply 6 contracts for the next trade. This is not the final size used to trade. It's just used to adjust the model for volatility levels so I measure one period against another without volatility being a consideration.
By doing so, I can see if the same level of opportunities persist from period to period. I can also use these normalized trades to feed into money management models as well as Monte Carlo tests to estimate future performance and drawdowns. If you were to use trades from say 2000 and 2004 for the SP market in a Monte Carlo test without normalizing volatility you'd get a much distorted estimate of future performance.
Comment from reader:
Just to make sure I understand the big picture, this is all being done to increase frequency in the desired profitable time period, correct? So if I understand, sub-par models tested individually with low frequency can be morphed into an above-par model when combined with other non-correlated sub-par models (assuming they're not too sub-par), thus increasing frequency, consistency, and lowering the need for a higher profit factor? Thank you for presenting the information the way you did, I would not have made that connection otherwise (assuming I’m on the right track). Fascinating.
Trading System Development
Trading system development – The DUM method
D - Define
All systems are based on finding and pulling a fundamental truth about the market. Define what fundamental truth you'll be going after. Eg. All markets have a tendency to trend beyond random. Now you've got the definition that most technical-based hedge funds are derived from.
U - Understand
Determine the conditions under which the defined truth tends to occur. In the case of a trend tendency it could be when does the trend tendency begin beyond random? This will lead you to how to measure a trend. Since trends can occur randomly, how do I determine if a trend is beyond a confidence level of randomness? Does the trending tendency beyond random exhibit the same degree of persistence beyond one year? two years? 5 years? If not, is there some point at which the persistence beyond random occurs every year? If so, does it also persist at the same frequency for 5, 10, 50 different markets? If so, you've discovered a fundamental truth and you now understand what you need to know about the behaviour.
M - Mine
Once you understand the conditions under which the behaviour occurs, you write the code necessary to map the understanding of the behaviour. Is the code going to be all inclusive of many markets? or try to just go after the best of the best? Once mapped it's a mechanical process to determine how well it maps against the behaviour. After you're satisfied you've developed a satisfactory method for mining the behaviour, you can do an edge test to see if it happens beyond random. If not, use Monte Carlo sims to determine confidence levels for trading the method. Determine at what confidence level you'll stop trading. Examine the drawdown versus the profit. Is it worth risking any money on this? If so, allocate money using a money management scheme.
After you're done with this, you'll have your first system. Next, develop a complimentary system (non-correlated). Go through the same process for (a different type) a range bound system. Once you've gone through the mining stage, use the correlation test to weight the two systems. Apply the weights to the money management scheme and move on to your third system.
Wednesday, July 16, 2008
Price Patterns
We have all tried to use indicators to build a system, I have tried to use them to build an auto trading system and have found that it’s not worth the effort of looking for a great combination of indicators only. You need to use price action first and foremost. Support and resistance with a few other gems like a simple momentum and moving average are, to me, what works.
I have also found that indicators and patterns are just another way of looking at the same price action. Now that I've been staring at indicators for a long time and understand price action, and memorized patterns such as head and shoulders, bull flag, etc, understating basic price action has allowed me to forget about all that stuff. It's all just another way of saying higher high, higher low. If you read the book you might also have this "moment". After watching Alphatrends videos and reading Brian Shannons new book, the approach that makes the most sense to me is support and resistance, higher high, lower low, etc. For auto trading, the trick is to be able to program the rules as you apply them when looking at a chart, into software. Not so easy.
I trade Forex based on an auto trading system I developed, currently, no real money as there are still bugs. I also trade the TSX stocks (real money) based on the approach demonstrated in Brian Shannon's book. Each night I’ll perform a scan of the TSX and set up the trades for the next day. My approach has been altered somewhat since reading his book. The original was based on reading his blog and watching his videos. I have created code to scan for these stocks that are in stage 2 or 4. So far so good. Obviously it would be more profitable to check on the trades intra day and adjust the stops to protect profit. That doesn't seem to be the most damaging though. The entries are planned to be on a stage 1 to 2 short time frame but a morning gap usually kills me as it retraces. This doesn't happen all the time. TWS (IB platform) does allow conditional orders but not to the extent I need them. I need to look for a pullback to a level, then a break above another level (buy stop) to pull the trigger. To do this I will need to program Amibroker to follow the stock and place the order. That's my next project after I get this auto trading forex code working well enough to make some $. As for the TSX, its working out, but it would be more profitable if I had the code mentioned above implemented.
Wednesday, April 16, 2008
Divergence and the Tick
Divergence:
Here is a tick chart of GBP.USD. I have inserted a line at the highs and lows. I continue to struggle with interpreting divergence. This chart looks like is is in consolidation, so a breakout set up is probably best, but I'd like some comments on where you readers see a divergence set-up. What is used for the MACD, the MACD itself or the histogram? I have seen both used. This chart shows two opposite divergences for MACD and the histogram, at line B and D. Perhaps the double top makes it null and void. Can anyone help on interpreting this chart?
Tick: I have read that back testing should be done with tick data, since it removes the time element. I am beginning to read more about using tick charts, after looking at them for a few days I like what I see. Price action matters more than time data. Comments?
Sunday, March 30, 2008
Forex Video
Here are a couple of videos that I find useful.
1. MACD Video
2. Where to get Forex Information
MACD = A trend-following momentum indicator that shows the relationship between two moving averages of prices. The MACD is calculated by subtracting the 26-day exponential moving average (EMA) from the 12-day EMA. A nine-day EMA of the MACD, called the "signal line", is then plotted on top of the MACD, functioning as a trigger for buy and sell signals.
Here is an educational video done by Brian Shannon at AlphaTrends.net He explains the MACD and how to use it. Good review if you don't have things memorized. The MACD is one of the ost verstaile indicators.
Wednesday, March 26, 2008
Trendline Trading
I have updated my code to include a trailing stop. When I see a possible set up, I can now set the "Study" in Amibroker to send two complete orders to IB. The first has a defined target and stop, better known as a bracket order. The second submits a buy at the same price with only a trailing stop. This will get me a quick profit and also let an order run with the price. Here is the window I use (Param window) to set it up.
The colored bars are a system in itself, I don't use the signals for automation, I use it for "watching the screen trading"
I set up the "Study" lines, green for buy, red for short, as solid lines. When the order is triggered the profit and stop lines turn grey, this tells me the lines are not active and locked out. I can also go to the menu and press "Lock-Out" so a trade will not be triggered for buy or short. I can also select "scale-out" to use the trailing stop feature. With more options comes more clutter. The plan is to figure out what suits me best, suits my style best and remove everything else.
Right now I'm scalping some evening (EST) USD.JPY and setting up night time trades for EUR.USD. So far it's break even with real money. This blog is one way to record how I'm doing instead of keeping a log. Perhaps my next posts will describe some trades, both good and bad, so I can track what my thoughts were and what worked. Haven't decided on that yet.
Friday, March 21, 2008
A Trading System
Much has been written on trading systems. I have tried many and found that the advice given by most honest traders is fitting, that is, the system has to fit well with the person. One such system is shown in the screen shot below. I was reading the latest post by Rich over at Forex Project and the comment section began a long topic of systems discussion. It was a discussion about trading systems and one trader offered up his charts. The discussion is very interesting, and should be read. I noticed that the chart looked familiar to one I use. All I really had to do was change the period variable. Read it here - Want To Buy a Forex Trading System?
I don't use this systems now, I stick to S/R and a 5 period EMA on the 1H or 4H charts. The top chart is mine, the lower chart is from the post at Forex Project.
This particular chart is from comment No. 16.
Tipster Code Above
System posted and discussed at Forex Project.
Wednesday, February 6, 2008
Automatic Order Transmission
When the charting software has a signal to buy, it transmits it to the broker software client running on the same computer, your laptop. That software in turn sends the order to the broker office computer for execution. Here the things I have found to be important, to safeguard against things like loss of data connection, or power failure, or anything that could go wrong.
The order is transmitted with a buy order, limit order, and stop order. The buy order has a "Good Until Time" attached to it. I'm going for 10 pips of profit, so once the order is sent, the PC can crash all it wants. I have this portion working.
The next issue is loss of data feed. When the data starts up again, a buy order could be sent, but it might not be current, that’s a problem, and I haven't solved it yet. I haven't actually tried yet.
Right now I'm using 6 pairs to work out the bugs, and while I'm doing this, I'm tracking all the trades in excel and summarizing the trades and pips per pair, and the number of minutes I'm in trades. Once I get enough data I'm going to look at the daily charts and see what the correlation is between good performers and bad performers. I'll look deeper into trading times as well. The plan so far is to trade 2 pairs on the 15 minute time frame. I will try the 1 hour to see results I get too. The comparison of 15 minute time frame to 1 hour can probably be done by back testing. My issue here is most data is dirty, such as FINAM. I see lots of spikes in the data. I may try EODDATA and see what I get.
The nice thing is that this is being testing during the night and day while I'm at work, then I fix the bugs in the evening.
For anyone that is entering orders manually, there is a simple script you could use with Amibroker to initiate your order all at once will little risk of error. It's fast and accurate. I use it to send bracket orders while I’m at work.
Wednesday, January 30, 2008
Nickel and Dime "Mother Market" to DEATH
The last two weeks have been rather enlightening for me... so far. Here's what developed that "turned on" the light in my head. I just hope the light is what I think it is. I was reading the book about Larry Livermore called "Reminiscences of a Stock Operator" that I downloaded from ForexProject in PDF. The book is outdated in terms of the trading laws and some other things but as far as trader psychology and approach I think there is value in reading this book. I have read comments from readers that suggest Livermore never tells how he made all his cash in trading, I disagree. He didn't give a step by step account but I picked up some pointers. One such pointer was that he was always talking about having a "bigger line to swing". That means the more capitol one has, the more you can put on any one trade and thus make more profit in terms of dollars. He made millions and lost millions, and did it all over again. Then I recalled looking at a spreadsheet I downloaded somewhere that showed how fast your dollars grow when compounding. So I returned to the spreadsheet and had a look. I didn't do what I need it to do so I made my own. Here are my assumptions;
make at least 10 pips per day
trade 5 days a week, one or two pairs
start with $2000 (one lot, 100,000, at 50:1)
Maximum size of 5 lots - assuming I get there!
Here is the spreadsheet that shows what you can do if you make 10 pips a day, if it has errors, I would love for you to point them out. As you can see, it would take 6 months to get to $50,000.
Then I thought "my account is another person or entity, just like a corporation", I don't need this money to pay bills, I have a day job for that. What if I could automate this and make this "entity" wake up at 3am and trade until 11am (the best time to trade Forex). This "entity" would only have to capture 10 pips a day to make this work. Is that achievable in an automated program? The short answer is "yes it is". So assuming the above is correct and without any glaring errors in calculation, I am now testing my system and checking the trades daily. I have found some problems but I'm getting close. I have completed only 2 days of testing and the first showed 24 pips, the seconds showed 8 pips. I'm not concentrating on the signals so much as the order transmission. I don't think it is very difficult to capture 10 pips a day, but to do it constantly may take some work. If that target can be achieved, then the power of time will make the coin grow. To capture 10 pips, I plan to trade the system only between 3am and 11am EST, and trade only one pair, probably EURUSD, but that remains to be decided. 10 pips do not seem like a mountain, that’s why I call this post "nickel and dime". For the testing of the order system, I'm using 6 pairs to work out the bugs. Comments on this are more than welcome, as are any suggestions.
What's next?
A couple of things to blog about tonight;
- New indicators
- Trading system update
I used to use a "squeeze" indicator when I traded stocks, it was mainly focused on day trades and worked well on the US markets but not too well on the Canadian markets due only to the number of high volume stocks on the TSE (lower than those on the NASDAQ). The pattern day trader rule in the US was an issues. It worked even worse for FOREX. I recently came across a system that uses the squeeze, MACD and bollinger breaksouts. I'm looking into combining the indicators for a system that will trade short term moves in FOREX. Take a look at the videos at this site. While I'm doing that, I'm letting my current system run on the IB paper account to work out any bugs. That brings us to the second item I want to talk about tonight.
I have been slowly working on my new automated trading system and have some success. Back testing has produced results that are too good to be true. I've check the trade logs and they look right, so things are promising. Let me give you a quick brief of a post I hope to make shortly, if things work out. I ran across an excel spreadsheet that allows you to quickly calculate the periods needed to make a million based on the percent gain per period (I used a daily period). This got me thinking about more of a long term approach to profits, with short term gains, as in baby steps. So all I want to say right now is that I have a plan and I'll outline it shortly. Stay tuned.
Friday, November 30, 2007
Highest High & Lowest Low
Here is an interesting chart of USDCAD that automatically plots the highest high and lowest low for a pre-defined period. I like the chart automation but I don't really like the pre-defined period. I would like to have a period that adjust automatically but I'm not sure what I want to use for the basis of the adjustment. I was think about using volatility like ATR or perhaps SAR to determine the look back period. The formula uses the moving average cross of the highest high and lowest low, that's why some lines are longer than others.
If you look closely and examine the chart, you can't really find a consistent pattern of support break through. I'm looking at the break through of support or resistance, specifically looking for a close above or below, and watching what happens to price.
The chart below has my special DOTS on it. This is a weekly chart. I'd say it still has some upward movement to work off. This week it briefly broke through the 1.00 level. About a month ago it hit 0.9054, what a profit that could have been eh? The weekly doesn't show any sign of this upswing weakening but the daily chart shows it's running out of gas. I'm going to wait for the downswing then jump on board.
A short using $2000 with 50:1 leverage at 1.0000 and covering at 0.9054 gives you some serious cash, not that this is possible but part of the move could be caught. The cash grab is about $10,000 for that move. It's nice to know the possibility. Remember, with this also goes that you could lose the $2000 just as fast. Practice risk management.
Saturday, November 10, 2007
The Q's
Here is my current system applied to the QQQQ. I have turned off all of the other lines, dots and stuff that clutter the chart and just left on the training stops that show when a buy and sell occured. I didn't participate in any of these trades. I find the system to work better on high volume issues such as the Q's and FOREX. I don't use volume in the system, but I do look at it when considering an order.
Sunday, October 14, 2007
The System, and auto trading
I have my system pretty much tuned for end of day and 1 hour FOREX (foreign exchange) charts. Why Forex? It's none stop data almost all week long, with no gaps (hardly). I started out using FOREX data just for testing, but lately I have traded it with success, although small success. I haven't gone full bore yet.
I have also been writing code for auto trading Forex through Interactive Brokers TWS platform, using Amibroker. I currently have a P3 laptop set up doing just that all day long. Windows Debug is also running, logging everything that happens. IB offers a 'paper' trading account, I'm using it for testing.
Technical analysis is an important part of my trading plan. To ensure I honour my stops, I figured that a auto trading platform would work well. It would not only get me in at the right time (defined here as buying based on signals and not emotions), it would also get me out on a stop that is pre-defined, without my second guessing. Since Forex is very liquid, placing stops on IB's TWS does not concern me. The stops aren't place at the typical levels either.
One of the things I had to do when setting up the auto trading interface was to draw out a block diagram, a decision tree. I go long and short (no up tick rule either). I discovered that there are many different things to watch out for while planning this out. Something as simple as exiting a position when I get a reverse signal, what to do?
I'm not ready to go live with the auto trading interface yet, but I have gone live with the system on a end of day time frame. In the coming weeks, I'll be posting screen shots of the system, as seen in previous posts, but tweaked with some new things on the screen.
To help me develop the system and auto trading platform I have been using Yahoo groups and Herman;s website, Amibroker.org/userkb, a great resource.
I'm interested to know how many readers have ever tried Forex and what their experience with it was?
Friday, August 31, 2007
Trading System Update
Still working on my system using Amibroker. Here are a few things that I've tried, the work continues.
- I've incorporated Heiken Ashi charts for visual representation of trends.
- I'm now using a plug in dll for stops. It's called REM, and it's available on the yahoo Amibroker groups.
- I'm using FOREX data to test with since there are hardly any gaps. The downfall is that there is also no volume info available. But since I'm not using any volume related indicators, it's OK.
- Trying to incorporate the IB Controller plug in. It allows the buy & sell signals generated within Amibroker to transmit order to Interactive Brokers TWS application for placing orders. Wave files can also be triggered to tell you what is going on without watching the screen. It's not quite there yet. I'm using the paper trading account to test this functionality.
- I have incorporated signals from a Zero Lag indicator. It give similar signals compared to Stochastic but a bit earlier. I got this indy from Amibrokerfan.com. Mike, the admin, uses 5 indy's and Heiken Ashi charts for trading the emini. His system looks like it works but he doesn't advertise his loosing trades too much so who really knows.
- I continue to struggle with finding something that can do better than 5 to 10% CAR (compound annual return) . This may be some setting in my code to do with positioning size or something so I'll have to examine that.
The struggle continues. Overall, this is harder than I thought it would be. Hopefully I'll have something in a week or two.