H.R.4180 - Sound Dollar Act of 2012
To amend the Federal Reserve Act to improve the functioning and transparency of the Board of Governors of the Federal Reserve System and the Federal Open Market Committee, and for other purposes.
Tuesday, March 20, 2012
Greeks turn to barter during crisis
http://www.guardian.co.uk/world/2012/mar/16/greece-on-breadline-cashless-currency
There's some dispute about where and when it all started, but Christos Kamenides, genial professor of agricultural marketing at the University of Thessaloniki, is pretty confident he and his students have made sure it's not about to stop any time soon.
What's sure is that the so-called potato movement, through which thousands of tonnes of potatoes and other agricultural produce – including, hopefully, next month, Easter lamb – are being sold directly to consumers by their producers, is taking off across Greece.
"It's because everyone benefits," said Kamenides, standing in a clearing in the woods above Thessaloniki in front of one 25-tonne truck of potatoes, another of onions, and smaller vans of rice and olives. "Consumers gets good-quality food for a third of the price they would normally pay, and the producers get their money straight away."
Wednesday, March 14, 2012
Random Trailing Limit
I got the idea for a trailing limit from Van Tharpe's market classic Trade Your Way to Financial Freedom. I had Jon Rackley read through the book as part of his training when I first hired him. He brought my attention to an abnormal claim made in the middle of the book. Van Tharpe says that it's repeatedly possible to make money even with random numbers.
The book states that trading with totally random entries and a 3 ATR trailing stop regularly leads to making profits. Flip a coin. You buy if it lands on heads. Go short if it lands on tails. One important note is that we elected to incorporate pure random numbers in our programming instead of the pseudo-random numbers that coding languages generate. Doing this allows us to avoid time biases in the seeding process that repeatedly pop up when the seeds applied are close together in time.
Random numbers are a hobby of mine. I've continuously put effort into figuring out whether I could develop a strategy that trades totally at random and still makes profits. As the owner of a programming company for traders, and as part of Jon's early training for NinjaTrader in December, I assigned him the task of programming the blueprint into code.
The first functioning version that I inspected exhibited everything contrary to Van Tharpe's assertions. Using a trailing stop, regardless of the instrument and period tested, inevitably led to devastating losses. The profit factors usually came in near 0.7, a truly terrible number.
We did what most of our clients do when they find abysmal strategies. We flipped the strategy on its head. The new strategy uses only a 3 ATR (50 period) trailing limit.
Trailing Limit Analysis
The early conclusion is that trailing limits appear to offer a great deal of potential. Although Jon sent me the code several months ago, it's only this evening that I had a chance to properly review and test everything.
The profit factors are very encouraging. It depends on the chart that I tested. The worst that I found was a 1.0 profit factor. Everything else came out with profit factors greater than 1. The small table below contains the initial test results. Before you go off salivating that this is the next hot winner, there are a few considerations to keep in mind:
- The results depend entirely on the sequence of random numbers used. Using different sets of random numbers will cause different outcomes.
- The better results on the higher time frames likely result from sampling error. The number of trades involved was only ~160, which is not enough to make definitive conclusions on the nature of the performance. I prefer to see 400 or more trades before drawing definitive conclusions.
- These results do not include spread costs or commissions.
| Currency | Period | Profit Factor | Dates Tested |
| EURUSD | M1 | 1.12 | 9/12/2011-3/12/2012 |
| GBPUSD | M1 | 1.16 | 9/12/2011-3/12/2012 |
| USDJPY | M1 | 1.25 | 9/12/2011-3/12/2012 |
| EURUSD | M5 | 1.11 | 2011 |
| GBPUSD | M5 | 1.0 | 2011 |
| USDJPY | M5 | 1.06 | 2011 |
| EURUSD | H1 | 1.26 | 2011 |
| GBPUSD | H1 | 1.25 | 2011 |
| USDJPY | H1 | 1.48 | 2011 |
What made me feel better about the results was reviewing the entry and exit efficiencies of each strategy. The number of trades involved really cluttered the graphic. I ran a backtest on a much shorter time period so that the horizontal, blue line would appear clearly on each graph.
 |  |
The entry efficiency tells us exactly what we would expect to find. Trades which enter at random do not produce better than random (obviously). What's fascinating is how the trailing limit exit strategy actually skews the entry efficiencies to read slightly worse than random. This is a good example of why it's hazardous to rely purely on statistics. Keeping the big picture in mind minimizes the likelihood of making an erroneous conclusion, in this case that there might be something "wrong" with our absolutely random entries.
Adding Money Management
The percentage of winners for the tested charts ranges from 60-67% accuracy. High percentages of winners generally lead to winning streaks. My cursory glance through the chart led to my easily finding a suitable example to cherry pick. Here, 5 trades consecutively reach their near maximum limit orders.
Testing that I've done in the past tells me that it's frequently advantageous to increase the position size based on consecutive winners when a strategy is more than 50% accurate. My first idea after studying the initial results was to modify the forex money management strategy to pursue the consecutive winners.
While increasing the risk after consecutive winners works out statistically in your favor, it does add to the risk. It's entirely likely for a "winning" set of trades to start losing based solely on the money management strategy. There's no way to know whether your set of trades will be the lucky beneficiary of the extra risk or whether it will be the unlikely loser.
Sunday, March 11, 2012
Forex Trading officially surpasses $5 Trillion per day mark
Forex Trading officially surpasses $5 Trillion per day mark
http://www.bloomberg.com/news/2012-03-11/currency-trading-at-5-trillion-a-day-surpassed-pre-lehman-high-bis-says.html
Currency trading may have risen to a record $5 trillion a day in September, surpassing the peak reached before Lehman Brothers (LEHMQ) Holdings Inc.’s collapse in 2008, according to theBank for International Settlements.
http://www.bloomberg.com/news/2012-03-11/currency-trading-at-5-trillion-a-day-surpassed-pre-lehman-high-bis-says.html
Currency trading may have risen to a record $5 trillion a day in September, surpassing the peak reached before Lehman Brothers (LEHMQ) Holdings Inc.’s collapse in 2008, according to theBank for International Settlements.
Password Manager
A secure, single click password manager. Roboform is also great for filling out forms, with it's built in form-filler with multiple identities. Roboform online backup, browser embedding, and other features make it far more useful than it's free alternatives.
Download EES Password Manager Roboform
Download EES Password Manager Roboform
Tuesday, March 6, 2012
Fixing bugs in MQL4
MQL is a really simple scripting language. Debugging MQL, unfortunately, is not simple at all. The MetaEditor compiler that MetaQuotes provides simply does not incorporate the advanced tools that most coders are familiar with using.
MQL bug fixing problems
Visual Studio and other sophisticated IDEs (integrated development environments) incorporate a number of offerings that make it simple to fix code while the http://www.blogger.com/img/blank.gifprogrammer writes it. The best example of this are break points. A break point is a site in the code where the compiler informs the computer to stop executing the code when it arrives at that stipulated line.
Take the example where a trailing stop implements a new stop inaccurately. The normal instinct for most programmers would be to run the expert advisor on the visual backtester, then introduce break points on the sections of code immediately after the trailing stop calculations. Break points block the code, allowing the coder to scan inside the brains of the software to see what it thought at the time it made a decision. The key advantage in Visual Studio is that the values of all of the variables are clearly visible. It is possible to walk through the program step by step. Whenever one of the steps does not follow the desired rules, the necessary modification is usually obvious. MetaQuotes thankfully included break points in MQL5. They are not available in MQL4.
The lack of full intellisense support affects my programming speed more than anything. Intellisense observes the use of reserved words like OrderSelect() or ObjectGet(). The MetaEditor includes a basic intellisense, but it lacks the fine details that make it so convenient in Visual Studio.
I am used to programming in C# where I can type the first few letters of a variable or class, then the IDE fills out the rest. When I type "Mes" in C# and push the space bar, I know that the MessageBox option will show up (assuming that I declared the required namespace). The MetaEditor includes a list of options for reserved words. The programmer must then either pick the option with the mouse or push enter.
I know it seems trivial to require pushing enter in lieu of the space bar, but think about how many times code resuses the same reserved words or variables. The extra presses of keys really do add up to a lot of unnecessary typing motions. That's doubly true for a thirty year old that already wears a wrist brace for carpal tunnel pain.
The MetaEditor's biggest weakness is that it does not see variable names. We often write EAs that incorporate several thousand lines of code. Tracking the names of tens of variables poses its own challenges. When the coder types in the same set of variable names repeatedly, it would be nice to simply type the first three letters and move on. Copy and paste might provide a decent alternative. The problem is that variables usually group together. You cannot keep 5 different copy and paste items readily available.
The MetaEditor allows functions to return invalid types. Functions declared as double can return strings, integers or nothing at all. The MQL4 compiler does not track whether or not these are valid. It leaves it up to the coder to discover the invalid type during real time testing. This oversight is a nightmare for the unwitting programmer that mistakenly returns the wrong type.
This is even more true when a double function is erroneously returned to an integer variable. MQL4 does not prevent illegal double to int casts. Even worse, the expert advisor continues running with a 0 value for the interger instead of throwing an exception or error message. I cannot count how many hours that I've wasted tracking down variables that look accurate, only to realize that I declared the wrong data type. This usually happens when I'm on autopilot, pounding out code. What seems efficient at the time commonly costs several hours of hair pulling frustration.
MQL troubleshooting techniques
The MQL programmers on staff here usually resort to any of the following techniques. You may find that testing them in groups helps with improving the bug fixing process even more.
Debug compiler error
This one can be the most frustrating. The MetaEditor attempts to hint at which line of code causes the compiling error. I say attempts because it gets it wrong more often than it gets it right. There's nothing more irritating than looking at a perfectly legitimate line of code that the compiler flags as problematic.
I almost always resort to commenting out more and more large blocks of the expert advisor until the issue disappears. I begin with commenting out a single line of code. If that doesn't work, then I comment out ten lines. If that doesn't work, I might comment out whole functions or sections of code. When the compiler finally runs properly, you know that the last section of commented out code comprises the bug.
Next, you work backwards. Start with making the offending commented-out section smaller and smaller until the error comes back. Now, you have finally zeroed in on the actual source of the problem.
Debug in real time or on the backtester
My favored manner of bug fixing is to comment most of the relevant decision information onto the screen, which is done using the Comment() function. I then run the visual backtester, watching how the data behaves in relation to the visual information.
On screen comments are essentially jury-rigged break points. Controlling how and when they appear allows the coder to step through the code to uncover the issue. The only difference is that comments do not forcefully prevent the code from running. The text which appears is very small. Aside from that, I really like that fact that it's so robust. The comment function always works without a hitch, making it the best friend of a coder that's troubleshooting code.
Taking screenshots takes this to the next level. Whenever consumers ask questions about why an expert advisor behaved a certain way, the easiest answers come with screenshots. Commenting the imitation break points usually provide bullet proof answers - the coder and client can actually see what the Ea thought at the time it made a decision. MQL4 offers the WindowScreenShot() function to do this.
The Expert advisors that we program always take screenshots during pivotal actions, such as entering a trade or modifying an exit condition. The goal is to provide a visual account of every decision with an eye to answering future inquiries about the action.
Our default template includes a true/false variable called WriteScreenshots. Traders control whether they want to bother with this bug fixing feature or not. The only downside to it is that every recorded action eats up about 50kb of hard drive space.
Log files offer the last bug fixing option. The major drawback is that they are so ugly and burdensome to read. I almost always prefer screenshots for this reason.
Nonetheless, log files do have their place. The main way to use them is as error catchers. Whenever a course of action goes awry due to a problem with either MetaTrader itself or with a broker transaction, the log file is the easiest place to catch it.
Bug fixing MQL files is a skill that takes awhile to learn. The methods at the programmer's disposal are very different from those available to higher level languages. Once you get accustomed to working with the much more basic tools in the MetaEditor and MetaTrader, the bug fixing process goes a lot quicker.
Thursday, March 1, 2012
Building a trading strategy
One common aspect typifies almost all of the trading robots that we develop. The finished product does not exactly follow the initial design.
All programming projects proceed in one of two ways. You can either code a task from the peak to the bottom, or you can build from the bottom to the top. The former alternative represents the old way of thinking, especially within higher education. When lecturors assign a project, the conditions never change. The task remains static.
Real life hardly ever occurs like that. Instead, people change their minds. They view the product in progress and arrive at the conclusion that a small adjustment may substantially modify the outcome. The expert advisor programmer is forced to regularly habituate to the changing goal posts.
I always tell clients that they will recondition their minds about several of the rules during the project. Many stand firm that they will not. The customers do, after all, anticipate that the conditions will be correct.
Then, the rules unavoidably recondition for most of the clients. Theory falls apart in practice. The traders uncover the real problems that expert advisors face in the market.
Programmers developed a new method to harmonize with the steadily shifting rules. It's termed agile development. This pickle doesn't only occur with forex traders. It also happens to businesses constructing web sites, database coders saving information, and just about any project that involves human beings. Human beings are prone to changing their minds. It's our nature. Agile development is the programmer's acknowledgement that getting upset at customers for changing their minds is not reasonable.
The agile blueprint segments a project into time periods with an arranged list of goals. The team decides which objectives are most valuable and achievable within the established window. The period starts and everyone rushes to accomplish as much code as possible. Everyone regroups after 2-6 weeks to sort out the progress and to send the updated version to the customer.
Most expert advisor tasks only span a few weeks, so a precise carbon copy of the agile approach would not adapt very well with developing a trading robot. We chuck the strategy development method on its head by allocating graded goals with a cutoff of 1-2 business days. The goal is to aid communication, which is inevitably the most challenging obstacle in this field. We also make the method less rigid to speed things along.
The quick answers lead to a series of yes/no answers. The client either gets excited that it works correctly, or more likely, feels that we did not interpret him accurately. The authentic goal of this strategy is to get the user away from explaining the idea for the 1,000th time in the same, identical way. The last brief did not help the plan move along. Client feedback from functioning software guides the coder into comprehending the thought on his own without making the consumer believe that he's echoing himself.
I like to think of it as the client steering the driver's wheel while the programmer controls the gas pedal. The strategy development process only works when the steering and power move in synch.
When customers feel like an expert advisor has not made the desired progress, the first step is to assign goals one at a time. Too often, we receive reports with 20 different bugs. The emphasis on priority falls away as the desire to "fix everything and fix it now" drowns out the details of each request. I always like to take a step back and reaffirm our commitment to developing the strategy, but we can only handle so many issues at a time.
I have a very patient client in New Jersey (I never though I'd say "patient" and "New Jersey" in the same sentence") who ordered a lot of custom elements in a basic expert advisor. Most of the complications arose from the fact that I had to remotely access his computer with LogMeIn to do the programming. His indicator only has one license. Purchasing a second for the expert advisor programming was not economical.
The requirements to remotely access the computer and the amount of custom code turned a simple project into a complicated one. The way I addressed the issue was to remove almost all of the custom components and to replace them with code from our usual template. Now that the customer sees that the "strategy" stuff works, he found it much easier to write a checklist of bugs to fix. More importantly, seeing critical components of the code working reinforced his confidence in the company's programming ability.
Although he remained patient throughout the debugging process, I could hear the flagging confidence in his voice before we switched gears. Changing the direction of the plan showed him that the EA actually does work; it's the little pieces and how they fit together that are the problem. Of equal importance was the fact that he felt confident in my ability to deliver the results that he wants.
The consumer gets full credit for releasing control of the debugging process, even when he wasn't entirely comfortable doing so. The approach of papering over the custom steps with pre-programmed template code struck him as a step backwards. He nonetheless followed the programmer's lead, letting him drive the process with himself providing feedback where it was needed.
Both parties agree that we're once again heading in the right direction. The consumer sees obvious progress in his expert advisor. The programmer is able to manage the debugging process in a methodical, organized manner. Everyone is happy.
The most realistic way to develop a strategy is to confess that you don't precisely know where you're going. You know it's likely to involve a certain indicator and that it either trends or ranges. The best way to get ready for the journey is to acknowledge in advance that it is indeed a journey. You more than likely will make changes, most of which you cannot expect.
It's 100% certain that something is going to go wrong when you program your first project. It's your strategy, but it's really a two-man team that builds it. Make sure to pick someone that knows what do when problems come up.
Before you select your programming partner, be sure it's someone that you genuinely trust to get the project done. You absolutely want to find someone that provides excellent communication skills and knows the importance of a timely answer. Organizing the project and overcoming unexpected challenges assists everyone get back to their true desire of trading.
All programming projects proceed in one of two ways. You can either code a task from the peak to the bottom, or you can build from the bottom to the top. The former alternative represents the old way of thinking, especially within higher education. When lecturors assign a project, the conditions never change. The task remains static.
Real life hardly ever occurs like that. Instead, people change their minds. They view the product in progress and arrive at the conclusion that a small adjustment may substantially modify the outcome. The expert advisor programmer is forced to regularly habituate to the changing goal posts.
I always tell clients that they will recondition their minds about several of the rules during the project. Many stand firm that they will not. The customers do, after all, anticipate that the conditions will be correct.
Then, the rules unavoidably recondition for most of the clients. Theory falls apart in practice. The traders uncover the real problems that expert advisors face in the market.
Programmers developed a new method to harmonize with the steadily shifting rules. It's termed agile development. This pickle doesn't only occur with forex traders. It also happens to businesses constructing web sites, database coders saving information, and just about any project that involves human beings. Human beings are prone to changing their minds. It's our nature. Agile development is the programmer's acknowledgement that getting upset at customers for changing their minds is not reasonable.
The agile blueprint segments a project into time periods with an arranged list of goals. The team decides which objectives are most valuable and achievable within the established window. The period starts and everyone rushes to accomplish as much code as possible. Everyone regroups after 2-6 weeks to sort out the progress and to send the updated version to the customer.
Most expert advisor tasks only span a few weeks, so a precise carbon copy of the agile approach would not adapt very well with developing a trading robot. We chuck the strategy development method on its head by allocating graded goals with a cutoff of 1-2 business days. The goal is to aid communication, which is inevitably the most challenging obstacle in this field. We also make the method less rigid to speed things along.
The quick answers lead to a series of yes/no answers. The client either gets excited that it works correctly, or more likely, feels that we did not interpret him accurately. The authentic goal of this strategy is to get the user away from explaining the idea for the 1,000th time in the same, identical way. The last brief did not help the plan move along. Client feedback from functioning software guides the coder into comprehending the thought on his own without making the consumer believe that he's echoing himself.
I like to think of it as the client steering the driver's wheel while the programmer controls the gas pedal. The strategy development process only works when the steering and power move in synch.
Overcoming expert advisor programming challenges
When customers feel like an expert advisor has not made the desired progress, the first step is to assign goals one at a time. Too often, we receive reports with 20 different bugs. The emphasis on priority falls away as the desire to "fix everything and fix it now" drowns out the details of each request. I always like to take a step back and reaffirm our commitment to developing the strategy, but we can only handle so many issues at a time.
I have a very patient client in New Jersey (I never though I'd say "patient" and "New Jersey" in the same sentence") who ordered a lot of custom elements in a basic expert advisor. Most of the complications arose from the fact that I had to remotely access his computer with LogMeIn to do the programming. His indicator only has one license. Purchasing a second for the expert advisor programming was not economical.
The requirements to remotely access the computer and the amount of custom code turned a simple project into a complicated one. The way I addressed the issue was to remove almost all of the custom components and to replace them with code from our usual template. Now that the customer sees that the "strategy" stuff works, he found it much easier to write a checklist of bugs to fix. More importantly, seeing critical components of the code working reinforced his confidence in the company's programming ability.
Although he remained patient throughout the debugging process, I could hear the flagging confidence in his voice before we switched gears. Changing the direction of the plan showed him that the EA actually does work; it's the little pieces and how they fit together that are the problem. Of equal importance was the fact that he felt confident in my ability to deliver the results that he wants.
The consumer gets full credit for releasing control of the debugging process, even when he wasn't entirely comfortable doing so. The approach of papering over the custom steps with pre-programmed template code struck him as a step backwards. He nonetheless followed the programmer's lead, letting him drive the process with himself providing feedback where it was needed.
Both parties agree that we're once again heading in the right direction. The consumer sees obvious progress in his expert advisor. The programmer is able to manage the debugging process in a methodical, organized manner. Everyone is happy.
Realistic strategy development
The most realistic way to develop a strategy is to confess that you don't precisely know where you're going. You know it's likely to involve a certain indicator and that it either trends or ranges. The best way to get ready for the journey is to acknowledge in advance that it is indeed a journey. You more than likely will make changes, most of which you cannot expect.
It's 100% certain that something is going to go wrong when you program your first project. It's your strategy, but it's really a two-man team that builds it. Make sure to pick someone that knows what do when problems come up.
Before you select your programming partner, be sure it's someone that you genuinely trust to get the project done. You absolutely want to find someone that provides excellent communication skills and knows the importance of a timely answer. Organizing the project and overcoming unexpected challenges assists everyone get back to their true desire of trading.
Magic Numbers in MT4
The magic number is a Mt4 concept used to track the open positions of an EA. The http://www.blogger.com/img/blank.gifconcept allows the Expert advisor to distinguish the trades that it opened versus those that it did not.
Every car uses a license plate. When you see a car in a foreign state or even a foreign country, you notice that every plate that you see is unique. The police can utitlize the number to determine who owns the car.
Magic numbers act like the license plates for EA's. When an expert advisor sees an open trade, called a ticket, it frequently looks up its magic number. If the magic number of the ticket is identical to the number that the expert advisor expects, then the EA concludes that it needs to manage the position.
Magic numbers are valuable, especially when you want to trade multiple time frames of the identical currency pair. Traders repeatedly use settings that differ from those on M1 charts versus those that they would use on the day chart. If they used the Expert advisor with the identical magic number on all different time frames, the consequence would be chaos. The expert advisor would enter and exit positions with no rhyme or reason. Setting each Expert advisor to utilize an independent magic number prevents the robots from obstructing the others.
The magic number of a trade that you open yourself is 0.
The value of a magic number must fall between 0 and 2147483647. The MQL programming language calls that last number EMPTY_VALUE and reserves the name as an integer value.
Every car uses a license plate. When you see a car in a foreign state or even a foreign country, you notice that every plate that you see is unique. The police can utitlize the number to determine who owns the car.
Magic numbers act like the license plates for EA's. When an expert advisor sees an open trade, called a ticket, it frequently looks up its magic number. If the magic number of the ticket is identical to the number that the expert advisor expects, then the EA concludes that it needs to manage the position.
Magic numbers are valuable, especially when you want to trade multiple time frames of the identical currency pair. Traders repeatedly use settings that differ from those on M1 charts versus those that they would use on the day chart. If they used the Expert advisor with the identical magic number on all different time frames, the consequence would be chaos. The expert advisor would enter and exit positions with no rhyme or reason. Setting each Expert advisor to utilize an independent magic number prevents the robots from obstructing the others.
Magic number factoids
The magic number of a trade that you open yourself is 0.
The value of a magic number must fall between 0 and 2147483647. The MQL programming language calls that last number EMPTY_VALUE and reserves the name as an integer value.
Tuesday, February 28, 2012
10 Heresies of Finance
I set a reading list for new programming hires before they ever interact with customers. OneStepRemoved's newest programmer, Jon Rackley, comes with an outstanding background in programming, but almost no market knowledge. Jon is under my wing for the next few months. My goal is to fully indoctrinate him on my market views.
One of the most useful, high level overview of trading dynamics is Benoit Mandelbrot's The (Mis)behavior of Markets. My philosophy is that before you attack a problem, you first need to understand the problem. Building trading robots is the problem that our customers face every day. One reason that many would-be automated traders fail is that they understand very little about how markets function. They try to solve a problem that they don't understand very well.
Mandelbot's "10 heresies of finance" sums up my view of markets quite well. I made Jon read the book as part of his job training because has no trading experience. If you're relatively new to markets or think it's time to step back and re-evaluate your approach, then I highly recommend this book.
Jon found Mandelbrot's comparison between wind turbulence and market dynamics the most compelling part of his argument. It's conceptually difficult to accept the idea of human behavior acting like the wind. The math of the two behaviors is identical, which is somewhat difficult to accept emotionally. What I like about the argument is that it makes for a beautiful metaphor.
When a storm comes up, it's noteworthy. The average wind speed picks up. People notice that part. What everyone really notices and gossips about are the wind gusts. Staring out the window and watching the family oak tree bend a quarter way to the ground makes an impression.
Wind gusts are essentially market volatility. Events like a collapse of the euro or a surprise NFP number are the violent gusts of wind that make the jaws of traders everywhere drop to the ground. One gust begets another gust, packing themselves densely in time. Mandelbrot would reference the
increase in gusts as trading time speeding up.
If you enjoy these types of ideas, you might find our free resetting moving average indicator useful. It builds on Mandelbrot's fractal approach to markets, negating the idea of a true average. The resetting moving average tries to act as a minimum reference point by changing its period in step with market volatility.
One of the most useful, high level overview of trading dynamics is Benoit Mandelbrot's The (Mis)behavior of Markets. My philosophy is that before you attack a problem, you first need to understand the problem. Building trading robots is the problem that our customers face every day. One reason that many would-be automated traders fail is that they understand very little about how markets function. They try to solve a problem that they don't understand very well.
Mandelbot's "10 heresies of finance" sums up my view of markets quite well. I made Jon read the book as part of his job training because has no trading experience. If you're relatively new to markets or think it's time to step back and re-evaluate your approach, then I highly recommend this book.
The 10 heresies of finance
- Markets are turbulent.
- Markets are far more risky than the standard theories imagine. Mandelbrot does do a very good job throughout his book explaining how unpredictable and complex market behavior really is.
- Market timing matters greatly. Big gains and losses concentrate into small packages of time. He calls this trading time.
- Prices often leap, not glide. That adds to the risk.
- Time is flexible. Some times of day are more important than others
- Markets in all places and ages work alike. Human behavior never changes, even if the mood does
- Markets are inherently uncertain. Bubbles are inevitable. We will always be greedy
- Markets are deceptive.
- Forecasting prices may be perilous, but you can estimate the odds of future volatility
- In financial markets, the idea of “value” has limited value.
Jon found Mandelbrot's comparison between wind turbulence and market dynamics the most compelling part of his argument. It's conceptually difficult to accept the idea of human behavior acting like the wind. The math of the two behaviors is identical, which is somewhat difficult to accept emotionally. What I like about the argument is that it makes for a beautiful metaphor.
When a storm comes up, it's noteworthy. The average wind speed picks up. People notice that part. What everyone really notices and gossips about are the wind gusts. Staring out the window and watching the family oak tree bend a quarter way to the ground makes an impression.
Wind gusts are essentially market volatility. Events like a collapse of the euro or a surprise NFP number are the violent gusts of wind that make the jaws of traders everywhere drop to the ground. One gust begets another gust, packing themselves densely in time. Mandelbrot would reference the
increase in gusts as trading time speeding up.
If you enjoy these types of ideas, you might find our free resetting moving average indicator useful. It builds on Mandelbrot's fractal approach to markets, negating the idea of a true average. The resetting moving average tries to act as a minimum reference point by changing its period in step with market volatility.
Sunday, February 26, 2012
Wyoming House advances doomsday bill
CHEYENNE — State representatives on Friday advanced legislation to launch a study into what Wyoming should do in the event of a complete economic or political collapse in the United States.
House Bill 85 passed on first reading by a voice vote. It would create a state-run government continuity task force, which would study and prepare Wyoming for potential catastrophes, from disruptions in food and energy supplies to a complete meltdown of the federal government.
The task force would look at the feasibility of
Wyoming issuing its own alternative currency, if needed. And House members approved an amendment Friday by state Rep. Kermit Brown, R-Laramie, to have the task force also examine conditions under which Wyoming would need to implement its own military draft, raise a standing army, and acquire strike aircraft and an aircraft carrier.
The bill’s sponsor, state Rep. David Miller, R-Riverton, has said he doesn’t anticipate any major crises hitting America anytime soon. But with the national debt exceeding $15 trillion and protest movements growing around the country, Miller said Wyoming — which has a comparatively good economy and sound state finances — needs to make sure it’s protected should any unexpected emergency hit the U.S.
Read more: http://trib.com/news/state-and-regional/govt-and-politics/wyoming-house-advances-doomsday-bill/article_af6e1b2b-0ca4-553f-85e9-92c0f58c00bd.html#ixzz1nXq3mKqw
How to work with mt4 brokers with non-standard symbol suffixes
Many MT4 traders have the problem that brokers will change the default symbols in MT4 (i.e. from EURUSD to EURUSD.usd)
We wanted to have some programmatic access to the symbol list, outside of manually right clicking on market watch window.
EES posed this question in MQL4.com here: http://forum.mql4.com/46521
..and received an excellent resource: http://codebase.mql4.com/6134
This could be useful if a programmer wanted to automatically update the symbols he is trading, or for the development of multi-currency strategies.
Some brokers also may have multiple suffixes for the same pair, allowing for different order processing rules (or different spreads). Some may even offer 2 EURUSD suffixes in the same account.
Having automated use of available symbols allows an EA to not be locked to trade only the symbol where the EA is attached. For example if you have a multi-currency strategy that trades EUR/GBP every time a EUR/USD signal is generated, it would be possible using this method to automatically find EUR/GBP by searching for EURGBP in the symbols list.
We wanted to have some programmatic access to the symbol list, outside of manually right clicking on market watch window.
EES posed this question in MQL4.com here: http://forum.mql4.com/46521
..and received an excellent resource: http://codebase.mql4.com/6134
This could be useful if a programmer wanted to automatically update the symbols he is trading, or for the development of multi-currency strategies.
Some brokers also may have multiple suffixes for the same pair, allowing for different order processing rules (or different spreads). Some may even offer 2 EURUSD suffixes in the same account.
Having automated use of available symbols allows an EA to not be locked to trade only the symbol where the EA is attached. For example if you have a multi-currency strategy that trades EUR/GBP every time a EUR/USD signal is generated, it would be possible using this method to automatically find EUR/GBP by searching for EURGBP in the symbols list.
Tuesday, February 21, 2012
Martingale Lessons
In the spring of 2011, there was no denying it. I adopted a decidely jaded view of the stock market and had grown to despise an industry full of self proclaimed oracles. Most of these people in my view were a bunch of textbook salesmen and women selling various dreams of financial security and fame.
Through sheer stubborness I remained invested and recovered from the down years of 2008 and 2009. However, single digit gains were taxing my patience. I was miles from my longterm goals. There was no doubt I would be working well into my 60s and if I suffered any type of dementia, it was not hard to imagine myself greeting people at Walmart. When I stumbled upon Forex, you could say I was open to a change.
Through my work as an expert advisor programmer, I became involved in coding all manner of peoples trading strategies into small trading robots called expert advisors or custom indicators. I am still surprised by all the permutations of indicators, chart patterns, money management mechanisms, etc, that trader are dreaming up out there.
I would code them all up according to their specifications, slap them in a backtester to put them through their paces, fix any problems. Rinse and repeat until all was working to everyone's satisfaction. Many if not most were flawed for one reason or another, some would work exceptionally well in one type of market but perform poorly in others.
Some were tied to faulty indicators that would change their earlier output signals. Some were based on what I viewed as various arcane chart patterns which seemed one step removed (no pun intended) from witchcraft. Others were complex to the point where they would rarely trade and when they did would do so at the most inopportune time.
After perhaps a year of this, I received a Martingale strategy to code for an overseas customer. After coding and running his EA in the strategy tester, I remember becoming very excited with how well it was performing on every time period and data set in all markets. I spent more time testing this EA that day than most, even though I was not finding any problems with the coding. After seeing so many crash and burn that week, I enjoyed watching it do its thing.
To be sure, it was not the first Martingale I had seen and I had always discounted that ilk as being to closely tied to Vegas and gambling. But, like the ones I had seen before, it was doing a bangup job of piling up cash during my test runs. I guess after seeing the nth Martingale doing that I could no longer ignore this method of trading.
That afternoon and for many days following, whenever my thoughts would wander ideas would pop into my head as to what sort of Martingale I would program in MetaTrader for myself. It was truly something of an epiphany. Up to that point my whole view of forex was that of a high stakes market best left to big banks, institutional investors, and governments. I had never thought of running one of these programs on my own money.
Like most Martingales, mine would double its lot size with every loss. I suppose what was different about my Martingale from many but certainly not all others, is that mine would reverse the direction of its trades. This would solve trending markets, the achilles heal of single-direction-trade martingales. It would also use a Take Profit and Stop Loss that would be somewhat sensitive and self-adjusting to Average True Range. This was my answer to excessive volatility. Finally, I put in logic to have hidden take profits and stops. This would satisfy my probably unwarranted suspicion of brokers.
I'm not going to get too detailed on leverage, margin requirements and such. With a Martingale, you need to consider how many levels you're going to let it go before it gives up or resets its lot size. And don't kid yourself; it will go deep more often then you would think. You need to make sure you have enough money in a Martingale trading account to support your biggest possible trade.
I went through the trouble of setting up a forex account and started trading. The first month felt underwhelming. I recall the EA only did a little better than breakeven but my enthusiasm was still intact. The fact that I did not lose any money comforted me, making it feel safe to press on.
I continued tweaking the EA and adjusting how I would use it. By the end of the 2nd month, mostly through trial an error I found the EA worked best on the EURUSD and the 15 min period. That's when the results started to give me that euphoria that I had so badly wanted. By the end of the fourth month, morale was sky high.
I'm not going to give actual numbers since I used to always suspect other type articles of fish tails when they did that, but it was enough to make me a changed person. If you can recall the nursery rhyme "Sing a Song of Sixpence". The line "The King was in his counting house, Counting out his money" pretty much sums it up. That was me. It almost felt illegal and I actually hid the results from others fearing that I would jinx everything if I mentioned it. I would often pick up a calculator or paper and pencil calculate out what the sum might be after 5 10 20 years at the current rate of returns.
I'm not a market statistician or financial wizard of any sort, so I can't give a very elegant explanation of what happened. I do know that the financial problems in Europe during the fall of 2011 provided an extremely volatile and choppy market that did not agree with my reversing Martingale EA. It resulted in some whiplash-packed trading weeks and numerous reality checks.
All my gains for the year and then some were wiped out during a few weeks. The deceptive thing about a Martingale is that 99.9% of time it will look good and make money. When you test it, it is likely you won't catch it at its worst. It feeds a sense of denial. In this case results were good for several months before it lost money.
Once it starts losing, the trades become big very quickly. If you happen to be asleep when this happens, you're in for a big surprise as you rub the sleep from your eyes and peer at your monitor in the morning. I know some of you are thinking, "well, duh", and to be honest, knowledge of this possibility was always in the back of my head. I thought since the Martingale was a reversing one and would adjust to volatility I was at least somewhat immune to this. I suppose it could have been much worse with a more conventional Martingale.
Another thing to watch out for is meddling. When things started going bad I slipped into the bad habit of supervising the EA. I would stop trades just before they were about to recover and the net result was usually to make things worse.
Since that time, I have scaled back use of the Martingale. In some type markets I won't even run it. I also added a non trading window feature to it. Specifically I can set it up to shut itself down at specific times of the day. Now when I go to bed, if there are multiple big financial news reports scheduled for the morning, I set the EA to go inactive before and for sometime after this time to let the associated volatility work itself out.
I sleep better when I know it won't be running at night. It's no longer producing killer returns for me as the volatility will usually make you money, but I don't have to worry so much about the times when it will create losses, either. There's a price to pay for peace of mind. If that means trading less often, I'm all in favor.
Through sheer stubborness I remained invested and recovered from the down years of 2008 and 2009. However, single digit gains were taxing my patience. I was miles from my longterm goals. There was no doubt I would be working well into my 60s and if I suffered any type of dementia, it was not hard to imagine myself greeting people at Walmart. When I stumbled upon Forex, you could say I was open to a change.
The Allure of the Martingale
Through my work as an expert advisor programmer, I became involved in coding all manner of peoples trading strategies into small trading robots called expert advisors or custom indicators. I am still surprised by all the permutations of indicators, chart patterns, money management mechanisms, etc, that trader are dreaming up out there.
I would code them all up according to their specifications, slap them in a backtester to put them through their paces, fix any problems. Rinse and repeat until all was working to everyone's satisfaction. Many if not most were flawed for one reason or another, some would work exceptionally well in one type of market but perform poorly in others.
Some were tied to faulty indicators that would change their earlier output signals. Some were based on what I viewed as various arcane chart patterns which seemed one step removed (no pun intended) from witchcraft. Others were complex to the point where they would rarely trade and when they did would do so at the most inopportune time.
After perhaps a year of this, I received a Martingale strategy to code for an overseas customer. After coding and running his EA in the strategy tester, I remember becoming very excited with how well it was performing on every time period and data set in all markets. I spent more time testing this EA that day than most, even though I was not finding any problems with the coding. After seeing so many crash and burn that week, I enjoyed watching it do its thing.
To be sure, it was not the first Martingale I had seen and I had always discounted that ilk as being to closely tied to Vegas and gambling. But, like the ones I had seen before, it was doing a bangup job of piling up cash during my test runs. I guess after seeing the nth Martingale doing that I could no longer ignore this method of trading.
That afternoon and for many days following, whenever my thoughts would wander ideas would pop into my head as to what sort of Martingale I would program in MetaTrader for myself. It was truly something of an epiphany. Up to that point my whole view of forex was that of a high stakes market best left to big banks, institutional investors, and governments. I had never thought of running one of these programs on my own money.
Like most Martingales, mine would double its lot size with every loss. I suppose what was different about my Martingale from many but certainly not all others, is that mine would reverse the direction of its trades. This would solve trending markets, the achilles heal of single-direction-trade martingales. It would also use a Take Profit and Stop Loss that would be somewhat sensitive and self-adjusting to Average True Range. This was my answer to excessive volatility. Finally, I put in logic to have hidden take profits and stops. This would satisfy my probably unwarranted suspicion of brokers.
I'm not going to get too detailed on leverage, margin requirements and such. With a Martingale, you need to consider how many levels you're going to let it go before it gives up or resets its lot size. And don't kid yourself; it will go deep more often then you would think. You need to make sure you have enough money in a Martingale trading account to support your biggest possible trade.
Happy Days and Riding the Martingale bull
I went through the trouble of setting up a forex account and started trading. The first month felt underwhelming. I recall the EA only did a little better than breakeven but my enthusiasm was still intact. The fact that I did not lose any money comforted me, making it feel safe to press on.
I continued tweaking the EA and adjusting how I would use it. By the end of the 2nd month, mostly through trial an error I found the EA worked best on the EURUSD and the 15 min period. That's when the results started to give me that euphoria that I had so badly wanted. By the end of the fourth month, morale was sky high.
I'm not going to give actual numbers since I used to always suspect other type articles of fish tails when they did that, but it was enough to make me a changed person. If you can recall the nursery rhyme "Sing a Song of Sixpence". The line "The King was in his counting house, Counting out his money" pretty much sums it up. That was me. It almost felt illegal and I actually hid the results from others fearing that I would jinx everything if I mentioned it. I would often pick up a calculator or paper and pencil calculate out what the sum might be after 5 10 20 years at the current rate of returns.
When a Martingale goes bad
I'm not a market statistician or financial wizard of any sort, so I can't give a very elegant explanation of what happened. I do know that the financial problems in Europe during the fall of 2011 provided an extremely volatile and choppy market that did not agree with my reversing Martingale EA. It resulted in some whiplash-packed trading weeks and numerous reality checks.
All my gains for the year and then some were wiped out during a few weeks. The deceptive thing about a Martingale is that 99.9% of time it will look good and make money. When you test it, it is likely you won't catch it at its worst. It feeds a sense of denial. In this case results were good for several months before it lost money.
Once it starts losing, the trades become big very quickly. If you happen to be asleep when this happens, you're in for a big surprise as you rub the sleep from your eyes and peer at your monitor in the morning. I know some of you are thinking, "well, duh", and to be honest, knowledge of this possibility was always in the back of my head. I thought since the Martingale was a reversing one and would adjust to volatility I was at least somewhat immune to this. I suppose it could have been much worse with a more conventional Martingale.
Another thing to watch out for is meddling. When things started going bad I slipped into the bad habit of supervising the EA. I would stop trades just before they were about to recover and the net result was usually to make things worse.
Summary
Since that time, I have scaled back use of the Martingale. In some type markets I won't even run it. I also added a non trading window feature to it. Specifically I can set it up to shut itself down at specific times of the day. Now when I go to bed, if there are multiple big financial news reports scheduled for the morning, I set the EA to go inactive before and for sometime after this time to let the associated volatility work itself out.
I sleep better when I know it won't be running at night. It's no longer producing killer returns for me as the volatility will usually make you money, but I don't have to worry so much about the times when it will create losses, either. There's a price to pay for peace of mind. If that means trading less often, I'm all in favor.
Monday, February 20, 2012
EA Trading vs Manual Trading
A client from Malaysia emailed yesterday asking about the merits of trading with EAs. He went through the usual motions of buying "quite a number of EAs [that] promised to make millions and ended up crashing or losing my live accounts." He is not the first, nor will be the last to experience the roller coaster.
He watched a number of videos on YouTube that emphasize the importance of discretionary trading over automated forex robots. They claim, correctly, that a human being understands subtleties that computer programs miss. They then stretch the argument even further and say that EA trading is a fallacy. Given his bad luck, he wondered to what extent that I agree with the argument.
The argument oversimplifies a difficult concept. Many of you know that I'm a self-confessed math geek. I like numbers. They are logical and they do not vary. What appeals so strongly to me about trading is that you can take a complex process and mold it into a set of logical rules to follow. It's a wonderfully challenging problem. You'll never crack it entirely, but I know from experience that you can develop something that works.
My gripe with most commercial EAs is that the people selling them are not traders. They are internet marketers that stumbled into a highly profitable niche industry. They tick the right boxes for their audience.
Most US forex traders are white men between the ages of 40-65 with an unusually high tendency to earn six figure incomes. The marketing gang knows that themes of financial security and the idea of an job that's not miserable sells well. They focus more on getting that message across than developing a decent product.
EAs are only as good as their designers. OneStepRemoved has designed 1,000+ EAs. I've never seen anyone accidentally program a profitable expert advisor. The ones that do turn out well almost always have years or even decades of trading experience. The rules that they propose usually include subtle distinctions that differentiate between various market conditions and types of volatility.
Those types of people are not marketers. They spent their career trading. It's part of the reason that the meagerly stocked talent pool of traders does not get come into contact with the marketing crew. The other problem is that the guys coming up with strategies worth selling have no incentive to do so.
Unless you're FAP Turbo or MegaDroid, the potential income for most trading robot oriented business is six figures. I've worked with several multimillion dollar trading educators. They seem to do better in the long run owing to the relationships that they form with their customers. They are the exception rather than the rule.
Now look at how much a truly profitable trading strategy can make. Millions. No marketers or affiliates required. No web site. Just a computer and a server. There's no effort required beyond passive monitoring. If you had a trading system that generated blockbuster returns, would you even bother with trying to form it into a business? I wouldn't.
Traders like this do exist. It's just that they're about 0.01% of the trading population. I know this from my brokerage experience because once in a blue moon I fielded their phone calls. They were the perfect clients. They traded all the time, rarely called (only people losing money call their broker) and when they did, it was for some routine matter like withdrawing profits.
The only thing that stood out about them was their account balance. These people are the source of the dream, and they're what keep the remaining 99.99% of algorithmic trading developers going.
Knowing how sweet the setup is if you have something worth selling, and also knowing that the financials really don't make sense for selling a blockbuster system, I honestly would not consider purchasing an EA from the internet. I would never yield control over my account to something that behaves in a manner that I don't understand.
When the drawdown inevitably comes, I would have no way to feel comfortable that the strategy is actually viable in the long haul. I would suspect that the end is nigh and that I better shut down the account before the losses accumulate. How anyone can make an informed financial decision with strictly marketing material is beyond my understanding.
Trading rules restrict discretionary traders in a way that I find beneficial. It cuts down on their tendency to over-trade, forcing them into situations where patience more often that not wins out.
Todd over at Triple Threat Trading runs a trading education business. He loves to go on and on about how he makes customers quantify their approaches as a set of rules. It's easy enough to lose money in forex or futures.
The rule based approach minimizes those hazards. Following rules eliminates many of the unknown variables from your performance evaluation. Knowing that you followed the rules and still lost reduces the blow to the ego. It's easier to say "the rules lost" instead of "I lost."
The logic naturally flows into the idea of automating the strategy. The chief advantage is that the computer does not fatigue or tire. It follows the rules blindly. So long as a human remotely monitors the execution, the semi-automated approach often works well for established, rule based traders.
Making a trading robot does create new problems like deciding whether to only trade during active sessions or to leave the expert advisor on 24 hours a day. These are usually problems that resolve themselves with a few months of hands on experience.
The main lesson that I learned while running my forex account up this year with a gray box EA is how important it is to stay patient. Most of the systems that I trade personally tend to eat losses way too quickly. I have yet to determine how to quantify patience in trading terms, despite my love for quantifying things.
Certain rules do work dramatically better during different market conditions. The key is to separate the conditions, then apply the strategy selectively. Most expert advisor designers try to develop a strategy that works all the time.
It doesn't work that way. When the EA starts to hit a wall on the design, the trader responds with new additions and tweaks to force the performance to improve the general performance.
What I'm working on now is a set of rules to identify various market types. I like to think of it as "don't compare lawn mowers when it's raining." There's a time and a place for each approach and behavior. The trick isn't so much the rules, but deciding when to apply them.
Categorizing the market helps make the strategy selection process far easier. If the market is highly volatile and ranging, then select a scaling strategy that works well in ranging markets. If the market is trending quietly, then perhaps a basic trend strategy would work perfectly well.
He watched a number of videos on YouTube that emphasize the importance of discretionary trading over automated forex robots. They claim, correctly, that a human being understands subtleties that computer programs miss. They then stretch the argument even further and say that EA trading is a fallacy. Given his bad luck, he wondered to what extent that I agree with the argument.
The wrong people are involved in selling EAs (usually)
The argument oversimplifies a difficult concept. Many of you know that I'm a self-confessed math geek. I like numbers. They are logical and they do not vary. What appeals so strongly to me about trading is that you can take a complex process and mold it into a set of logical rules to follow. It's a wonderfully challenging problem. You'll never crack it entirely, but I know from experience that you can develop something that works.
My gripe with most commercial EAs is that the people selling them are not traders. They are internet marketers that stumbled into a highly profitable niche industry. They tick the right boxes for their audience.
Most US forex traders are white men between the ages of 40-65 with an unusually high tendency to earn six figure incomes. The marketing gang knows that themes of financial security and the idea of an job that's not miserable sells well. They focus more on getting that message across than developing a decent product.
EAs are only as good as their designers. OneStepRemoved has designed 1,000+ EAs. I've never seen anyone accidentally program a profitable expert advisor. The ones that do turn out well almost always have years or even decades of trading experience. The rules that they propose usually include subtle distinctions that differentiate between various market conditions and types of volatility.
Those types of people are not marketers. They spent their career trading. It's part of the reason that the meagerly stocked talent pool of traders does not get come into contact with the marketing crew. The other problem is that the guys coming up with strategies worth selling have no incentive to do so.
Unless you're FAP Turbo or MegaDroid, the potential income for most trading robot oriented business is six figures. I've worked with several multimillion dollar trading educators. They seem to do better in the long run owing to the relationships that they form with their customers. They are the exception rather than the rule.
Now look at how much a truly profitable trading strategy can make. Millions. No marketers or affiliates required. No web site. Just a computer and a server. There's no effort required beyond passive monitoring. If you had a trading system that generated blockbuster returns, would you even bother with trying to form it into a business? I wouldn't.
Traders like this do exist. It's just that they're about 0.01% of the trading population. I know this from my brokerage experience because once in a blue moon I fielded their phone calls. They were the perfect clients. They traded all the time, rarely called (only people losing money call their broker) and when they did, it was for some routine matter like withdrawing profits.
The only thing that stood out about them was their account balance. These people are the source of the dream, and they're what keep the remaining 99.99% of algorithmic trading developers going.
Knowing how sweet the setup is if you have something worth selling, and also knowing that the financials really don't make sense for selling a blockbuster system, I honestly would not consider purchasing an EA from the internet. I would never yield control over my account to something that behaves in a manner that I don't understand.
When the drawdown inevitably comes, I would have no way to feel comfortable that the strategy is actually viable in the long haul. I would suspect that the end is nigh and that I better shut down the account before the losses accumulate. How anyone can make an informed financial decision with strictly marketing material is beyond my understanding.
What most expert advisors miss
Trading rules restrict discretionary traders in a way that I find beneficial. It cuts down on their tendency to over-trade, forcing them into situations where patience more often that not wins out.
Todd over at Triple Threat Trading runs a trading education business. He loves to go on and on about how he makes customers quantify their approaches as a set of rules. It's easy enough to lose money in forex or futures.
The rule based approach minimizes those hazards. Following rules eliminates many of the unknown variables from your performance evaluation. Knowing that you followed the rules and still lost reduces the blow to the ego. It's easier to say "the rules lost" instead of "I lost."
The logic naturally flows into the idea of automating the strategy. The chief advantage is that the computer does not fatigue or tire. It follows the rules blindly. So long as a human remotely monitors the execution, the semi-automated approach often works well for established, rule based traders.
Making a trading robot does create new problems like deciding whether to only trade during active sessions or to leave the expert advisor on 24 hours a day. These are usually problems that resolve themselves with a few months of hands on experience.
The main lesson that I learned while running my forex account up this year with a gray box EA is how important it is to stay patient. Most of the systems that I trade personally tend to eat losses way too quickly. I have yet to determine how to quantify patience in trading terms, despite my love for quantifying things.
Certain rules do work dramatically better during different market conditions. The key is to separate the conditions, then apply the strategy selectively. Most expert advisor designers try to develop a strategy that works all the time.
It doesn't work that way. When the EA starts to hit a wall on the design, the trader responds with new additions and tweaks to force the performance to improve the general performance.
What I'm working on now is a set of rules to identify various market types. I like to think of it as "don't compare lawn mowers when it's raining." There's a time and a place for each approach and behavior. The trick isn't so much the rules, but deciding when to apply them.
Categorizing the market helps make the strategy selection process far easier. If the market is highly volatile and ranging, then select a scaling strategy that works well in ranging markets. If the market is trending quietly, then perhaps a basic trend strategy would work perfectly well.
Friday, February 17, 2012
How to handle the Trade Context is Busy error
The trade context is busy error causes nightmares for scalpers and high frequency traders. MetaTrader chokes when they open up too many charts or place too many order. The problems stem from how MT4 processes and receives information from the broker server.
The OrderSend() command, which MQL programmers use to open a trade, also tells the MT4 client terminal to await a server response. Everything locks up. The code cannot do anything until the server permits the client terminal to continue. More importantly, the trading terminal does not allow any additional trades to open.
Consider a scenario where you trade EURUSD, AUDUSD and USDJPY currency pairs on H1 charts. MT4 must await an incoming tick before MQL allows it to make a trading decision. I discussed this issue several months ago with time based orders.
The ticks of multiple currencies often group together during liquid hours. If the terminal receives the incoming ticks within a few hundred milliseconds of each other, MetaTrader creates its own bottleneck.
The AUDUSD tick comes in first. The Expert Advisor sees a trading opportunity and decides to open an AUDUSD trade with OrderSend(). 150 ms later a EURUSD tick comes in and creates another trading opportunity. The EURUSD EA, which runs on a completely different chart, is not allowed to trade.
Our Expert Advisor programmers check for "Trade Context is Busy" errors before sending orders to the broker. When the situation exits, the EA checks the trade context every 50 ms up to 50 times.
It is not uncommon for MetaTrader orders to take a second or more to execute. The original AUDUSD trade from our example is still hanging out there. The EURUSD EA enters a 50 ms cycle where it constantly asks, "Did the trade context open up?"
Then the USDJPY tick comes in 70 ms after the EURUSD tick. USDJPY encounters the same trade context issue, kicking it into a separate, "Is the trade context busy" loop.
When the AUDUSD trade finally executes, it leaves the EURUSD and USDJPY expert advisors in a frantic race to discover the newly opened trading window. The first EA that discovers that the trading context busy error disappeared will send its request first. The second EA, unfortunately, cannot communicate with the first. Although the second EA will discover that the trade context opened up, it will receive a trade conext is busy error when it attempts to trade; the firs EA already sent a trading request. The second expert advisor allots itself a maximum of 10 failures before it gives up on the trade.
Assuming that the EURUSD discovers the new trading window first, then the USDJPY receives its first failed order attempt. The USDJPY once again inquires about the trading context every 50 ms. The EURUSD trade eventually executes, allowing the USDJPY to finally enter the market.
Most of our customers never notice this delay, which means that we did our job. There is, however, a breaking point where MetaTrader simply cannot issue enough orders at the speed the trader desires. Any combination of charts where 10 or more orders go off near-simultaneously will result in missed trading opportunities.
MetaQuotes designed MT4 with the trade context error in order to limit the processing requirements of the brokers. As I've stated repeatedly, the brokers get MetaQuotes' preferential treatment because they are the ones paying MetaQuotes' fee. MetaTrader is not a high frequency trading platform. It's not in the broker's interest to support high speed trading. It usually opens them up to too many traders taking advantage of arbitrage opportunities.
The OrderSend() command, which MQL programmers use to open a trade, also tells the MT4 client terminal to await a server response. Everything locks up. The code cannot do anything until the server permits the client terminal to continue. More importantly, the trading terminal does not allow any additional trades to open.
Example
Consider a scenario where you trade EURUSD, AUDUSD and USDJPY currency pairs on H1 charts. MT4 must await an incoming tick before MQL allows it to make a trading decision. I discussed this issue several months ago with time based orders.
The ticks of multiple currencies often group together during liquid hours. If the terminal receives the incoming ticks within a few hundred milliseconds of each other, MetaTrader creates its own bottleneck.
The AUDUSD tick comes in first. The Expert Advisor sees a trading opportunity and decides to open an AUDUSD trade with OrderSend(). 150 ms later a EURUSD tick comes in and creates another trading opportunity. The EURUSD EA, which runs on a completely different chart, is not allowed to trade.
Retry delay for Trade Context is Busy
Our Expert Advisor programmers check for "Trade Context is Busy" errors before sending orders to the broker. When the situation exits, the EA checks the trade context every 50 ms up to 50 times.
It is not uncommon for MetaTrader orders to take a second or more to execute. The original AUDUSD trade from our example is still hanging out there. The EURUSD EA enters a 50 ms cycle where it constantly asks, "Did the trade context open up?"
Then the USDJPY tick comes in 70 ms after the EURUSD tick. USDJPY encounters the same trade context issue, kicking it into a separate, "Is the trade context busy" loop.
When the AUDUSD trade finally executes, it leaves the EURUSD and USDJPY expert advisors in a frantic race to discover the newly opened trading window. The first EA that discovers that the trading context busy error disappeared will send its request first. The second EA, unfortunately, cannot communicate with the first. Although the second EA will discover that the trade context opened up, it will receive a trade conext is busy error when it attempts to trade; the firs EA already sent a trading request. The second expert advisor allots itself a maximum of 10 failures before it gives up on the trade.
Assuming that the EURUSD discovers the new trading window first, then the USDJPY receives its first failed order attempt. The USDJPY once again inquires about the trading context every 50 ms. The EURUSD trade eventually executes, allowing the USDJPY to finally enter the market.
Most of our customers never notice this delay, which means that we did our job. There is, however, a breaking point where MetaTrader simply cannot issue enough orders at the speed the trader desires. Any combination of charts where 10 or more orders go off near-simultaneously will result in missed trading opportunities.
MetaQuotes designed MT4 with the trade context error in order to limit the processing requirements of the brokers. As I've stated repeatedly, the brokers get MetaQuotes' preferential treatment because they are the ones paying MetaQuotes' fee. MetaTrader is not a high frequency trading platform. It's not in the broker's interest to support high speed trading. It usually opens them up to too many traders taking advantage of arbitrage opportunities.
Friday, February 3, 2012
High Frequency Forex Seminar
One exciting opportunity popped up while I'm in Dublin, Ireland, next week. Best of all, it's free and open to the public. If you're in the western Europe and would like to discuss trading in person, I'd love to meet you.
Trinity College Dublin invited me to present a graduate level seminar to MSc students in Finance and Alternative Investments on Wednesday, February 8, at 6 pm. The seminar will be hosted in the MBA room, which is on the second floor of the business school. The topic will be high frequency market making in forex.
Topics for the high frequency forex trading seminar (about 10 minutes per subject):
Market making versus price taking
Comparing frequency to expectation. The more you trade, the more you make
Liquidity risks and self-feedback loops
Technical approaches and limitations
Trinity College Dublin invited me to present a graduate level seminar to MSc students in Finance and Alternative Investments on Wednesday, February 8, at 6 pm. The seminar will be hosted in the MBA room, which is on the second floor of the business school. The topic will be high frequency market making in forex.
Topics for the high frequency forex trading seminar (about 10 minutes per subject):
Market making versus price taking
Comparing frequency to expectation. The more you trade, the more you make
Liquidity risks and self-feedback loops
Technical approaches and limitations
Subscribe to:
Posts (Atom)