Tuesday, March 6, 2012

Fixing bugs in MQL4

MQL is a really simple scripting language. Debugging MQL, unfortunately, is not simple at all. The MetaEditor compiler that MetaQuotes provides simply does not incorporate the advanced tools that most coders are familiar with using.

MQL bug fixing problems

Visual Studio and other sophisticated IDEs (integrated development environments) incorporate a number of offerings that make it simple to fix code while the http://www.blogger.com/img/blank.gifprogrammer writes it. The best example of this are break points. A break point is a site in the code where the compiler informs the computer to stop executing the code when it arrives at that stipulated line.

Take the example where a trailing stop implements a new stop inaccurately. The normal instinct for most programmers would be to run the expert advisor on the visual backtester, then introduce break points on the sections of code immediately after the trailing stop calculations. Break points block the code, allowing the coder to scan inside the brains of the software to see what it thought at the time it made a decision. The key advantage in Visual Studio is that the values of all of the variables are clearly visible. It is possible to walk through the program step by step. Whenever one of the steps does not follow the desired rules, the necessary modification is usually obvious. MetaQuotes thankfully included break points in MQL5. They are not available in MQL4.

The lack of full intellisense support affects my programming speed more than anything. Intellisense observes the use of reserved words like OrderSelect() or ObjectGet(). The MetaEditor includes a basic intellisense, but it lacks the fine details that make it so convenient in Visual Studio.

I am used to programming in C# where I can type the first few letters of a variable or class, then the IDE fills out the rest. When I type "Mes" in C# and push the space bar, I know that the MessageBox option will show up (assuming that I declared the required namespace). The MetaEditor includes a list of options for reserved words. The programmer must then either pick the option with the mouse or push enter.

I know it seems trivial to require pushing enter in lieu of the space bar, but think about how many times code resuses the same reserved words or variables. The extra presses of keys really do add up to a lot of unnecessary typing motions. That's doubly true for a thirty year old that already wears a wrist brace for carpal tunnel pain.

The MetaEditor's biggest weakness is that it does not see variable names. We often write EAs that incorporate several thousand lines of code. Tracking the names of tens of variables poses its own challenges. When the coder types in the same set of variable names repeatedly, it would be nice to simply type the first three letters and move on. Copy and paste might provide a decent alternative. The problem is that variables usually group together. You cannot keep 5 different copy and paste items readily available.

The MetaEditor allows functions to return invalid types. Functions declared as double can return strings, integers or nothing at all. The MQL4 compiler does not track whether or not these are valid. It leaves it up to the coder to discover the invalid type during real time testing. This oversight is a nightmare for the unwitting programmer that mistakenly returns the wrong type.

This is even more true when a double function is erroneously returned to an integer variable. MQL4 does not prevent illegal double to int casts. Even worse, the expert advisor continues running with a 0 value for the interger instead of throwing an exception or error message. I cannot count how many hours that I've wasted tracking down variables that look accurate, only to realize that I declared the wrong data type. This usually happens when I'm on autopilot, pounding out code. What seems efficient at the time commonly costs several hours of hair pulling frustration.

MQL troubleshooting techniques

The MQL programmers on staff here usually resort to any of the following techniques. You may find that testing them in groups helps with improving the bug fixing process even more.

Debug compiler error

This one can be the most frustrating. The MetaEditor attempts to hint at which line of code causes the compiling error. I say attempts because it gets it wrong more often than it gets it right. There's nothing more irritating than looking at a perfectly legitimate line of code that the compiler flags as problematic.

I almost always resort to commenting out more and more large blocks of the expert advisor until the issue disappears. I begin with commenting out a single line of code. If that doesn't work, then I comment out ten lines. If that doesn't work, I might comment out whole functions or sections of code. When the compiler finally runs properly, you know that the last section of commented out code comprises the bug.

Next, you work backwards. Start with making the offending commented-out section smaller and smaller until the error comes back. Now, you have finally zeroed in on the actual source of the problem.

Debug in real time or on the backtester

My favored manner of bug fixing is to comment most of the relevant decision information onto the screen, which is done using the Comment() function. I then run the visual backtester, watching how the data behaves in relation to the visual information.

On screen comments are essentially jury-rigged break points. Controlling how and when they appear allows the coder to step through the code to uncover the issue. The only difference is that comments do not forcefully prevent the code from running. The text which appears is very small. Aside from that, I really like that fact that it's so robust. The comment function always works without a hitch, making it the best friend of a coder that's troubleshooting code.

Taking screenshots takes this to the next level. Whenever consumers ask questions about why an expert advisor behaved a certain way, the easiest answers come with screenshots. Commenting the imitation break points usually provide bullet proof answers - the coder and client can actually see what the Ea thought at the time it made a decision. MQL4 offers the WindowScreenShot() function to do this.

The Expert advisors that we program always take screenshots during pivotal actions, such as entering a trade or modifying an exit condition. The goal is to provide a visual account of every decision with an eye to answering future inquiries about the action.

Our default template includes a true/false variable called WriteScreenshots. Traders control whether they want to bother with this bug fixing feature or not. The only downside to it is that every recorded action eats up about 50kb of hard drive space.

Log files offer the last bug fixing option. The major drawback is that they are so ugly and burdensome to read. I almost always prefer screenshots for this reason.

Nonetheless, log files do have their place. The main way to use them is as error catchers. Whenever a course of action goes awry due to a problem with either MetaTrader itself or with a broker transaction, the log file is the easiest place to catch it.

Bug fixing MQL files is a skill that takes awhile to learn. The methods at the programmer's disposal are very different from those available to higher level languages. Once you get accustomed to working with the much more basic tools in the MetaEditor and MetaTrader, the bug fixing process goes a lot quicker.