Showing posts with label algorithm. Show all posts
Showing posts with label algorithm. Show all posts

Wednesday, October 1, 2014

Caught On Tape: HFT Algo Manipulating GOOGL 1000 Times Per Second

It is very common to find examples of stock quotes changing rapidly - hundreds and sometimes thousands of times per second in a single stock. At the extreme, we've seen in excess of 25,000 quote changes in a single stock in one second of time or less (this page has a chart that documents every extreme example). Often there are no trades during these events. Sometimes a simple pattern evolves from the quote price changes, such as in the case of a certain High Frequency Trading (HFT) algorithm that we've recently seen run every day in Google stock.
This particular algorithm starts with a bid (or offer) several dollars away from the bids (offers) from one of the other 10 exchanges trading Google Class A stock (symbol GOOGL). We've also seen this algo running in other higher priced stocks. The algo in this example only appears to run from the Nasdaq-Boston (BOST) exchange. In the chart below, we show bids and offers color coded by reporting exchange (there are 10 exchanges in GOOGL). Note that these are "top of book" quotes - that is, they are the highest bid price and lowest ask price from that exchange. The best top-of-book bid and ask become the National Best Bid/Offer (NBBO) and  is shown as light gray shading in these charts. Note, this algo only affects the NBBO when it gets near the end of its price stepping loop.
The algo starts with an order to buy 100 shares at $581.87. This is replaced, sometimes only milliseconds later, with an order to buy 100 shares at $581.88 (1 penny higher). Over the course of 1.5 seconds, this process repeats another 253 times, ending with a order to buy at $584.41. Within less than a second, the $584.41 order is canceled and replaced with an order several dollars lower, and the cycle repeats.
In the case below, the number of quote changes from this HFT algo is averaging 175 per second, but during some periods the rate approaches 1000 per second (1 per millisecond).
1. GOOGL bids and asks color coded by reporting exchange over a 5 second period of time.


Now, some folks (particularly the math/physics challenged) will say:
"So what? HFT needs to be able to cancel quotes fast so they can tighten spreads, add liquidity and lower costs."
The problem is that when HFT cancels a quote after just 1 millisecond (ms), then anyone located more than 93 miles (150 km) away will see a stale quote. Worse, they won't know it's stale unless and until they try to act on it and wait for a response. The animation below shows how this works. Note, this animation assumes zero processing time on the part of the investor or any other real-world delays. In other words, this is the best possible case, and it will be much worse for the investor in the real-world.
  • The animation starts at an Elapsed Time of 0 microseconds. 1 microsecond (?s) is 1 millionth of a second. 1000 ?s is 1 millisecond (ms).
  • HFT places an order at the top of an exchange's order book, which causes a quote to be transmitted out to investors.
  • An investor 93 miles away receives the quote after 500 ?s (0.5 ms).
  • Assuming the perfect case, the investor immediately acts on the quote and transmits an order to the exchange (really their broker, but let's assume a perfect world).
  • The exchange won't see this investor's order until a total elapsed time of at least 1000 ?s (1 ms).
  • HFT changes its mind after 1 ms and cancels the order - just before the investor's order arrives.
  • The investor won't know that their order failed for another 500 ?s or a total of 1500 ?s since HFT sent the initial order!


Effectively, when HFT changes its mind 1000 times a second (or after 1 millisecond) anyone located outside the (93 miles/150 km) circle below will receive stale quotes:
An expanded map is shown below. Each red circle shows how far quotes can travel before expiring at different update rates. For example, the ring labeled 150 is how far quotes will get if HFT is canceling and replacing 150 quotes each second. At 150 quotes/second, people in Chicago will be processing quotes they can't act on! People in Los Angeles have it even worse - quotes changing just 38 times a second will render them all obsolete by the time Los Angelians or anyone on the Google Campus in Stanford California first sees them.
Now, look back at the example in Google above - that HFT algo was changing Google quotes an average of 175 per second, which means those quotes were expiring somewhere between the two rings labeled 150 and 250 in the map below.

Back to our Google example, let's zoom out and see how often this HFT algo is running.
2. Zoomed out to just under 30 seconds of time (the zoom box is detailed in chart 1 at the top).



3. Zoomed out to about 18 minutes of time (the zoom box is detailed in the chart above). Note how often this algo runs!
Each green sliver is made up of one HFT algo's bids or offers changing 1 penny at a time at rates exceeding 100 times and sometimes 1000 times a second.


4. Zoomed out showing 9am to about 3pm Eastern Time (the zoom box is detailed in the chart above).
Note the distinct periods of time when this algo runs. Too bad other HFT algo's don't make themselves this visible.



5. Another close-up, showing how the algo does the same thing on the offer size.



6. A different pattern in a different stock on another day. This involves multiple exchanges and affects the NBBO.
Note: The chart shows 3,549 quote changes in about 1/4 of a second of time. These are not rare! Not many people outside of the exchange datacenter will see these quotes before they expire. Yet everyone will have to process them (because there is no way to know how long before they are canceled).


Friday, September 16, 2011

A Breakthrough in Algorithm Design

Computer scientists at Carnegie Mellon University have devised an algorithm that might be able to solve a certain class of linear systems much more quickly than today's fastest solvers.

http://cacm.acm.org/magazines/2011/9/122799-a-breakthrough-in-algorithm-design/fulltext

http://www.cc.gatech.edu/~ninamf/papers/md_ml_jcss.pdf

http://theory.cs.cmu.edu/

CM Rwanda?

http://www.cmu.edu/news/stories/archives/2011/september/sept15_rwanda.html


Quantitative Analysis of Real-Time Systems Using Priced Timed Automata



http://cacm.acm.org/magazines/2011/9/122806-quantitative-analysis-of-real-time-systems-using-priced-timed-automata/fulltext