For broker-dealers, the difference in performance of their algorithms in executing a trade of just fractions of a basis point can be the difference between winning and losing a mandate from the buy side. Eric Stockland, managing director, electronic trading at BMO Capital Markets, said that as a result, brokers need to constantly run experiments on their algos to ensure they meet client needs and to improve performance.

BMO’s electronic trading team constantly runs experiments and analysis in two broad classes. The first is a deep forensic analysis of a specific algorithm which is performed for many clients, many times a month, in order to investigate performance and make potential improvements. The second is large scale long-duration experimentation, or A/B testing, which also happens regularly but can last several months.

“Every client has a different definition of best execution for their own trading strategy, so we tailor solutions to their needs, which involves experimentation, iteration and collaboration,” Stockland added.

He compared the iterative process of improving algos to learning to play darts. A complete novice will first try just to get their dart onto the board. As they practise their technique improves, and they get closer to the bullseye. Once they are really good, they can regularly hit their target.

Stockland* said: “Whenever you put customization in place for a client, you want to show them that you’re doing everything that you told them you would do. People use the cliche that algorithms are a ‘black box’ and one of the many objectives of forensic analysis is to make that box more transparent.”

Collaborative Benefits

Collaboration leads to multiple benefits when experiments can be run in partnership with an asset manager or owner.

The perspective of an asset manager or owner is invaluable and also has the advantage of decreasing the risk of ‘groupthink’, as their view comes from outside the broker and its collective perspective.

“You have the buy-in that allows you to experiment more aggressively which we couldn’t do without their permission, as it is their order and their underlying investors,” said Stockland.

BMO partnered with OMERS, the Ontario Municipal Employees Retirement System, a jointly sponsored defined benefit pension plan which had net assets of $114bn as of June 30, 2021, to explore the impact of IEX Group’s D-Limit order type.

IEX Exchange fully launched its D-Limit, discretionary limit order type, in October 2020 after finally receiving approval from the US Securities and Exchange Commission. The venue’s machine learning signal seeks to predict how the market is going to move in the next fraction of a second. D-Limit then uses the power of the IEX Signal to move an order out of the way if the price is about to crumble or become imminently stale i.e., the order avoids being “run over” when the price is unstable.

BMO upgraded its technology infrastructure to support the novel workflow associated with IEX’s D-Limit order type in the spring of 2021. After a discussion with Rob Gouley, Equity Trading Principal at OMERS, BMO was given approval to run a joint experiment with an algorithm to isolate and evaluate the impact of D-Limit. Gouley stated, “At OMERS, best execution is recognized as a process, not just a result, and thoughtful experimentation is integral to the best execution process. We chose to partner with BMO on this project because of their deep market microstructure expertise and frankly, because BMO asked the right question: what can we do to differentiate ourselves and add value to your trading process over at OMERS?”

An analysis was performed of a VWAP algo to sell 30,800 Microsoft shares with a limit price of $243.98 on 13 May 2021. Stockland said the size of the order, which is medium to small for an institution, was one reason why it made a great candidate for disclosure.

“OMERS was extraordinarily courageous to publicly disclose the symbol, size and day,” he added. “That level of disclosure is radically new.”

During the experiment, a single child D-Limit order to sell with a limit price of $243.98 was repriced by IEX in multiple Crumbling Quotes prior to it executing at $244.05, for a savings of $0.07 per share. Overall, the parent order’s total child order arrival price improvement was $518.05 averaged over all shares executed, amounting to $.0168 per share in savings.

BMO acknowledged that benchmark performance cannot be extrapolated from a single order, but experimentation allows the team to dig into an individual order to learn how microstructure can benefit child-level executions. As execution quality improves for child orders, then the performance gains may accrue towards improving parent level benchmarks.

Cutting Out ‘Inefficiency Monetizers’

But before the D-Limit order was introduced it was not possible to sell at a higher price than a limit order on an exchange according to Stockland.

“D-Limit is radical because you can do better than your limit by cutting out people who monetize the inefficiency of the market,” he added. “The asset manager has some protection against latency arbitrage and the impact on results is eye-popping.”

The results of the experiment highlighted D-Limit’s benefit for asset managers and the investing public – such as maximising returns for the more than 525,000 beneficiaries of the OMERS pension plan, who are drawn from Ontario’s municipal sector. In addition to receiving better execution, BMO and OMERS also did not have to place child orders on venues with higher adverse selection.

“Selling 35,000 shares of Microsoft will generally move the stock but you want to move it the least amount possible,” said Stockland.

When running experiments in live markets it is always a huge challenge to control for the impact of broad price movements versus the impact of an algo on a given security. However, statistical techniques can be used to ‘find the signal’ through the noise.

“It is this empirical and methodical approach to trading which forces brokers into competition to the benefit of both asset managers and end investors,” said Stockland.

Stockland compared the process to a professional sports team using marginal gains to improve their performance.

“You have got to come up with new plays every season to make your product better,” he added. “If you stop innovating, you are dead.”

*Eric Stockland worked at IEX from July 2016 through October 2020, is listed as an inventor on IEX’s D-Limit patent application, and owns an illiquid equity position in the IEX Exchange Group.