Pragma, a multi-asset quantitative trading technology provider, has published a new research paper highlighting the difficulties traders face when attempting to measure execution quality.

Algo “wheels” have become more widely used by traders with the goal of identifying which execution algorithm or broker performs better. Pragma demonstrates through 500 simulated experiments of production trading data across Q2 and Q3 2020 that even using several months of data form such a wheel, the wrong algorithm can be chosen almost half of the time.

Pragma presents several best practices for avoiding such mistakes, including a new trading benchmark developed by Pragma, Trajectory shortfall. This lower-noise variant of VWAP shortfall is designed to help identify the better performing execution algorithm more reliably or with less trading data. Pragma’s simulated example shows that using Trajectory shortfall can reduce the likelihood of an error when choosing the better algo by a factor of 3 over using VWAP shortfall.

Pragma also shares a list of best practices for traders to work into their TCA processes to further avoid errors, such as square-root weighting of orders, and ensuring there are no idiosyncratic differences in order flow between the execution algos.

David Mechner, CEO of Pragma, said: “More traders are using algo wheels to compare execution algorithms from multiple providers more systematically, which is great. But correctly interpreting the data that comes out of these experiments is challenging. Naïve interpretation can lead to “garbage-in, garbage-out” decisions that appear quantitative, but are really random, and subvert the goals of best execution. We shared this set of best practices to help traders get more value out of their data, and to be able to better recognize situations when they simply don’t yet have enough data to make a decision.”

Source: Pragma