Greetings –
I want to say nothing that detracts from the AmiBroker platform and its capabilities. I continue to believe and publicly state that AmiBroker is by far the best of the “traditional” trading system development platforms.
The difficulties being discussed in this thread and forum arise from limitations that restrict all traditional platforms.
- The models that are available for development using traditional platforms are almost always limited to individual “decision trees.”
- The paradigm is “identify a pattern or compute an indicator, then see what happens after.”
- There are limited facilities to determine the “hyperparameters” of the model.
- There are limited alternatives to measure the fitness of the model.
- There are limited facilities to validate the model.
As background – I began studying and applying artificial intelligence, including to financial problems, in the late 1960s while I was in graduate school. My mentors were among AI’s first generation. Some of my early research contributed to what is now know as the K-nearest neighbor technique. I have continued study and application of AI to financial data throughout my life.
Neural networks were one of the first AI applications that had promise and could be run on desktop computers. In the 1990s, at the request of one of my employers, I tried very hard to fit NN models to stock data. And failed.
That has since changed. AI capabilities useful for home development of trading systems started to become available about 2000, and mushroomed over the past few years with Python and scikit-learn. Hedge funds and banks were among the first to show high profit using AI and machine learning (ML). Refer to David Shaw, James Simons, etal.
In April 2014 I spoke at a conference in Australia and presented a workshop introducing ML-based trading system development. During one Q&A session, I was asked when I thought AI/ML would overtake traditional trading systems. I said two years. I was too conservative. It happened before that. AI and ML facilities that can be run on desktop computers are now state-of-the-art for trading system development.
The techniques are continuing to improve rapidly. At that same conference, I was asked what was the best ML technique. There is a formal theorem that shows there is not a single technique that is always best, but I suggested beginning with support vector machines together with genetic algorithms. That is no longer the case. As of today, ensembles, boosting, and stacking are winning AI competitions. Refer to Kaggle. All of these are supported by Python and scikit-learn, will run on any home computer – Windows, Mac, or Linux – and are free.
As a very rough and admittedly unscientific comparison. Assume a single decision tree system fits data well. If it was a trading system, it would be tradable. Using one of the machine learning scoring metrics, it might score 70 on a scale of 0 to 100. Five years ago any of us would have considered it an excellent system. Today, at home, ensembles, boosting, and / or stacking fit those models to that same data better. And regularly score mid 90s.
All five of the limitations I listed above are lifted for machine learning.
- There are dozens to hundreds of models available. Those available in scikit-learn have a common interface. You can test and compare twenty or more in a single computer run. Individual “decision trees” are essentially never at the top.
- The paradigm is “identify an important event, then see what happened earlier.”
- Cross validation routines are available to determine the “hyperparameters” of the model without using up degrees of freedom or contaminating out-of-sample data.
- There are unlimited alternatives to measure the fitness of the model. All of the computations necessary to computer CAR25 can be done “on the fly” during development.
- There are unlimited facilities to validate the model.
Additionally, the trading management system follows easily, computes CAR25 for any number of alternative fund allocation, ranks them, and indicates which to use and at what position size. Day-by-day, trade-by-trade.
The discussion we are having here is only one aspect of skills and facilities needed to improve trading systems. Among others, all of which you can do with AmiBroker, are:
- Work with a single issue long/flat.
- Mark to market daily and use state signals.
- Understand that risk is “the” primary limitation. Be willing to forgo opportunity in order to avoid losing trades. Know your personal risk tolerance. Know the risk of whatever you are considering trading. Compute the risk-normalized profit potential using your best estimate of future performance.
- Understand the importance of the mantra:
A. Trade frequently.
B. Trade accurately.
C. Hold a short period.
D. Avoid losses.
- Modern portfolio theory is dead (actually, it was never alive – it just had very vocal supporters). Work with a single issue / model combination per system.
- CAR25 is a Dominant metric. It is far superior to mean-variance / modern portfolio theory. Never use Sharpe ratio when its use penalizes winners. Rank all alternative uses of funds and allocate all available funds to that alternative that has the highest CAR25. Splitting funds is suboptimal.
Returning to the difficulties of implementing CAR25 as a development metric ----
Tomasz is providing an outstanding development platform in AmiBroker. It is powerful, fast, inexpensive, extremely well supported. By far, the best available.
I did not include in the list of limitations above those that AmiBroker has already overcome, but that many other traditional platforms have not. Limited objective function metrics, lack of two-stage processing as with CBT, inability to use state signals, lack of out-of-sample testing for validation, etc. AmiBroker is outstanding in its capabilities in these areas.
Traditional platforms are the extensions of the early days of trading and technical analysis – tape reading, chart reading, trend lines, flags, divergences, chart patterns, indicators, and eventually rule-based decision tree models.
The trading system development community has been extremely slow to adopt the scientific method. Many (most?) do not have technical educations. It is understandable that there is fear among them that their profession is becoming obsolete. I regularly receive bitter criticism from readers (or non-readers), whose names you would recognize, of my research and suggestions.
In summary.
If you want to stay with a traditional platform and individual decision tree models, AmiBroker is by far the best choice.
But machine learning is here to stay and it produces better trading systems. As long as you are trading an issue where your system has an acceptable return, continue to do that. Carefully monitor your performance and be quick to take a system offline when it begins to show signs it is no longer competitive.
Be aware that there is learning curve to machine learning. Expect to read a lot – dozens of books. (Begin with Daniel Kahneman’s “Thinking, Fast and Slow.”) And to brush up on your math (through linear algebra and calculus), statistics (it is pretty much all Bayesian now), and programming (Python).
Thanks for listening,
Best regards, Howard
