I am interested to learn if there is an array based way to accomplish the following. Please note this is an example only however any potential coding strategy will work for my intended system.
Let's say we start by drawing a trend-line from the beginning of the year, that is from the very first bar that appears for the calendar year. The trend-line is drawn between the close of the first bar of the year and the current close. Sounds simple enough so far but keep in mind that the trend-line will change with each new bar.
What I would like to do is calculate for each bar of the year, the number of bars (close) above the trend-line and the number of bars closing below the trend-line. The trend-line being the line from the evaluated bars close back to the close of the first bar.
The point where I am stumbling is that the intermediate points of the trend-line change with each bar, so in effect a new trend-line has to be calculated for each bar, and then the number of closes above and below can be determined from this new trend-line. This process has to be repeated for each bar.
Please note, I can already accomplish this using a bar-by-bar loop and matrices but the execution of a backtest using this approach with many tickers is very time-consuming. I thought I'd ask if any experts could help point me in the right direction to help convert this to array based processing.
Logically, the number of bars above and below the trendline will change with each new bar. You could create two arrays called e.g. numbarsabove() and numbarsbelow(), and then as each new bar closes, loop back to bar 0, sum these values for number above and below, and then fill the array element for the most recent bar with these values. AB should be able to do with in a snap. Next, you'll have the data you want, and then just use it.
Looking again at your question (title), there are probably 500 ways to memorize past occurrences in a time series. Examples are autocorrelation (temporal), mutual information (spatial), time series forecasting, and non-linear based function approximation, that is, predictive analytics.
So what you're asking is how to learn from the past trending, in order to inform rule-based decisions. The first way to approach this would be to parametrize the information by collapsing it so that it represents what you wants but with out looping. But sooner or later, fitting parameters is always iterative, so you have to loop through the iterations anyhow -- there's no way out. In machine learning, this would be accomplished by use of generative or discriminative models, and use of a greedy uphill metaheuristic, like particle swarm optimization, ant colony optimization, or a genetic algorithm.
Getting back to this (had to leave earlier), you are essentially asking to somehow use the information related to all the values of numbbarsabove (the trend) and numbarsbelow for each new bar, but without using all that data for every bar over and over again during optimization. This involves collapsing information down to a lower number of dimensions. If you are not going to develop a novel system and want a parametric approach, then use a trend-following system like Schaff Trend Cycle (by PatternExplorer), Jurik DDR (jurikres.com), Instant trendline (by WiseTrader), all available for AB. John Ehlers has also developed free indicators like the Ultimate Oscillator for AB. You get what you pay for (or program), so the free version of anything is not typically profitable. DDR will collapse redundant (non-informative) bars and features down to lower dimensions before they are into an e.g. neural network.
Thank you for the response but please don't worry too much about the final calculation- it's not actually my trading strategy but just a calculation I can use to illustrate my point. This is a higher level discussion regarding the possibility of performing such a calculation using arrays instead of loops.
Your response has made me think about my approach and quite frankly the code I created to accomplish my goal involved two bar loops per security. The first one creating a trend-line at each bar and storing the value in a BarCount x BarCount matrix. Then a second loop to extract the time-line and perform various calculations.
I am certain, I can condense this to one bar loop... but can it be further condensed to array processing? Let me know if I've complicated you too much, I appreciate your answer, you've already half solved my quandary.
@paligap, sounds good. Indeed, you can always take the delta of the lookback numbarsabove() and numbarsbelow() the trend line for each bar, and then use that. A more experienced indicator freak should be able to chime in the disccusion, and say e.g.: :just use indicator X, which does what you want. I learned early on in trading that any public indicators are not profitable, and many commercial indicators are junk as well.
@paligap: I don't believe there's any way to accomplish this without a bar-bar-bar loop because as you pointed out, in a test covering N bars there will be N different trend lines to evaluate. Unless you need the trend lines later, I don't think there's any need for a BarCount x BarCount matrix. For each bar, you can just create the trend line in a regular array (no looping required) and then compare that array to the Close array (again no looping required). Unless you object to grouping Close == TrendLine into either the "above" or "below" bucket, you don't even need two comparisons, since Below = YTDBars - Above.