No problem, just read the docs, was: "big problem" with SparseCompress/SparseExpand

AFL Programming
1

The Amibroker I am using (6.30.0) seems to have a big problem with SparseCompress function.

My Aim: I am plotting intraday data (say, 5 min candles). I want to plot MA (20 periods) of Volume of only first bar of the day. So, basically, I want to grab Vol of only first bar of the day of last 20 days, then create Average of these volumes and plot.

I used this quote:

DayStarts = (DateNum() != Ref(DateNum(), -1)); 

V_Compressed = SparseCompress(DayStarts, V);

MA_V_Compressed = MA(V_Compressed, 20);
MA_V_Expanded = SparseExpand(DayStarts, Ref(MA_V_Compressed,-1)); 

Plot( MA_V_Expanded, "Avg. First bar Volume", colorLightOrange, styleHistogram|styleThick);

When I run this code, it all seems fine. But when I start zooming in and zooming out my chart (using + and - signs), the plot disappears. Basically, the plot appears only at a very 'zoomed out' view.

The BIG PROBLEM:
Ok. so, to get to the crux of the problem, I thought of plotting the compressed version only (without expanding). My expectation was that it should give me a continuous plot because the array is yet compressed. I am not even calculating MA of the compressed array. Just plotting the compressed array of Volume directly.

So, I used this code:

DayStarts = (DateNum() != Ref(DateNum(), -1)); 

V_Compressed = SparseCompress(DayStarts, V);

Plot( V_Compressed, "Compressed Vol", colorLightOrange, styleHistogram|styleThick);

As I run the code, I get the continuous array of Vol as per the expectation.

BUT, the moment I click anywhere in the chart, the entire plot is messed up. Plot changes completely and I get only few entries at the right end of the chart.

My Conclusion:
It seems the SparseCompress function works ONLY at a very ZOOMED OUT level.

1 Like
2

There is zero problem!

You should read here about QuickAFL.

So for example disabling QuickAFL. (Note: May be COSTLY!)

SetBarsRequired(sbrAll); // disables QuickAfL, Note: May be COSTLY!

dn = DateNum();
DayStarts = dn != Ref(dn, -1); 

V_Compressed = SparseCompress(DayStarts, V);

Plot( V_Compressed, "Compressed Vol", colorLightOrange, styleHistogram|styleThick);

In order to just calculate required number bars for last 20 days of set interval:

daysback = 20;

SetBarsRequired(daysback*inDaily/Max(1,Interval()));

dn = DateNum();
DayStarts = (dn != Ref(dn, -1)); 

V_Compressed = SparseCompress(DayStarts, V);

MA_V_Compressed = MA(V_Compressed, daysback);
MA_V_Expanded = SparseExpand(DayStarts, Ref(MA_V_Compressed,-1)); 

Plot( MA_V_Expanded, "Avg. First bar Volume", colorLightOrange, styleHistogram|styleThick);
6 Likes
3

WOW, this works like charm !
Thanks a lot fxshrat :slight_smile:

4

fxshrat,

I am trying to edit the below function so that it will calculate the MA based on the 1500 bar's close. I am using hourly data. It is calculating the MA just not anchoring to the 1500 values.

Thanks,

Mike

InputDate_1 = "2022-10-24 15:00:00";
daysback = 2;
SetBarsRequired(daysback*inDaily/Max(1,Interval()));
dn = _DT( InputDate_1) ;//DateNum();
DayStarts = dn;//(dn != Ref(dn, -1)); 

C_Compressed = SparseCompress(DayStarts, C);

MA_C_Compressed = MA(C_Compressed, daysback);
//MA_V_Expanded = SparseExpand(DayStarts, Ref(MA_V_Compressed,-1)); 
MA_C_Expanded = SparseExpand(DayStarts, MA_C_Compressed);
AddColumn(MA_C_Expanded,"MA",1.6);
5

DayStarts needs to be an array of boolean values, perhaps something like:

DayStarts = TimeNum() == 150000;

The code you posted is setting DayStarts to a single date, which means your SparseCompress isn't actually doing anything.

6

Hi Matt,

Thank you for time & insights. I did try your code and a few variations but when using that the code did not produce any moving average in "Explore".

I will keep trying.

Thanks again,

Mike

7

Check the timestamps on your intraday bars. Are any actually timestamped 15:00:00, or are they perhaps 14:59:59? You'll need your TimeNum() check to match the data timestamps or else change the logic a bit to capture the last bar timestamped on or before 15:00:00.

8

Hourly data is stamped 15:00:00.

I am not sure why it is not calculating the MA on the Close of that bar.

Will keep at it.

Thanks,

Mike

9

I find the interactive debugger helpful in situations like these, because you can look at what's in your Sparse array. Also, I don't see any Filter statement in your original post; without that, you'll get no output at all.

This code appears to work for me:

sampleTime = 150000;
daysback = 2;
SetBarsRequired(daysback*inDaily/Max(1,Interval()));
dn = DateNum();
tn = TimeNum();
DayStarts = tn <= sampleTime AND
			(Ref(tn,1) > sampleTime OR Ref(dn,1) > dn);

C_Compressed = SparseCompress(DayStarts, C);

MA_C_Compressed = MA(C_Compressed, daysback);
MA_C_Expanded = SparseExpand(DayStarts, MA_C_Compressed);
AddColumn(MA_C_Expanded,"MA",1.6);
Filter = DayStarts;

Also note that with SparseExpand the resulting array will contain Null for all points not referenced by the query points array.

3 Likes
10

Hi Matt,

Your code worked perfectly - thank you!

I made two edits. I did not mention that when I run "explore" I only pull the 1500 bar. The edit allows me to pull only the MA of the 1500 bar. My error in not letting you know when I posted the question.

Thank you for your help!

Mike

InputDate_1 = "2022-10-24 15:00:00";
sampleTime = 150000;
daysback = 2;
SetBarsRequired(daysback*inDaily/Max(1,Interval()));
dn = DateNum();
tn = TimeNum();
DayStarts = tn <= sampleTime AND
			(Ref(tn,1) > sampleTime OR Ref(dn,1) > dn);

C_Compressed = SparseCompress(DayStarts, C);

MA_C_Compressed = MA(C_Compressed, daysback);
MA_C_Expanded = SparseExpand(DayStarts, MA_C_Compressed);
//AddColumn(MA_C_Expanded,"MA",1.6);
AddColumn( Lookup( MA_C_Expanded, (_DT( InputDate_1) )) , "MA", 1.6) ;
Filter = DayStarts;