Backtesting Debug

[JeanChristophe]

Hello,
I have an error message while testing my strategy. I am computing the PriceVolDistribution for every point as part of my strategy. Here is below the code used, based on this post Volume at Price --> Explore Possible?
l

//Volume line
priceH = IIf( H == L, H + 1e-2, H); // to avoid error if H = L , 
Lookback = 63;

if(BarCount >=100)
{
for ( i = 0; i < BarCount; i++ )
    {
    if (i < lookback+1)
    {
    vol_line[i] = 0;
	}
	else
	{
	//Compute VAP
	_TRACE( "Hello world");
	mx = PriceVolDistribution( priceH, L, V, bins = 100, true, i-lookback, i );
	mat_sort = MxSortRows(mx, False, 1);
	vol_line[i] = mat_sort[0][0];
    }
    }

I have an error message stating that priceH cannot be lower than priceL. The message is self explanatory, I just do not understand why this is happening.
I do not find a way to return on which ticker the error is occuring, so that I can check the raw data. I have tried using _TRACE to return some text (starting with just a simple Hello World as a starter), with no success (no info in the log window).
Is the_TRACE function the correct solution?

PS: I have added the if(BarCount >=100) because I think I have some empty tickers. Not sure this is the smartest way of checking this. Any advice would be welcomed

[nsm51]

hello

instead of
priceH = IIf( H == L, H + 1e-2, H);
doing this at the beginning

if you think it has to do with empty tickers or bad data
why dont you perform the check per bar by nesting it in IF inside

else
	{
      if( /* Good bar check  */  ) {
	   //Compute VAP
   	   _TRACE( "Hello world");
	   mx = PriceVolDistribution( priceH, L, V, bins = 100, true, i-lookback, i );
	   mat_sort = MxSortRows(mx, False, 1);
	   vol_line[i] = mat_sort[0][0];
      } else {
        // Trace bad bars here
         vol_line[i] = /* default value */;
      }
    }

[nsm51]

it also struck me that PriceH and PriceL is the H and L of the range so you might not be able to go plainly the way you are.

[fxshrat]

Even if you set O H L C to single value and volume to zero you will not get error with that priceH line.

H = 100;
L = C = O = H;
V = 0;

So this got nothing to do with that.
But you may use AlmostEqual instead of equality check.
priceH = IIf(AlmostEqual(H,L), H + 1e-2, H );

But no matter if this

H = 100;
L = C = O = H;
V = 0;

priceH = IIf(AlmostEqual(H,L), H + 1e-2, H ); 

// PriceVolDistribution code below
// ....

or that

H = 100;
L = C = O = H;
V = 0;

priceH = IIf(H==L, H + 1e-2, H ); 

// PriceVolDistribution code below
// ....

I don't get any error but just straight lines.

@JeanChristophe

I have an error message stating that priceH cannot be lower than priceL.

You seem to have invalid data where L > H !!
Have you imported incorrectly?
Or maybe you have overwritten L array somewhere in that code (L = ... or Low = ... ) and becoming larger than H.

If you do such non sense

H = 100;
L = C = O = H;
V = 0;
L = 101;

Then obviously you will get this

So message is pretty clear.

And conclusion is invalid input data.

And BTW if you intend to use PriceVolDistribution in some trading system code then why do you need to interate entire barcount? That's waste considering heaviness of that function. So instead of that why not storing only when your other rules are TRUE?

And as reminder to you if you copy code then credit belongs into the code (no matter which copied code of whoever made it) but not removed.
It is you coming here asking and copy and pasting and having no clue (in your first post even writing "here is my code" which you have deleted). In short you are completely lost.
No "thank you" whatsoever. Just copy and pasting and grabbing grabbing grabbing.

[nsm51]

i didn't see ur reply @fxshrat

thanks for the write-up. I haven't used pricevoldistribution but i had a funny doubt but i re-read and sorted it.
so basically, calling this function on ever bar is indeed something i didn't see the need for.

[JeanChristophe]

Hi guys,

Thanks for the feedback.

And as reminder to you if you copy code then credit belongs into the code (no matter which copied code of whoever made it) but not removed.

@fxshrat, I let the link to your piece of the in the core message, I will add it also in the code directly as a comment.

So instead of that why not storing only when your other rules are TRUE ?

Indeed, thank you. I used a cross(C, vol_line) as an entry condition, need to figure out how to implement it only when other conditions are true.

You seem to have invalid data where L > H !!
Have you imported incorrectly?
Or maybe you have overwritten L array somewhere in that code ( L = ... or Low = ... ) and becoming larger than H .

That is, I think, the core of the issue. raw data being invalid. I have used the Auto Update Quotes tool and the import from AmiQuote, using Yahoo data (not super confident on how reliable Yahoo data are).

The PriceVolDistribution is literally the first evaluation I do in the code, I am not calling L or H before. Backtest is only crashing at 58%, I have lots of tickers that are evaluated properly.

I wanted to be able to pinpoint on which ticker the backtest is failing to go & check manually the data, but I do not manage to return any message related to the ticker under evaluation at the moment of the crash.
I tried using _TRACE with no sucess.

[fxshrat]

If you have invalid data where L > H then simply do an Exploration via single line of code.

Filter = L > H;

[JeanChristophe]

Facepalm... Now I feel stupid. Thanks fxshrat for the tip.

[Tomasz]

Under normal conditions Low > High would never happen in the database, UNLESS you deliberately turned OFF all error checking during import using $ALLOWNEG flag.

I always say, do not change the defaults unless you know what you are doing because defaults (error checking ON) are here to prevent errors/mistakes.

[JeanChristophe]

I need to look that ALLOWNEG flag, but as far as I am aware of, I didn't change any default setting.
I will check as soon as I get back to my home computer & try to sort it out.

[JeanChristophe]

aving a look on the documentation, I am positive, I cannot have modified $ALLOWNEG, I only downloaded data via AMIQUOTE, without modifying any parameter.

After doing the exploration, here is the result:

ELJ ticker

[Tomasz]

AmiQuote stored data in files with .AQH extension and uses aqh.format file. The $ALLOWNEG is an importer command and if it is specified it would be inside format file.
Check the ELJ.aqh files downloaded and verify the content of aqh.format (inside Formats).
Look for bad data there or for incorrect format definition.

[JeanChristophe]

@Tomasz
Here is the content of the aqh.format. Last modified date of this file is 2017, so I think this is the original version with no modification from me.

$FORMAT Date_DMY,Open,High,Low,Close,Skip,Volume
$SKIPLINES 0
$BREAKONERR 0
$SEPARATOR ,
$DEBUG 1
$AUTOADD 1
$CONT 1
$GROUP 254
# the following switches are optional - please consult
# the read-me for description of those options
# $RAWCLOSE2OI 1   
# $RECALCSPLITS 1
# $ROUNDADJ 4

Regarding ELJ, I updated AmiQuote this morning, and ELJ ticher has changed a lot:

After a check on Yahoo website, this is consistent with what they show:

However, checking on another website, ELJ data are not quite the same:

All in all, I think ELJ data on Yahoo side are screwed up.

I still have an error on SSI with a high lower than the low for one day, and no error message during the AmiQuote import.

Within the SSI.aqh , i see the values but nothing weird in the file (appart for this low value being higher than a high on June 5th 2014).

Overall, it is not an issue, I can manually correct the data or ignore this ticker.

I also share some code based on fxshrat recommendation, in case someone is interested. Nothing fancy, but it works for me :).

//// Volume Distribution, based on fxshart code, ref: https://forum.amibroker.com/t/volume-at-price-explore-possible/8910/2
priceH = IIf( H == L, H + 1e-2, H);

function volatP(lookback, i)
{
if (i > lookback+1)
	{
	mx = PriceVolDistribution( priceH, L, V, bins = 100, true, i-lookback, i);
	mat_sort = MxSortRows(mx, False, 1);
	result = mat_sort[0][0];
	}
else
	{result = 0;
	}
return result;
}


Buy_conditions_partial =  //list of conditions;


for( i = 0; i < BarCount; i++ )
{
	if (Buy_conditions_partial[i] == 1)
		{
		vol[i] = volatP(63, i);
		vol[i-1] = volatP(63,i-1);
		}
}

Final_Conditions = Buy_conditions_partial AND Ref(C, -1) < Ref(vol,-1) AND C > vol;
Buy = Final_Conditions;

%Rest of strategy%

@fxshrat & @nsm51 thanks for your help, I can now evaluate my strategy as I expect

[Tomasz]

There is a built-in tool for data checks. No need to use formulas.

You can use
Tools->Database Purify
menu, for much more detailed checks.