I have some computation that consumes large amount of time. In order to save time I store the processed value in static variables and persist. I call this value barScore . It is longest distance a line drawn from the given bar can travel (towards right) without intersection. This may not mean anything to you but in my world it has meaning. The meaning of barScore and it's computation has nothing to do with my problem. Every bar has two BarScore (s) each. One is BarScoreHH (High to High) and the other BarScoreLL (Low to Low). I create/store barscore for different timeframes with different variable names. So I add the interval() to the static variable name to differentiate.
Everything works perfectly when I work on one single timeframe (lets say 10 minutes) and in the same database. BarScores are generated, stored statically and persisted, retrieved everything works perfectly. I have a stock universe of 80 stocks and it works flawlessly. I can see that PersistVars.bin getting created and getting loaded on startup.
But the moment I change timeframe for example If i was working on 10 minutes earlier and now i switch to 20 minutes which means I ran my code on 20 minutes timeframe which in turn generated barscore for the said timeframe and stores/persists, it starts loosing values for the earlier 10 minutes timeframe. Some stocks still hold the value for 10 minutes but many stocks show value as 0. This happens to roughly 50% of my stocks. Some stocks lose data partially for example a stock may lose value from bar 500 to last bar. And some stocks lose value for all the bars. I have noticed that this also happens when i switch Database. For example lets say I was working on DB_Jan_March earlier and had created barscore for 80 stocks in timeframe 10 minutes and now i switch to DB_April_June and again create barscore in the same timeframe or some other timeframe and then later switch back to earlier DB (DB_Jan_March) i can see that many stocks have lost the barscore. I will reiterate that some stocks still hold the values but many have lost.
Below I am posting complete code that is runnable on copy paste.
_SECTION_BEGIN( "TestBarScoreNew" );
SetChartOptions( 0, chartShowArrows | chartShowDates | chartWrapTitle );
_N( Title = StrFormat( "{{NAME}} - {{INTERVAL}} {{DATE}} Open %g, Hi %g, Lo %g, Close %g (%.1f%%) {{VALUES}} ", O, H, L, C, SelectedValue( ROC( C, 1 ) ) ) );
PlotOHLC( O, H, L, C, "Price", colorRed, styleBar , Null, Null, 0, 1, 1 );
Plot( BarIndex(), "Barindex", colorwhite, styleNoLine + styleNoLabel + styleNoRescale );
Plot( BarCount, "BarCount", colorwhite, styleNoLine + styleNoLabel + styleNoRescale );
_SECTION_END();
SetBarsRequired( sbrAll, sbrAll );
_TRACE( "!CLEAR!" );
global runmode;
dist = Param( "distance", 50, 1, 500, 1 );
timeframeBaseInSeconds = Interval();
InstrumentNameAndTFtext = Name() + "_interval_" + timeframeBaseInSeconds + "_";
maxSearchDistanceForBarScore = Min( dist, 100 );
finalSyntheticPrice = ( H + L ) / 2;
minSearchDistanceForBarScore = 2;
mapKey = ( dist - minSearchDistanceForBarScore );
function selectRunMode()
{
defaultRunMode = 0;
global txtAlongTheTrend, txtCounterTrend, txtNetEffect;
txtViewOnly = "View Only";
txtProcessing = "Processing";
global runModeViewOnly, runModeProcessing ;
runModeViewOnly = 1;
runModeProcessing = 2;
global runMode;
usrInput = ParamList( "Mode", txtViewOnly + "|" + txtProcessing, defaultRunMode );
runMode = IIf( usrInput == txtViewOnly, runModeViewOnly, IIf( usrInput == txtProcessing, runModeProcessing, 0 ) ) ;
}
function getEffectiveStartBarIndx()
{
effStartBarIndx = Max( 0, BarCount - round( BarCount * timeframeBaseInSeconds / Max( timeframeBaseInSeconds, Interval() ) ) ) + 2;
//_TRACE("getEffectiveStartBarIndx() BarCount="+BarCount +", timeframeBaseInSeconds="+timeframeBaseInSeconds+", Interval()="+Interval()+", effStartBarIndx="+effStartBarIndx);
return( effStartBarIndx );
//return(10);
}
function calculateBarScore()
{
if( Status( "actionex" ) != actionExAAParameters && Status( "actionex" ) != actionPortfolio )
{
startTime = Now( 5 );
barHeight = H - L; //ATR( 5 );//
thisTimeframeinseconds = Interval();
thisTimeframeinMinutes = thisTimeframeinseconds / 60;
InstrumentNameAndTFtext = Name() + "_interval_" + thisTimeframeinseconds + "_";
sourceBarStartIndexBasedOnTimeframe = round( BarCount - Barcount / Max( 1, ( thisTimeframeinseconds / Max( 1, timeframeBaseInSeconds ) ) ) );
barScoreComputationCompleteTillDistance = Nz( staticVarGet( InstrumentNameAndTFtext + "barScoreComputationCompleteTillDistance" ) );_TRACE("barScoreComputationCompleteTillDistance="+barScoreComputationCompleteTillDistance+", isnull(barScoreComputationCompleteTillDistance)="+isnull(barScoreComputationCompleteTillDistance));
barScoreComputationCompleteTillDistanceInitial=barScoreComputationCompleteTillDistance;
maxSearchDistanceEff = Max( 0, Min( maxSearchDistanceForBarScore, BarCount - BarIndex() - minSearchDistanceForBarScore ) );
effStartBarIndex = getEffectiveStartBarIndx();
barScoreComputationIncompleteCondition = barScoreComputationCompleteTillDistance < maxSearchDistanceEff && ( BarCount - BarIndex() -1) > maxSearchDistanceEff && maxSearchDistanceEff > effStartBarIndex && BarIndex()>=effStartBarIndex && BarIndex()<BarCount-1;_TRACE("barScoreComputationIncompleteCondition="+barScoreComputationIncompleteCondition);
barScoreComputationIncompleteCondition[effStartBarIndex] = 1;
SourceBarCounterStartValue = ValueWhen( barScoreComputationIncompleteCondition, BarIndex(), -1 );
SourceBarCounterEndValue = ValueWhen( barScoreComputationIncompleteCondition && !Ref(barScoreComputationIncompleteCondition, 1), BarIndex(), -1);
SourceBarCounterStartValueScalar = SourceBarCounterStartValue[effStartBarIndex];
SourceBarCounterEndValueScalar = SourceBarCounterEndValue[effStartBarIndex];
_TRACE("1. SourceBarCounterStartValueScalar="+SourceBarCounterStartValueScalar+", isnull(SourceBarCounterStartValueScalar)="+isnull(SourceBarCounterStartValueScalar)+", SourceBarCounterEndValueScalar="+SourceBarCounterEndValueScalar+", isnull(SourceBarCounterEndValueScalar)="+isnull(SourceBarCounterEndValueScalar));
//_TRACE("barScoreComputationIncompleteCondition="+barScoreComputationIncompleteCondition+", isnull(barScoreComputationIncompleteCondition)="+isnull(barScoreComputationIncompleteCondition)+", effStartBarIndex="+effStartBarIndex+", maxSearchDistanceEff="+maxSearchDistanceEff+", maxSearchDistanceForBarScore="+maxSearchDistanceForBarScore+", barScoreComputationCompleteTillDistance="+barScoreComputationCompleteTillDistance);
if( IsNull( SourceBarCounterStartValueScalar ) || SourceBarCounterStartValueScalar == effStartBarIndex || SourceBarCounterStartValueScalar<=effStartBarIndex)
SourceBarCounterStartValueScalar = BarCount;
if( IsNull( SourceBarCounterEndValueScalar ) || SourceBarCounterEndValueScalar == effStartBarIndex ||SourceBarCounterEndValueScalar<SourceBarCounterStartValueScalar)
SourceBarCounterEndValueScalar = BarCount;
barScoreComputationIsAlreadyCompleteForThisTimeFrame = SourceBarCounterStartValueScalar == BarCount;
_TRACE("2. SourceBarCounterStartValueScalar="+SourceBarCounterStartValueScalar+", isnull(SourceBarCounterStartValueScalar)="+isnull(SourceBarCounterStartValueScalar)+", SourceBarCounterEndValueScalar="+SourceBarCounterEndValueScalar+", isnull(SourceBarCounterEndValueScalar)="+isnull(SourceBarCounterEndValueScalar)+", barScoreComputationIsAlreadyCompleteForThisTimeFrame="+barScoreComputationIsAlreadyCompleteForThisTimeFrame);
//_trace("BarCount="+BarCount+", barScoreComputationIsAlreadyCompleteForThisTimeFrame="+barScoreComputationIsAlreadyCompleteForThisTimeFrame);
//barScoreComputationIsAlreadyCompleteForThisTimeFrame=1;
if( barScoreComputationIsAlreadyCompleteForThisTimeFrame )
_trace( "barScore computation is ALREADY COMPLETE for all bars for instrument '" + Name() + "' upto SearchDistance of " + maxSearchDistanceForBarScore + " bars for interval " + thisTimeframeinMinutes + " minutes." );
//else
//_trace("SourceBarCounterStartValueScalar = "+SourceBarCounterStartValueScalar+", barScoreComputationIncompleteCondition="+barScoreComputationIncompleteCondition+", barScoreComputationCompleteTillDistance="+barScoreComputationCompleteTillDistance+", maxSearchDistanceEff="+maxSearchDistanceEff+", isnull(barScoreComputationCompleteTillDistance)="+isnull(barScoreComputationCompleteTillDistance));
else
{
for( i = SourceBarCounterStartValueScalar; i < SourceBarCounterEndValueScalar; i++ ) //source bar
{
if( IsNull( barScoreComputationCompleteTillDistance[i] ) )
serD = minSearchDistanceForBarScore;
else
serD = Max( minSearchDistanceForBarScore, barScoreComputationCompleteTillDistance[i] + 1 );
//_TRACE( "...calculating bar score for bar "+i +" for dist "+serD+ " and onwards.");
sourceBarHigh = high[ i ];
sourceBarLow = Low[ i ];
distCounterStartValue = Max( minSearchDistanceForBarScore, barScoreComputationCompleteTillDistance[i] + 1 );
for( dist = distCounterStartValue; dist <= maxSearchDistanceEff[i] ; dist++ ) //Search Distance aka destination bar distance
{
// the trendline is given by equation
// y = ax + b;
// where x would be (dist - x)
// b would be high[ i ]
// a would be (high[ dist ] - high[ i ] ) / (dist - i );
destBarIndex = i + dist;
destBarHigh = H[destBarIndex];
destBarLow = L[destBarIndex];
bsArraySuffixMappingToSearchDist = dist - minSearchDistanceForBarScore ;
scoreForThisiDistPairHH = 0;
intersectionReachedForThisSynthPrice = 0;
for( k = 1; k < dist; k++ ) //intervening price bar between source and destination; k is the distance between sorce and intervening bar
{
cleanlineHH = 0;
interveningBarIndex = i + k;
interveningBarHigh = H[interveningBarIndex];
aHH = ( destBarHigh - sourceBarHigh ) / dist;
lineHH = aHH * k + sourceBarHigh; //value of line at bar (i+k)
if( interveningBarHigh < lineHH && intersectionReachedForThisSynthPrice == 0 ) // line is clean
{
cleanlineHH = 1;
scoreForThisiDistPairHH = k;
}
else //intersection found
{
intersectionReachedForThisSynthPrice = 1;
if( scoreForThisiDistPairHH == 0 && k == ( dist - 1 ) )
breakLoopSHH = 1;
}
matrixIndxOffset = 1;
a = staticVarGet( InstrumentNameAndTFtext + "bsHH_" + Max( 0, bsArraySuffixMappingToSearchDist - matrixIndxOffset ) );
bs = Max( scoreForThisiDistPairHH, a[i] );
bsHH = staticVarGet( InstrumentNameAndTFtext + "bsHH_" + bsArraySuffixMappingToSearchDist ) ;
bsHH[i] = bs;
staticVarSet( InstrumentNameAndTFtext + "bsHH_" + bsArraySuffixMappingToSearchDist, bsHH, true );
//if( i == 38 && dist == 30)
//_TRACE( "i="+i+", dist="+dist+", s="+s+", k=" + k+ ", bsArraySuffixMappingToSearchDist=" + bsArraySuffixMappingToSearchDist + ", lineHH=" + lineHH + ", maxSearchDistanceEff=" + maxSearchDistanceEff[i] +", cleanlineHH="+cleanlineHH +", scoreForThisiDistPairHH="+scoreForThisiDistPairHH+", intersectionReachedForThisSynthPrice="+intersectionReachedForThisSynthPrice+", breakLoopSHH="+breakLoopSHH);
}//end of loop k
bsArraySuffixMappingToSearchDist = dist - minSearchDistanceForBarScore ;
scoreForThisiDistPairLL = 0;
intersectionReachedForThisSynthPrice = 0;
for( k = 1; k < dist; k++ ) //intervening price bar between source and destination; k is the distance between sorce and intervening bar
{
cleanlineLL = 0;
interveningBarIndex = i + k;
interveningBarLow = L[interveningBarIndex];
aLL = ( destBarLow - sourceBarLow ) / dist;
lineLL = aLL * k + sourceBarLow; //value of line at bar (i+k)
if( interveningBarLow > lineLL && intersectionReachedForThisSynthPrice == 0 ) // line is clean
{
cleanlineLL = 1;
scoreForThisiDistPairLL = k;
}
else //intersection found
{
intersectionReachedForThisSynthPrice = 1;
if( scoreForThisiDistPairLL == 0 && k == ( dist - 1 ) )
breakLoopSLL = 1;
}
matrixIndxOffset = 1;
a = staticVarGet( InstrumentNameAndTFtext + "bsLL_" + Max( 0, bsArraySuffixMappingToSearchDist - matrixIndxOffset ) );
bs = Max( scoreForThisiDistPairLL, a[i] );
bsLL = staticVarGet( InstrumentNameAndTFtext + "bsLL_" + bsArraySuffixMappingToSearchDist ) ;
bsLL[i] = bs;
staticVarSet( InstrumentNameAndTFtext + "bsLL_" + bsArraySuffixMappingToSearchDist, bsLL, true );
//if( i == 10 && dist == 30 )
//_TRACE( "i=" + i + ", dist=" + dist + ", k=" + k + ", s=" + s + ", bsArraySuffixMappingToSearchDist=" + bsArraySuffixMappingToSearchDist + ", lineLL=" + lineLL + ", maxSearchDistanceEff=" + maxSearchDistanceEff[i] + ", cleanlineLL=" + cleanlineLL + ", scoreForThisiDistPairLL=" + scoreForThisiDistPairLL + ", intersectionReachedForThisSynthPrice=" + intersectionReachedForThisSynthPrice + ", breakLoopSLL=" + breakLoopSLL );
if (k==dist-1)
{
barScoreComputationCompleteTillDistance = staticVarGet( InstrumentNameAndTFtext + "barScoreComputationCompleteTillDistance" );
barScoreComputationCompleteTillDistance[i] = dist;
staticVarSet( InstrumentNameAndTFtext + "barScoreComputationCompleteTillDistance", barScoreComputationCompleteTillDistance, True );
}
}//end of loop k
}// end of loop dist
}// end of loop i
}// end if
endTime = Now( 5 );
runTime = round( DateTimeDiff( endTime, startTime ) ) ;
if( !barScoreComputationIsAlreadyCompleteForThisTimeFrame )
_TRACE( "...calculateBarScore run completed for Instrument '" + Name() + "' for interval " + thisTimeframeinMinutes + " minutes for SearchDistance of " + Min( maxSearchDistanceForBarScore, BarCount ) + " bars for barcount "
+ ( BarCount - sourceBarStartIndexBasedOnTimeframe ) + " bars, in " + runTime + " seconds. Computation began from bar "+SourceBarCounterStartValueScalar+" . "
+writeif(barScoreComputationCompleteTillDistanceInitial>0, "barScore was already complete upto distance "+barScoreComputationCompleteTillDistanceInitial, ""));
}// end if
}//end of function
function getBarScore( arrayFwdSearchDistance, barScoreType )
{
timeframeinseconds = Interval();
InstrumentNameAndTFtext = Name() + "_interval_" + timeframeinseconds + "_";
retval = Null;
for( dist = minSearchDistanceForBarScore; dist < ( maxSearchDistanceForBarScore ); dist++ )
{
mapKey = ( dist - minSearchDistanceForBarScore );
retval = IIf( arrayFwdSearchDistance == dist, iif( barScoreType == constScoreTypeHH, staticVarGet( InstrumentNameAndTFtext + "bsHH_" + mapKey ), staticVarGet( InstrumentNameAndTFtext + "bsLL_" + mapKey ) ) , retval );
}
return retval;
}
function displayBarScore()
{
bsHH = Nz( staticVarGet( InstrumentNameAndTFtext + "bsHH_" + mapKey ) );
bsLL = Nz( staticVarGet( InstrumentNameAndTFtext + "bsLL_" + mapKey ) );
//_Trace( "using matrix getBarScore(40)="+bs );
ATR10 = IIf( Cum( 1 ) > 10, ATR( 10 ), ATR( 1 ) );
for( i = 0; i < BarCount; i++ )
{
PlotTextSetFont( "" + bsHH[i], "windings", 7, i , H[i] + Atr10[i] / 3, colorYellow );
PlotTextSetFont( "" + bsLL[i], "windings", 7, i , L[i] - Atr10[i] / 3, colorYellow );
}
}
selectRunMode();
if( runMode == runModeProcessing )
calculateBarScore();
else
_TRACE("in view mode");
displayBarScore();
Apply this on a chart of lets say 10 minutes. I have also provided 1 minute data for 100 odd stocks see below. Right click parameters select distance as 50 and mode as Processing. Hit Ok. This will start the processing. Once processing is complete it will say so in log window. Then go to parameters window again and switch to view only mode. The chart should look something like below.
The respective barscore is displayed on top and bottom of every bar.
Now switch to a different timeframe lets say 20 minutes. generate barscore and view using steps given above. And then switch back to 10 minutes. You will see that some stocks have lost their barscore. Why does this happen?
I am trying to troubleshoot the reason for missing values of static variables and seek your indulgence.
Thankyou.
Data