Aligning a custom array's index to symbol's index

Just wondering is there any function or code available that aligns an array (created from external Text file content) to the symbol’s index?

Note: this array can be a subset or superset in terms of date of the Symbol’s date. So I want to align the custom array’s index to that of Symbol’s Index.

You can use “Pad and align to reference symbol” in the Settings.

@Tomasz

When I said custom array, I meant not the Quotation data but rather an array that has data like “number of times the company was mentioned in twitter for a particular date”, I am importing it from a text file, which has two colums.

Date(yyyy-mm-dd format), mention count
2017-01-01,3
2017-01-02,5
2017-01-10,8
2017-02-01,7
etc…

“Pad and align to reference symbol” is useful to align the Quotation data, but this custom array is imported from txt file on the fly.

Initially you said reverse. It is important to make your points clear.

Also - do not read text files on the fly. It is BAD idea to re-read, parse, process the data over and over from text file during each execution of the formula. It slows down performance. Text files are slow. Did I say they are slow? They ARE. Really SLOW.

You should import the data. That way they will be sitting in RAM ready for hyper-quick access.

You should either import that data into Aux fields (already instructed on the forum use search)
or you can import that data into separate ticker. If it is in separate ticker, the synchronization
is automatic and done on the fly when you just use Foreign() function (click on link and make sure to read whole docs INCLUDING comment)

1 Like

There is another way of achieving it.
For example if it is a multi column file (for example this source http://www.stockpup.com/data/ offers CSVs with 30 columns of historical fundamental data) then I use InternetOpenURL() to access the data and store it to a matrix. I do store that matrix once to static variable (it is updated if static var is empty or new column is selected or new symbol is selected or some other custom trigger in order to save resources).
The date column of the matrix is compared with the symbol's date array.
The advantage of storing such multi column data to matrix is that you can quickly slide through each available matrix column.

10 Likes

@fxshrat Very creative implementation.

Now I feel more empowered to think effectively on my design.

I do store that matrix once to static variable (it is updated if static var is empty or new column is selected or new symbol is selected or some other custom trigger in order to save resources).
The date column of the matrix is compared with the symbol’s date array.

If it’s not too much to ask, can you share the code for the Quoted description above? It will be a good start for me to code my design. Which is very much in line with what you showcased.

I believe this is very creative way of importing on the fly datapoints which are temporal in nature and when user is comfortable to forego performance over the power of overlaying much needed and relevant datapoint and gain insight with respect to price actions and volume.

Accessing Internet over and over to get the same data is even worse idea as Internet access is another few orders of magnitude slower than local file access.

A general principle of efficient coding is: never do the same thing twice (1)

The data you are using change only ONCE per day. And this means you should access internet only ONCE per day, not any single time more. Proper way is to: access internet once and STORE the data locally. Best way is to store in AB native database since it is fastest and provides auto-alignment so there is literally no problem. Imports from internet can be automated too. It has been covered in many places. Even “Update US stocks and categories” tool uses automated download and imports from web.

(1) unless doing things twice is faster than checking/conditional execution. Anecdotally, when you write in assembler sometimes it is faster to repeat calculation than to do conditional jump as jumps cause pipeline stall. So if your check is costly in terms of time doing thing again may be quicker. As always the devil is in the detail and you should always measure the difference

1 Like

@Neil81, being difficult to share since it is partly DLL functions.

@Tomasz, I’m not sure who are referring to but if it is me then all I can say is yes, that’s what I have written too.

I’m quoting myself

So I’m not sure whether I’m speaking extra-terrestrial language but upper quote clearly hints to Internet access once per day since it is just quarterly data example on top of that.

staticurl = StaticVarGetText( "InternetMat_Fundamentals_URL_" + Now(1) );
if( typeof( staticmat ) != "matrix" || url != staticurl ) {
   ....
   // Internet access to store data to mat and mat stored to static var.

   // set static mat
   // set static url   (url containing "+ Name() +" )
   .... 
} ....

So it accesses Internet if staticvar matrix is empty (-> staticmat var is of type “number” then) or if in case AB is kept open for more than one day then update is next day once again since Now(1) refers to current date string.

So access to Internet is once (each day) in upper example case!

So honestly I don’t see how it is [quote=“Tomasz, post:7, topic:867”]
…even worse idea…
[/quote]

Note: my example referred to multiple columns of data (…10…30…50). I can’t imagine a quicker way for such many columns of data (not saying there is none). But just take a look at the GIF picture. Access is immediately. Yes, it also depends of number of rows so (even if it is just once per day) the approach is clearly not recommended to access a table i.e. of tens of thousands of rows via InternetOpenURL. But in the mentioned sample case it is just some 70 rows or something (per symbol)…

1 Like

@fxshrat - in case you did not notice I was responding to original poster, not you. Specifically to this sentence

comfortable to forego performance

Which was inspired by the idea of using direct internet access.

Not every response on this forum is directed towards you. Just the opposite. There is simple rule, unless otherwise noted, the answer applies to post directly preceding. If I was answering to you I would mention your name.

The curse of AmiBroker is that it is TOO powerful for average user. It provides many tools but in the hand of irresponsible they become weapons that can hurt user him/herself and in the end me
because once irresponsible/inefficient coding is spread people blame me, not ignorant user, when they suffer from bad performance.

3 Likes

No problem. All fine. Just wasn’t sure who it was directed to because of
"The data you are using change only ONCE per day."

Yep, that’s certainly true in several cases. I can only imagine what your are going through via support channel (EVERY day). It might be even worse than I can imagine. :sob:

1 Like

Hi @fxshrat, nice example on using matrix.

You mentioned that you compare matrix date with symbol date.
From what I understood, matrix has less data points when compared to symbol quotes.

Could you please explain in a bit more detail how you align matrix to symbol array, in order to plot it?

thanks in advance,
pm