Greetings Amibroker community,
I'm entirely new to the Amibroker community but I've had some experience using the Metastock software. I like what this software has to offer over Metastock but I'm having difficulties setting up the databases with the data from Metastock Legacy (or Local) files. What I want to do is set up separate Amibroker databases for each of my Metastock legacy (or local) folders. Currently, these folders are organized as AMEX, NYSE, NASDAQ, CDN Equities, US ETFs, CDN ETFs, Indexes, Optionable US. It seems that every time I want to set up a new database it overwrites the data in the "markets", and "groups" areas of these databases. I'm basically trying to figure out the best way to manage these databases so that I can easily group them accordingly. I've tried looking throughout here and google as well but can't seem to find a solution. Any help is greatly appreciated.
Welcome, @Brossy.
I'm not particularly familiar with Metastock, but I have some Amibroker databases that use data from a data provider that only supplies them in Metastock format.
I would like to understand better if you want to have ALL your data in a single database and distinguish the assets based on their Market, typology, etc., or if you want to access your legacy "folders" in multiple AmibBroker databases.
In general, IMO, if the base currency is the same, the best choice in AmiBroker is to use a single database and then classify the assets using the Markets, Groups, and Watchlists.
In any case, if you have not already done it, I suggest reading this KB article that describe the manual procedure I used to classify assets in my "Metastock data" based Amibroker databases.
Hi Beppe,
Thank you for responding to me so quickly. I tried again today for a bit to set up the database structure. So there are two things at play here. First is that my "Legacy MetaStock" data structure for downloading Metastock data is currently set up so that I download the 8 folders of data (NYSE, AMEX, NASDAQ, CDN Equities, US ETFs, CDN ETFs, Optionable US, Indexes) so each folder has a separate MASTER or EMASTER or XMASTER file specific to each of those 8 folder structures. The other thing to note is that I have another converted set of data for each of those 8 folders for data that was converted to Heikin-Ashi values. Without getting into the details of why I want to load in the pre-calculated Heikin Ashi data, I basically end up with a second set of data. The way I've been trying to handle this so far has been to try and create two separate databases, one for the regular value set and the other for the Heikin Ashi value set.
So to trace through how I've been doing it so far, I started by reading in the first 8 sets of data by creating a database on a separate drive (E: drive for example) and read in the MASTER files for each of those using the Configure button. For EOD data I'm selecting about 5200 bars (20 years). I repeat this for the Heikin Ashi ones.
When I start to assign the groups using the "File" then "Import Metastock data" wizard, it seemed to have the correct number of files (ie. AMEX was 7209 symbols). but as I add other Groups for example, then after I download the data, the number of symbols in that previous folder decreases. (I guess it
I read the links you've sent and I believe I'm doing things correctly as was described in them but I think in the end it's made me even more confused.
I feel that since this is a very important topic for getting things set up correctly, I wish there were good video tutorials on how to set up this initial step. The ones that exist are of quite poor quality and don't address my specific needs. Anyways, I really appreciate your advice and thanks again for your assistance. If you have any other thoughts that can help me get up the proper data I'd be certainly all ears. I'll keep fiddling around with things to try to make it work.
Cheers,
Ryan (aka Brossy)
I guess let's assume I want to setup two databases. One for regular prices and a separate database for the Heikin Ashi values. I think I have the 2 databases set up properly though. They are reading in their data to their specific database, but being able to do the "assignments" using the assignments organize seems to be where it goes wonky.
I guess the first step i'd like to do is make those 8 mentioned folders as 8 entries within the Markets button of the symbols tab. If I can get those set up correctly without the data magically disappearing, then I'll consider that a win as I can likely build from there. I just don't understand why it reads the data in correctly and then for some reason it decides it wants to store a much fewer number of values.
@Brossy I would like to understand what you mean by "magically disappearing"!
If you click on the "All" item on the Symbols window, i.e.
are you able to identify any "missing" symbols or are all the imported ones still there?
If I click on the All button, no instruments appear at all anymore. Thanks again for your kind help.
The message is crystal clear - just follow instructions that you see on-screen.
You don't have DATA
You should NOT be using "Import Metastock data".
You should just ONCE configure database to read MS database directly as instructed in the manual:
http://www.amibroker.com/guide/h_extsources.html
That's what I meant by "magically disappearing" I had data, but then on next reload of the program, it's gone.
I'll try what you mentioned Thomasz and let you know how it goes. Thanks for your patience and help. Cheers.
Hi Tomasz, Thanks for your response. Still working through it here. One question is it possible to have multiple databases. In more concise terms, Is it possible to have 2 local databases and one online database. Basically, I want to be able to use local data if I'm away from an internet connection. The two databases would be for normal Metastock data (legacy "MASTER" data) and then I have a second dataset with data converted to Heikin Ashi values.
On another note, here's a screenshot that shows I'm getting the 14391 symbols into the "undefined" under "markets".
but if I hover over "All", It doesn't show anything there.
My apologies if this seems like a noob issue for you guys.
@Brossy, this is normal.
Hovering over "All" does not show any hint, but if you click on "All" (without any text in the search field above) you should see ALL the imported tickers in the symbol list below.
Ah ok, yes, thanks beppe for the clarification regarding that. Much appreciated.
So now I have all 14931 symbols but they are in the undefined block under the markets tab. How do I start putting these into their respective categories? I started renaming the different markets I want to follow as shown below:
@Brossy, you should use Symbol->Organize Assignments and reassign the newly imported symbols from the undefined market into your desired "markets," as indicated in the first link I posted - ideally doing it after each folder import to streamline the process.
In any case, you can always use this dialog to reassign tickers when you need to change the assignments and/or classify the tickers in different groups or watchlists using different criteria (keep in mind that a ticker can be assigned to multiple watchlists but it can belong only to a single market and to a single group ).
You can create import definition / import format file to assign multiple symbols to categories in one go.
Read here
In addition to format file create symbol file where you have columns for ticker name in one column and category name in other column(s). See example article of knowledge base.
Thanks for the advice, beppe. I'll try that out tomorrow......getting late here on Tokyo time.
Thanks for the message fxshrat, but my preference is to try to read in the data using the MetaStock importer. If I can't seem to get it sorted how I want it, I'll revert to this, but it's fewer steps on my data collection and storage requirements by not going with the ASCII importer for now.
@beppe @Tomasz @fxshrat
So one thing that is bugging the crap out of me is when I download the data, the number of securities downloaded is 26553 as depicted below:
Here is the converted lists. The ones ending in -HA are my pre-converted files to Heikin-Ashi values. Overall the number of data instruments are 26553 * 2 = 53106
But when I create the database in Amibroker, it only pulls in 14931 instruments. This is where I seem to get hung up. It seems it doesn't read in all of the data. I can confirm that there should be 26553 points from Metastock legacy values.
Certain folders such as the CDN ETFs folder have 0 instruments for some reason. Others have a considerably reduced amount of instruments. Some only a couple of missing instruments.
What could be causing this incomplete set of data showing up in the Amibroker databases that I'm creating?
Here's the value shown in the undefined area of the Heikin Ashi database.
It's got me stumped.
That is 3rd party downloader problem.
Most likely your Metastock data has DUPLICATES and this 3rd party downlaoder counts FILES not unique symbols. For example you have folders from Nasdaq, Nasdaq 100 and Nasdaq Comp that all will have duplicates across them AmiBroker on the other hand would display only UNIQUE symbols, because symbol in AmiBroker is primary unique key.
We do NOT SUPPORT 3rd party downloaders on this forum.
Instead of using 3rd party downloaders causing trouble and MS format that is incapable of storing more than 65K bars, you should be using AmiQuote
AmiQuote works out of the box, and provides automatic categorisation and everything just works.
The procedure is described in detail here:
http://www.amibroker.com/docs/ab305.html
On second look, I might consider the ASCII importer for the Metastock data as well. I'll review some other options as well such as Barcharts data and also since I'm moving to Interactive Brokers, their service may be all I need. So much data to sift through in this switch over from Metastock. Thanks again for all the guidance i'm receiving from this community.
@beppe @Tomasz @fxshrat
Thanks, Gents!! I think with the historical data provided through the AmiQuote program (using Yahoo, Quandl, Tiingo and Barcharts) I'll be able to do what I want to do with EOD data with AmiBroker. Realtime (or near tick realtime) through IB should be good enough for my live data purposes. Thanks all for your patience and help with this Database Noob !! Cheers.
Brossy