New to Amibroker and looking for guidance on setting up database (s). Is there any reason not to have just one database that has all NYSE , NASDAQ and AMEX stocks? I’ve read everything I could find. The only thing I saw was someone mentioned best to have 2 databases where one loads 15 to 20 years of data suitable for backtesting and another database (same stocks) that has much reduce number of days say 1000 days for explorations, scanning and just going through your charts. Reason given was speed improvement. Not sure how true that is. Interested in EOD data only at this point. I will only load price data for just a subset of all symbols in NYSE,NASDAQ and AMEX, say only about 2000 symbols total.
Also, I was going to click on the “ Tools->Update US symbol list and categories menu, but hesitated so far since the caveat about “wiping” out existing sectors/industries made me pause since i’ve already loaded 400 symbols and didn’t want to risk losing price data. I’m OK if it updates the sector/industry only and price data and other “fundamental extra new” information remains as is. Any guidance on best approach for databases would be appreciated. I did think about one database for stocks and a different one for ETFs, but not sure that even buys me something since I would have a separate watchlist for ETFs anyway.
Maybe you want to share your hardware Specs like CPU,RAM,SSD etc because your question depends quite a lot on hardware. Then someone operating at that scale can tell you better.
and the type of Data in number of bars, a Daily bar is one bar per day. if you are fetching bars in minutes and then thinking of using in Daily TF will also consume more resources.
And whether data is from plugin or Local DB.
Search for more threads like the ones below.
Even the computational complexity of the Formula being run (AFL) would vary Load on CPU... a lot factors here
If you're just starting out, don't worry too much about performance, learn the features of AmiBroker on a few symbols, learn AFL, do some explorations.
By the time you start hitting bottlenecks, you'll have a better idea what dataset you want to run your expensive formulas over and you can always create new dbs specifically for those runs.
I'm running fairly time consuming AFL over 1m interval DOW from the start of the year, that's about 120k bars, equiv to 460 years of eod candles or 200 symbols, on one db.
For real-time calculations, then you have to start thinking about visible bars, using static variables to store expensive calculations.
My specs are similar, double your ram but AmiBroker is not a memory hog nor a cpu hog so you should be fine for the first few months at least.
If you're not sure what this does, start a new DB and do it there, if you're happy with the results just go back to your default db and perform the action again. I use AmiQuote to update my eod stock data so can't comment on that feature specifically.
The message you get is quite clear about what it does, it wipes sector/industry assignments and adds symbols from US exchanges. It does NOT change your existing price data. But as @mordrax correctly wrote - you can try it on new empty database if you are unsure.
And as @mordrax wrote - speed is not a problem for EOD databases and 20 years of EOD is nothing.
If you don’t mind me asking, what data source do you use? I use AmiQuote as well for EOD, and have tried Yahoo, Tiingo and Stooq. Also, used the Yahoo Fundamental Extra New but did not get the company full name and trying to get the correct sector/industry for each symbol.
New to Amibroker, but been trading for years and have written code in other systems. AFL similar to what I’m use to and I really like the flexibility, backtesting and exploration features of Amibroker!
I'm using AmiQuote for getting EOD yahoo data, not using that atm for anything. Happy to pay for Norgate if I end up developing a system with an edge...
I'm using IB for getting intraday data, indexes and commodities and trying to develop some basic system to guide my discretionary swing trades.
AmiBroker integrates directly with IB and I can get realtime feedback on my indicators which is a nice to have. It will also save that data to the DB which is good because otherwise you can only go back 180 days.
With IB, minute data, I've got DOW from feb this year, that's about 120k bars. My database setting is set to 1 million bars or ~8 years of minute data per symbol. It takes AB a few seconds to load... not sure why and not sure if this will degrade linearly which may become a problem with large datasets. I'm currently tracking 15 symbols over 180 days mostly. So about 1m bars.
I'm also fairly new to AB. I think if you ask well thought out questions, you'll get some great insights from the highly experienced users who frequent the site.
Welcome to the forums!!
Over the weekend, I did create a US stock database and did the Tools->Update US symbol list and categories. I then did the Yahoo Fundamental Extra New download. About 20% failed - of those about 30% from the symbol not having any fundamental data (not surprised) but the other 70% (0f the 20%) or a little over 1,000 symbols failed because the symbol was not found. So, my question - is the symbol list when you do the “Update US symbol list” current or is it a symbol list from when the feature was added?
I also using the ASCII importer added a $Market 4, NYSE ARCA and added about 500 ETFs with ticker,full name into the new database. Used a format file. Worked very well and pretty straightforward. I omitted ETFs trading less than 100,000 shares a day.