AmiQuote 3.15 is coming

As you know AmiQuote 3.14 was an “emergency” release issued quickly to address Yahoo changes.

But there are other changes along the way, namely:

These two will be addressed in AmiQuote 3.15 hopefully coming next week.

Hi Tomasz,

First of all would like to say I very much appreciate your very quick response to address the Yahoo change issues, you’ve really gone out of your way to solve this problem for your customers, and I really can’t ask for more for your continuous work on updating and improving Amibroker and Amiquote.

As I see this post on Amiquote 3.15, I have a suggestion which I hope you can consider as you make changes to it. Quandl has actually many different datasets, for example “ICE/xxx” for ICE data, “TSE/xxx” for Tokyo stocks etc, is it possible that you create the flexibility in Amiquote for users to be able to download these other datasets rather than just US stocks? It will make this download option a lot more useful by tapping on the breadth of Quandl’s data and also increase the power of Amiquote. Hope it is something you can consider.

Thank you

Yes I hope to include that, but problematic thing is when they generate different layout of fields (not Date/O/H/L/C/V) because then you need separate import format for each.

Thank you for the words of appreciation. I don’t know what happened to people since 2000 but apparently only 1 person per 100 sends something else than complaints and my batteries are running out.

Please don’t be discouraged Tomasz. I’m sure there are a lot of folks who (like me) are generally quite silent but really appreciate the work you do. I always recommend Amibroker to my friends and a large part of it is because of the continuous improvements and efforts that you keep putting in over the years. Thank you.

Agree with THT, I think people do see the passion behind all these works. The quality of the platform stands out.

If Amiquote capability can be expanded, the target market can be expanded too. Students, economist, researchers round the world.

The more widely amibroker is used the more the AFL community and of course you Tomasz will benefit.

Jimmy

Thank you for adding support for Quandl's EOD data to AmiQuote. Very much appreciated.

TIA: The EOD data supports both unadjusted as well as dividend adjusted data. See the attached screenshot for SPY.

Hello Tomasz,
I agree with THT.
I think a lot of people only speak when there is something they would like to improve. Usually when everything is ok most of the users don’t express it. Part of it it’s human nature… I think.
From my part, I would also like to say thanks for building such a great software, with well thought design and trying to listen to customers as much as possible.
I don’t have a programming background and Amibroker has helped me immensely throughout the years.
Thanks and keep up the good work!

Is Finam still a working source ? If not, I would like to suggest removing it from the drop down selection. Would 3.15 also be able to take care of the change in yahoo's data for certain tickers that changed the columns ? as reported by fxshrat ? TIA

AmiQuote just downloads what is available on Yahoo web site. It does not re-arrange columns for "certain tickers"
because it does not know what “certain tickers” are. If Yahoo has a problem on their server with column ordering
for “certain tickers” only then you should report that to Yahoo.

Hi Tomasz,
I did install Amiquote 3.15. There is no problem in downloading Yahoo Current data . However, when I download Yahoo historical data, it shows “Give up. Check if you have IE 10 or higher”. Actually, my IE is 11 and should not have any problem.

Pls. help.

Brian

Apparently it is outdated anyway or otherwise broken. Some people had IE disabled or otherwise made inactive.

Open Internet Explorer manually and go to Download Data link and see if that works

I’ve checked the download of Yahoo histical data by following way. https://finance.yahoo.com/quote/0001.HK/history?p=0001.HK
It did operate properly.

Is there any problem on my amiquote 3.15. How can I remove & re-install it.

Many thanks in advance.

Brian

It simply works. On thousands of computers.

Reinstalling does not do anything.

If it doesn't - it is your computer / OS (Windows) / lack of current Internet Explorer.

  1. Enable Cookies in Internet Explorer
  2. Perform Windows Update
  3. Update Internet Explorer to most recent version (at least 10 but higher is better)
  4. Disable antivirus
  5. Try different computer

The page alone isn't what you should check. Did you open that with Internet Explorer (NOT any other browser). Did you click on DOWNLOAD DATA link on that page as presented on the SCREENSHOT and did you open that in Excel?

As to why Internet Explorer, read this:
http://ask.amibroker.org/index.php?qa=155&qa_1=i-use-chrome-firefox-on-windows-why-do-i-need-ie10-or-higher

There is one extra hint that has been discovered recently:

If you are using “Server” edition of Windows (like Windows 2012 Server or Windows 2008 Server or Windows 2016 Server), then “Give up. Check if you have IE 10 or higher” message usually means that your default security configuration in the IE is to “deny” 3rd party web sites.

The Server version of Windows and IE that comes with it has security settings that prevent
from access certain content (like JavaScript and 3rd party sites).
Desktop versions have “allow” policy, while Server versions have default “deny” policy.

You would need to check if you have added finance.yahoo.com in “trusted” sites and allowed
JScript.

Hi to all,

has anyone observed the same issue with Yahoo data in Amiquote 3.15 (see below). Seems to be new issue again (recognized with ETF e.g. EWJ):

As soon as dividend is paid Close (seems to be adjusted now) may be lower now than the unadjusted Low:
although I use $FORMAT Date_DMY,Open,High,Low,Skip,Close,Volume

If I use Adj. Close (old format style) that seems to work again (so Open, High, Low, Close is adjusted) but there seems to be issues with splits in the past (so split does not work and the chart looks weird again):
$FORMAT Date_DMY,Open,High,Low,Close,AdjClose,Volume

No improvment if I try with $RECALCSPLITS 1

So no adjustment delivers correct chart, but it looks like as there could be a solution. Any idea?

Many thanks

dubi

Hi All -
Since yesterday I am also getting same issue with Yahoo data in Amiquote 3.15. I can see charts properly for indexes like ^IXIC and ^GSPC. On other hand, for ETFs like SPY, IWM etc, the chart is as shown in the thread above.

I downloaded the historical data from Yahoo manually and the data is correct. On other hand when I run Amibroker database purify tool, it says the low is greater than high.

The above issue seems to happen only with historical Yahoo data. Other data sources (like Google etc) seem to be fine. I am not sure it is problem with Yahoo data.

Any suggestions?

Note: I reinstalled with latest amibroker and amiquote this morning hoping problem will resolve. But it did not. So one should be able to reproduce the problem fairly quickly.

Regards,
Durga

Hi Durga,

yes, I made the same observation. Historical data downlad from Yahoo directly/manually is correct, but somehow Amiquote is mixing up the data. For e.g. XLF or VXX

  1. $FORMAT Date_DMY,Open,High,Low,Skip,Close,Volume delivers:
    unadjusted open, high, low BUT adjusted Close (problem: historical close in Amibroker is lower than low), splits seems to be correct

  2. $FORMAT Date_DMY,Open,High,Low,Close,AdjClose,Volume delivers:
    adjusted open, high, low BUT adjusted on adjusted Close.… So adjusted on adjusted close delivers again wrong data, historical splits are wrong now.

Please Tomasz, if possible can you take a look at it.

Many thanks.

dubi

AmiQuote does NOT change the data in ANY way.

AmiQuote is the web browser/downloader, like Firefox, Internet Explorer, Opera, Chrome. It does not modify the data. What you get from AmIQuote is exactly the same what you would get by clicking on “Download data” button on Yahoo historical prices page.

The thing is Yahoo is messing things up constantly changing column order and meaning every few days. You should complain to YAHOO.

To properly import data, import definition must MATCH the column order and column MEANING. If Yahoo changes the order or meaning of columns the data may/will be interpreted incorrectly unless you adjust the format file (aqh.format)

This is third time when they change the meaning and/or order of columns. What is worse the heading they are sending is incorrect to. They as sending header like this

# Date,Open,High,Low,Close,Adj Close,Volume

But the data actually (for VXX) are:
02-08-2010, 5629.439941, 5657.600098, 5432.319824, 21.230000, 5434.879883, 74500

So as you can see the columns are really

Date, AdjOpen, AdjHigh, AdjLow, RawClose, AdjClose, Volume

Currently format definition file (.aqh) should look like this:

# AmiQuote historical quotes download format (.AQH extension)
# Revision 1.3 - changed the order of columns (due to May 17, 2017 Yahoo site change)
# Revision 1.1 - removed $VOLFACTOR 0.01
$FORMAT Date_DMY,Open,High,Low,Skip,Close,Volume
$SKIPLINES 0
$BREAKONERR 0
$SEPARATOR ,
$DEBUG 1
$AUTOADD 1
$CONT 1
$GROUP 254
# the following switches are optional - please consult
# the read-me for description of those options
# $RAWCLOSE2OI 1   
# $RECALCSPLITS 1
# $ROUNDADJ 4

As you can see we need to SKIP RawClose column.

And that is the format that is already included in the setup program of AmiBroker 6.00 and 6.20 since June 2017 and in the setup of AmiQuote 3.15.

Trouble is when you download some data that is not only adjusted for Splits but adjusted for Dividends.

In the period May 17 - June 10 they were NOT adjusting for dividends so OHLC were adjusted for splits only.

Then people sent complaints and “geniuses” @Yahoo started adjusting for Dividends, but… only Close price, leaving High, Low, Open adjusted for Splits only.

Here is example of EWJ data downloaded from Yahoo:

02-08-2010, 39.000000, 39.360001, 38.880001, 9.840000, 35.239758, 4488800

The columns are as follows:
Date, AdjOpen, AdjHigh, AdjLow, RawClose, AdjCloseForSplitsAndDividends, Volume

So here is where the mess is coming from. Yahoo is adjusting High, Low, Open fields for splits but at the same time they are adjusting Close field for splits and dividends.

This needs to be fixed by Yahoo on their site. You need to complain to Yahoo to fix their mess.

There is already a thread that describes this data error on Yahoo web site https://forums.yahoo.net/t5/Yahoo-Finance-help/Historical-Data-Split-and-Dividend-Adjustment-Errors-in-EEM/td-p/279800 but apparently Yahoo does not see / understand that.

Hi Tomasz,

many thanks for your quick response. I have the same format definition, as you posted.
Seems that the problem, as you mentioned, is because of Yahoo mixing/changing up the columns and the date. Seems historically neither close nor adjusted close delivers correct data.

AmiBroker Result with exactly same format definition

AmiBroker Result but with $FORMAT Date_DMY,Open,High,Low,Close,AdjClose,Volume
(look at open, high, low .... this is adjusted)

This is Yahoo data (mixed up adj close):

Here Reuters data to compare:

Many thanks, dub

You are comparing apples to oranges. Reuters data is not adjusted for dividends.

The aqh.format file shipped with AMiBroker is correct and you should not change it.

It is just Yahoo that needs to fix their output. They need to adjust Open, High, Low fields the same way as they are adjusting Close.
Or…
Yahoo needs to send RawOpen, RawHigh, RawLow instead of adjusted ones. Then adjusted OHL can be calculated on the fly by deriving split ratio from AdjClose/Close.