AmiQuote 3.15 is coming

Tomasz · June 8, 2017, 3:15pm

It simply works. On thousands of computers.

Reinstalling does not do anything.

If it doesn't - it is your computer / OS (Windows) / lack of current Internet Explorer.

Enable Cookies in Internet Explorer
Perform Windows Update
Update Internet Explorer to most recent version (at least 10 but higher is better)
Disable antivirus
Try different computer

The page alone isn't what you should check. Did you open that with Internet Explorer (NOT any other browser). Did you click on DOWNLOAD DATA link on that page as presented on the SCREENSHOT and did you open that in Excel?

As to why Internet Explorer, read this:
http://ask.amibroker.org/index.php?qa=155&qa_1=i-use-chrome-firefox-on-windows-why-do-i-need-ie10-or-higher

Tomasz · June 18, 2017, 8:30am

There is one extra hint that has been discovered recently:

If you are using “Server” edition of Windows (like Windows 2012 Server or Windows 2008 Server or Windows 2016 Server), then “Give up. Check if you have IE 10 or higher” message usually means that your default security configuration in the IE is to “deny” 3rd party web sites.

The Server version of Windows and IE that comes with it has security settings that prevent
from access certain content (like JavaScript and 3rd party sites).
Desktop versions have “allow” policy, while Server versions have default “deny” policy.

You would need to check if you have added finance.yahoo.com in “trusted” sites and allowed
JScript.

dubi1974 · June 28, 2017, 8:41pm

Hi to all,

has anyone observed the same issue with Yahoo data in Amiquote 3.15 (see below). Seems to be new issue again (recognized with ETF e.g. EWJ):

As soon as dividend is paid Close (seems to be adjusted now) may be lower now than the unadjusted Low:
although I use $FORMAT Date_DMY,Open,High,Low,Skip,Close,Volume

If I use Adj. Close (old format style) that seems to work again (so Open, High, Low, Close is adjusted) but there seems to be issues with splits in the past (so split does not work and the chart looks weird again):
$FORMAT Date_DMY,Open,High,Low,Close,AdjClose,Volume

No improvment if I try with $RECALCSPLITS 1

So no adjustment delivers correct chart, but it looks like as there could be a solution. Any idea?

Many thanks

dubi

durga_public · June 28, 2017, 9:47pm

Hi All -
Since yesterday I am also getting same issue with Yahoo data in Amiquote 3.15. I can see charts properly for indexes like ^IXIC and ^GSPC. On other hand, for ETFs like SPY, IWM etc, the chart is as shown in the thread above.

I downloaded the historical data from Yahoo manually and the data is correct. On other hand when I run Amibroker database purify tool, it says the low is greater than high.

The above issue seems to happen only with historical Yahoo data. Other data sources (like Google etc) seem to be fine. I am not sure it is problem with Yahoo data.

Any suggestions?

Note: I reinstalled with latest amibroker and amiquote this morning hoping problem will resolve. But it did not. So one should be able to reproduce the problem fairly quickly.

Regards,
Durga

dubi1974 · June 30, 2017, 8:48am

Hi Durga,

yes, I made the same observation. Historical data downlad from Yahoo directly/manually is correct, but somehow Amiquote is mixing up the data. For e.g. XLF or VXX

$FORMAT Date_DMY,Open,High,Low,Skip,Close,Volume delivers:
unadjusted open, high, low BUT adjusted Close (problem: historical close in Amibroker is lower than low), splits seems to be correct
$FORMAT Date_DMY,Open,High,Low,Close,AdjClose,Volume delivers:
adjusted open, high, low BUT adjusted on adjusted Close.… So adjusted on adjusted close delivers again wrong data, historical splits are wrong now.

Please Tomasz, if possible can you take a look at it.

Many thanks.

dubi

Tomasz · June 30, 2017, 9:19am

AmiQuote does NOT change the data in ANY way.

AmiQuote is the web browser/downloader, like Firefox, Internet Explorer, Opera, Chrome. It does not modify the data. What you get from AmIQuote is exactly the same what you would get by clicking on “Download data” button on Yahoo historical prices page.

The thing is Yahoo is messing things up constantly changing column order and meaning every few days. You should complain to YAHOO.

To properly import data, import definition must MATCH the column order and column MEANING. If Yahoo changes the order or meaning of columns the data may/will be interpreted incorrectly unless you adjust the format file (aqh.format)

This is third time when they change the meaning and/or order of columns. What is worse the heading they are sending is incorrect to. They as sending header like this

# Date,Open,High,Low,Close,Adj Close,Volume

But the data actually (for VXX) are:
02-08-2010, 5629.439941, 5657.600098, 5432.319824, 21.230000, 5434.879883, 74500

So as you can see the columns are really

Date, AdjOpen, AdjHigh, AdjLow, RawClose, AdjClose, Volume

Currently format definition file (.aqh) should look like this:

# AmiQuote historical quotes download format (.AQH extension)
# Revision 1.3 - changed the order of columns (due to May 17, 2017 Yahoo site change)
# Revision 1.1 - removed $VOLFACTOR 0.01
$FORMAT Date_DMY,Open,High,Low,Skip,Close,Volume
$SKIPLINES 0
$BREAKONERR 0
$SEPARATOR ,
$DEBUG 1
$AUTOADD 1
$CONT 1
$GROUP 254
# the following switches are optional - please consult
# the read-me for description of those options
# $RAWCLOSE2OI 1   
# $RECALCSPLITS 1
# $ROUNDADJ 4

As you can see we need to SKIP RawClose column.

And that is the format that is already included in the setup program of AmiBroker 6.00 and 6.20 since June 2017 and in the setup of AmiQuote 3.15.

Trouble is when you download some data that is not only adjusted for Splits but adjusted for Dividends.

In the period May 17 - June 10 they were NOT adjusting for dividends so OHLC were adjusted for splits only.

Then people sent complaints and “geniuses” @Yahoo started adjusting for Dividends, but… only Close price, leaving High, Low, Open adjusted for Splits only.

Here is example of EWJ data downloaded from Yahoo:

02-08-2010, 39.000000, 39.360001, 38.880001, 9.840000, 35.239758, 4488800

The columns are as follows:
Date, AdjOpen, AdjHigh, AdjLow, RawClose, AdjCloseForSplitsAndDividends, Volume

So here is where the mess is coming from. Yahoo is adjusting High, Low, Open fields for splits but at the same time they are adjusting Close field for splits and dividends.

This needs to be fixed by Yahoo on their site. You need to complain to Yahoo to fix their mess.

There is already a thread that describes this data error on Yahoo web site https://forums.yahoo.net/t5/Yahoo-Finance-help/Historical-Data-Split-and-Dividend-Adjustment-Errors-in-EEM/td-p/279800 but apparently Yahoo does not see / understand that.

dubi1974 · June 30, 2017, 9:54am

Hi Tomasz,

many thanks for your quick response. I have the same format definition, as you posted.
Seems that the problem, as you mentioned, is because of Yahoo mixing/changing up the columns and the date. Seems historically neither close nor adjusted close delivers correct data.

AmiBroker Result with exactly same format definition

AmiBroker Result but with $FORMAT Date_DMY,Open,High,Low,Close,AdjClose,Volume
(look at open, high, low .... this is adjusted)

This is Yahoo data (mixed up adj close):

Here Reuters data to compare:

Many thanks, dub

Tomasz · June 30, 2017, 10:00am

You are comparing apples to oranges. Reuters data is not adjusted for dividends.

The aqh.format file shipped with AMiBroker is correct and you should not change it.

It is just Yahoo that needs to fix their output. They need to adjust Open, High, Low fields the same way as they are adjusting Close.
Or…
Yahoo needs to send RawOpen, RawHigh, RawLow instead of adjusted ones. Then adjusted OHL can be calculated on the fly by deriving split ratio from AdjClose/Close.

dubi1974 · June 30, 2017, 10:01am

As Reuters has just adjusted the split on Sept 15, 2016: here Bloomberg with their adjusted historical data

durga_public · June 30, 2017, 4:50pm

@Tomasz -
Thanks. Currently I manually download the historical data as CSV and then import into Amibroker database. Is there anyway to inject a custom script (or) commands into AmiQuote to apply as part of data download from yahoo and import to amibroker?

Also I am not clear why the above i.e., incorporating dividends vs not incorporating would cause amibroker to think lows are higher than close in historical data when I download with amiquote. If I do manual upload then it is fine. I could be missing something. Anyway that was my experience when I did with vanguard funds and etfs like SPY, EEM etc.

Regards,
Durga

Tomasz · June 30, 2017, 10:20pm

You're wrong. AmiQuote downloads THE SAME as "manual" click on "Download Data" link. Re-read my posts PRECISELY. Apparently you did not understand a bit.

Tomasz · July 1, 2017, 12:46pm

Another example - this time straight from Yahoo web site.

Link:
https://finance.yahoo.com/quote/IP/history?period1=804549600&period2=807055200&interval=1d&filter=history&frequency=1d

What you get:

Clearly visible Adjusted Close is lower than Adjusted Low, because they subtract dividends from Close, but don't do that for High, Low, Open prices.

durga_public · July 2, 2017, 4:11am

@Tomasz -
Thanks. I see the problem. I doubt my complaints to Yahoo would make much difference

Tomasz · July 2, 2017, 7:37am

In my opinion they are unaware that their data in this shape are unusable, so please do complain to make them aware.

Tomasz · July 2, 2017, 9:26am

For what it is worth: new AmiQuote 3.16 will offer more alternative data sources

bistrader · July 5, 2017, 2:30pm

Hello,

What is best way to complain to Yahoo Finance about problem with data?

Moderator comment: links to Yahoo site for complaints are already in the post above

Tomasz · July 5, 2017, 7:41pm

As AmiQuote 3.20 is out, v3.15 is not longer relevant.

A post was merged into an existing topic: AmiQuote 3.20 is released now

Tomasz · July 5, 2017, 7:39pm

Tomasz · July 25, 2017, 10:12am

Tomasz · July 25, 2017, 10:12am